Application construction method, system, device and storage medium based on virtual data warehouse

Abstract

The present application provides an application construction method, system, device and storage medium based on a virtual data warehouse. The method obtains multiple logical components in a preset component library and multiple metadata in a preset knowledge base, constructs a directed acyclic application execution graph according to the multiple metadata and multiple logical components, configures the execution parameters of the first target component, and configures the output component of the output node of the directed acyclic application execution graph, configures the execution parameters of the second target component, and configures the parameter input component of the execution parameters of the second target component. The target application is constructed based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component and the parameter input component. Through metadata standardization and application componentization, the application construction is separated from the actual data resources, so that the application can switch smoothly under different data resources, which can effectively reduce the application development cost and maintenance fee.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system, device, and storage medium for constructing an application based on a virtual data warehouse. Background Art

[0002] With the development of social economy, various enterprises / institutions / organizations have accumulated a large amount of various business data, which are distributed across different platforms, systems and applications. How to effectively integrate this data and make it available to decision makers and analysts of enterprises / institutions / organizations has become a difficulty for big data analysis tools.

[0003] A virtual data warehouse (VDW) is a fast, low-cost solution for heterogeneous data integration. Although VDW completes data integration and provides a unified external data view, it is powerless in online analytical processing (OLAP).

[0004] Currently, the common practice for application implementation is to use data extraction, transformation, and load (ETL) tools to periodically extract or synchronize heterogeneous data in real time to a unified OLAP database, and then rely on a specific OLAP database or big data computing engine to perform specific business analysis. This is undoubtedly very costly, requiring at least double the amount of existing storage facilities in the enterprise / institution / organization, and relying on specific projects for development and implementation from data to application. This leads to high application development costs and a high threshold for development. Summary of the Invention

[0005] In view of the shortcomings of the existing technology described above, the purpose of the present invention is to provide an application construction method, system, device and storage medium based on a virtual data warehouse, which is used to solve the technical problems in the related technology that application development requires regular extraction of business data stored in different locations into a unified database before application development can be carried out, resulting in high application development costs and high development thresholds.

[0006] To address the above issues, the present invention provides an application construction method based on a virtual data warehouse, wherein the virtual data warehouse includes a preset knowledge base. The method includes:

[0007] Acquire multiple metadata from the preset knowledge base and multiple logical components from the preset component library, wherein the metadata is used to represent data structure information of the data stream, the preset knowledge base is configured with resource mapping information between the metadata and the data stream, and the logical components are used to convert the metadata by executing parameters to obtain output data;

[0008] constructing a directed acyclic application execution graph according to the plurality of metadata and the plurality of logic components;

[0009] Configuring execution parameters of a first target component and configuring an output component of an output node in the directed acyclic application execution graph, wherein the first target component is the logic component in the directed acyclic application execution graph including a plurality of preset parameters, and the output component is used to display the output data;

[0010] Configuring execution parameters of a second target component, and configuring a parameter input component for the execution parameters of the second target component, wherein the second target component is the logic component in the directed acyclic application execution graph including zero preset parameters, and the parameter input component is used to input the execution parameters of the second target component;

[0011] A target application is constructed based on the directed acyclic application execution graph, the execution parameter of the first target component, the output component, the execution parameter of the second target component, and the parameter input component.

[0012] In one embodiment of the present invention, constructing a directed acyclic application execution graph according to the plurality of metadata and the plurality of logic components includes:

[0013] Connecting the metadata in series with a logic component, and determining the metadata as input data of one or more logic components, or determining the metadata as output data of one logic component;

[0014] A plurality of the logic components are connected in series or in parallel to obtain the directed acyclic application execution graph, wherein a directed connection line exists between two connected logic components, and no loop is formed between the logic components.

[0015] In one embodiment of the present invention, the logic component is a preset component in a preset component library. Before obtaining the plurality of logic components and the plurality of metadata in the preset knowledge base, the method further includes:

[0016] Obtaining the logical component information and metadata information to be obtained;

[0017] If the preset component library does not include the preset component of the logical component information to be obtained, a new component is determined based on the logical component information to be obtained and the preset component rule, the new component is added to the preset component library, and the new component is determined as the logical component.

[0018] In one embodiment of the present invention, configuring the output component of the output node of the directed acyclic application execution graph includes:

[0019] Obtain the number of data labels of the output data of the output node;

[0020] Based on the number of data tags, at least one of a cluster display component and an output parameter component is determined as the output component. The cluster display component is used to display multiple output data in a preset graphic display format when the number of data tags is multiple. The output parameter component is used for data presentation of one output data.

[0021] In one embodiment of the present invention, the metadata includes subject data, the resource mapping information includes a mapping relationship between the metadata and the target resource library, the target resource library stores the data flow, and after constructing a directed acyclic application execution graph based on the plurality of metadata and the plurality of logical components, the method further includes:

[0022] Determining a target resource repository for the metadata on each node according to resource mapping information of the metadata on each node in the directed acyclic application execution graph;

[0023] Based on the preset analysis statement and the target resource library type of the target resource library, the preset analysis statement is converted into a conversion analysis statement of the target resource library, so that the logical component is converted according to the preset analysis syntax and the subject data of the metadata corresponding to the logical component to obtain the output data.

[0024] In one embodiment of the present invention, the resource mapping information includes a mapping relationship between the metadata and the target resource library. After constructing the target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component, the method further includes:

[0025] According to the directed acyclic application execution graph, traverse the logical components in the directed acyclic application execution graph in reverse from the node with out-degree zero;

[0026] Obtaining a target repository of metadata of each of the logical components;

[0027] If the input data of the logic component corresponds to at least two metadata, and the target resource repositories of at least two metadata are different, determining the node where the logic component is located as a data flow transfer node;

[0028] Splitting the directed acyclic application execution graph into same-resource library fragment execution graphs according to the data flow transfer nodes;

[0029] Executing each logical component of the directed acyclic application execution graph starting from a node with an in-degree of zero according to the directed acyclic application execution graph;

[0030] If the node where the logic component is located is determined to be the data flow transfer node, the upstream input data of the data flow transfer node is collected to perform logic operations, and the logic operation results are written into the target resource library of the data flow transfer node;

[0031] If the same resource library fragment execution graph is encountered during execution, the conversion analysis statements of each of the logic components are submitted to the target resource library of the data flow transfer node for execution, and the execution results are written into the first preset space of the target resource library of the data flow transfer node;

[0032] If the execution reaches the output logic component node, the final calculation result is stored in the second preset space and returned to the preset interface for display.

[0033] In one embodiment of the present invention, the resource mapping information includes a mapping relationship between the metadata and the target resource library. After constructing the target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component, the method further includes:

[0034] Rendering an application input area based on the execution parameters of the parameter input component and the execution parameters of the second target component;

[0035] If the output component includes a cluster display component, obtaining metadata of the output node and the cluster display component, and rendering the application output area;

[0036] If the output component includes an output parameter component, metadata of the output node and the output parameter component are obtained, and the application execution result is rendered in the application output area.

[0037] An embodiment of the present invention further provides an application construction system based on a virtual data warehouse, wherein the virtual data warehouse includes a preset knowledge base. The system includes a construction layer, wherein the construction layer includes:

[0038] an acquisition module, configured to acquire a plurality of metadata from the preset knowledge base and a plurality of logical components from the preset component library, wherein the metadata is used to represent data structure information of the data stream, the preset knowledge base is configured with resource mapping information between the metadata and the data stream, and the logical components are used to convert the metadata by executing parameters to obtain output data;

[0039] An execution graph construction module, configured to construct a directed acyclic application execution graph according to the plurality of metadata and the plurality of logic components;

[0040] a first configuration module, configured to configure execution parameters of a first target component and an output component of an output node of the directed acyclic application execution graph, wherein the first target component is the logic component in the directed acyclic application execution graph including a plurality of preset parameters, and the output component is configured to display the output data;

[0041] a second configuration module, configured to configure execution parameters of a second target component, and a parameter input component for configuring the execution parameters of the second target component, wherein the second target component is the logic component in the directed acyclic application execution graph including zero preset parameters, and the parameter input component is used to input the execution parameters of the second target component;

[0042] A target application construction module is used to construct a target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component.

[0043] In one embodiment of the present invention, the system further includes:

[0044] a conversion layer for determining, based on the resource mapping information of the metadata on each node in the directed acyclic application execution graph, a target resource library for the metadata on the node; and converting, based on a preset analysis statement and a target resource library type of the target resource library, the preset analysis statement into a conversion analysis statement of the target resource library, so that the logic component performs conversion according to the preset analysis syntax and the subject data of the metadata corresponding to the logic component to obtain the output data;

[0045] The execution layer is used to reversely traverse each of the logical components in the directed acyclic application execution graph from the node with an out-degree of zero according to the directed acyclic application execution graph, obtain the target resource library of the metadata of each of the logical components, and if the input data of the logical component corresponds to at least two metadata, and the target resource libraries of at least two metadata are different, determine the node where the logical component is located as the data flow transfer node, divide the directed acyclic application execution graph into the same resource library fragment execution graph according to the data flow transfer node, and execute the directed acyclic application execution graph from the node with an in-degree of zero according to the directed acyclic application execution graph. For each logic component in the row graph, if the node where the logic component is located is determined to be the data flow transfer node, the upstream input data of the data flow transfer node is collected for logical operation, and the logical operation result is written into the target resource library of the data flow transfer node. If the same resource library fragment execution graph is encountered during execution, the conversion analysis statement of each logic component is submitted to the target resource library of the data flow transfer node for execution, and the execution result is written into the first preset space of the target resource library of the data flow transfer node. If the execution reaches the output logic component node, the final calculation result is stored in the second preset space, and returned to the preset interface for display;

[0046] The resource layer includes a plurality of target resource libraries, and is used to provide the data stream for the target application.

[0047] In one embodiment of the present invention, the system further includes:

[0048] A preset knowledge base stores a plurality of metadata, wherein the metadata is configured with resource mapping information, the metadata is used to represent data structure information of the data stream, and the resource mapping information is used to represent a mapping relationship between the metadata and the data stream;

[0049] A preset component library stores multiple preset components, which are used to convert the metadata into output data by executing parameters.

[0050] An embodiment of the present invention further provides an electronic device, including a processor, a memory, and a communication bus;

[0051] The communication bus is used to connect the processor and the memory;

[0052] The processor is configured to execute the computer program stored in the memory to implement the method as described in any one of the above embodiments.

[0053] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to enable the computer to execute the method as described in any one of the above embodiments.

[0054] As described above, the present invention provides a method, system, device, and storage medium for building an application based on a virtual data warehouse, which has the following beneficial effects:

[0055] The method obtains multiple logical components in a preset component library and multiple metadata in a preset knowledge base, constructs a directed acyclic application execution graph according to the multiple metadata and the multiple logical components, configures the execution parameters of the first target component, and configures the output component of the output node of the directed acyclic application execution graph, configures the execution parameters of the second target component, and configures the parameter input component of the execution parameters of the second target component. The target application is constructed based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component. Through metadata standardization and application componentization, the application construction is separated from the actual data resources, so that the application can smoothly switch between different data resources, which can effectively reduce the application development cost and maintenance fee. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of a method for building an application based on a virtual data warehouse, shown as an exemplary embodiment of the present application.

[0057] Figure 2 This is a specific flowchart of a method for building an application based on a virtual data warehouse, shown as an exemplary embodiment of the present application.

[0058] Figure 3 This is a flowchart of an application conversion method in the application construction process based on a virtual data warehouse, shown as an exemplary embodiment of the present application.

[0059] Figure 4 This is a flowchart of an application execution method in the application construction process based on a virtual data warehouse, shown as an exemplary embodiment of the present application.

[0060] Figure 5 This is a flowchart of an application execution method in the application construction process based on a virtual data warehouse, shown as an exemplary embodiment of the present application.

[0061] Figure 6 This is a block diagram of an application construction system based on a virtual data warehouse, shown as an exemplary embodiment of the present application.

[0062] Figure 7 It is a block diagram of an application construction system based on a virtual data warehouse, shown in another exemplary embodiment of the present application.

[0063] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment.

[0064] Figure 9This is another execution diagram of an application execution method in the application construction process based on a virtual data warehouse, shown as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0065] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0066] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0067] See also Figure 1 , Figure 1 FIG. 1 is a flowchart of a method for constructing an application based on a virtual data warehouse, as shown in an exemplary embodiment of the present application. Figure 1 As shown, in an exemplary embodiment, the application construction method is implemented based on a virtual data warehouse, which includes at least a preset knowledge base. The method includes at least steps S101 to S103, which are described in detail as follows:

[0068] Step S101: Acquire multiple metadata in a preset knowledge base and multiple logical components in a preset component library.

[0069] The metadata is used to characterize the data structure information of the data stream, the preset knowledge base is configured with resource mapping information of the metadata and the data stream, and the logic component is used to convert the metadata through execution parameters to obtain output data.

[0070] In one embodiment, the data stream can be information from a specific database or local storage, or it can be temporary virtual data. This virtual data is then replaced with new data once the required real data is available. This allows the target application to be pre-built even without the original data required. Once the relevant information about the real data is available, the virtual data is replaced with the real data. Furthermore, if the data source is updated, the target application can be updated by modifying the relevant metadata, making this a more convenient update method.

[0071] Metadata can be obtained by selecting the required metadata from multiple metadata in a pre-defined knowledge base. The pre-defined knowledge base provides a unified, standard data format for multiple applications to be built, managing application metadata. Metadata represents the data structure of the data flow, including topics, categories, and indicators. The data flow includes input data that is fed into the logical component.

[0072] Among them, the subject is an abstract description of specific metadata; the category represents the classification to which the metadata belongs and is used for classification management and retrieval; the indicator is the specific description of the metadata, including the indicator name, data type, length, precision, scale, minimum value, maximum value, data dictionary, etc.

[0073] The indicator name represents the identity of the indicator, which is divided into program identification name and user identification name. The program identification name is used when the program processes data, and the user identification name is used to show the meaning of the indicator to the user; the data type represents the basic format of the indicator data, including integer, floating point, logical, character, etc.; the length represents the maximum number of characters that the indicator can accommodate; the precision represents the maximum total number of significant digits before and after the decimal point of the indicator; the scale represents the maximum number of significant digits after the decimal point of the indicator; the minimum value represents the minimum value that can be stored in the indicator; the maximum value represents the maximum value that can be stored in the indicator; the dictionary represents the one-to-one correspondence between the indicator value and the meaning.

[0074] The logic components may be stored in a preset component library, and multiple preset components that meet the needs in the preset component library may be obtained as the logic components.

[0075] The preset component library provides a unified and standard data presentation and data analysis method for multiple applications to be built, including but not limited to display components and logic components.

[0076] The display component is mainly used for data ingestion and data presentation, specifically including cluster display components, input parameter components, and output parameter components. The cluster display component is mainly used for the overall presentation of the output data of the logical component, including but not limited to at least one of line charts, bar charts, donut charts, column charts, pie charts, bubble charts, radar charts, distribution charts, funnel charts, scatter charts, surface charts, lists, cards, maps, and trajectories; the input parameter component is mainly used for the data acquisition of metadata indicators input by the logical component, including but not limited to at least one of text input, number input, drop-down selection, date selection, date range, time selection, image upload, device selection, region selection, personnel labeling, and aggregate input; the output parameter component is mainly used for the data presentation of metadata indicator data output by the logical component, including but not limited to at least one of text, text overflow, enumeration mapping, picture, picture collection, time, and date.

[0077] The logic component is mainly used for data processing, and contains two inputs (execution parameters and input data streams) and one output (output data stream after processing by the logic component). It specifically includes basic components and business components. Basic components mainly cover general data analysis and processing capabilities, including but not limited to at least one of filtering, conversion, association, deduplication, aggregation, collection, interface, script, image search, and illustrations; business components are data aggregation analysis in specific fields. When the metadata data stream includes but is not limited to peer data, trailing data, aggregated data, trajectory data, landing point data, frequent appearance data, etc., the business component includes but is not limited to at least one of peers, trailing data, aggregation, trajectory, landing point, frequent appearance, etc. The business component shields the complex logic of data analysis in a specific field, and only needs to provide simple input to obtain the corresponding analysis results. The processing and reference of data within the logic component can only use standard analysis syntax (preset analysis statements) and the subject metadata provided by the knowledge base. Among them, the standard analysis syntax can be implemented using open source components such as Spark SQL and Flink SQL.

[0078] In one embodiment, the logical component is a preset component in a preset component library. Before obtaining the plurality of logical components and the plurality of metadata in the preset knowledge base, the method further includes:

[0079] Acquire the logical component information to be acquired and the metadata information to be acquired, wherein the logical component information to be acquired and the metadata information to be acquired can be preset by those skilled in the art, and the mapping relationship between each logical component information to be acquired and the preset components in the preset component library can be preset by those skilled in the art, as can the similarity, and the mapping relationship between each metadata information to be acquired and the metadata in the preset metadata library can be preset by those skilled in the art. For example, the metadata and the preset components can be respectively set with multiple tags, and the tags can be used as the logical component information to be acquired and the metadata information to be acquired, thereby realizing the selection of metadata and the selection of preset components;

[0080] If the preset component library does not include the preset component of the logical component information to be obtained, a new component is determined based on the logical component information to be obtained and the preset component rules, the new component is added to the preset component library, and the new component is determined as a logical component. That is, in theory, the preset component library needs to include all required logical components, but if the required logical component does not exist in the preset component library, it is necessary to determine the constructed logical component in advance based on the logical component information to be obtained and the preset component rules, and add the newly added component to the preset component library. This can ensure that the newly added component can be applied when building the application next time, which facilitates the construction of subsequent applications and saves labor costs.

[0081] In one embodiment, if the preset knowledge base does not include the metadata of the metadata information to be obtained, the newly added metadata is determined based on the metadata information to be obtained and the preset metadata rules, and the newly added resource mapping information of the newly added metadata is configured, and the newly added metadata and the newly added resource mapping information are added to the preset metadata database, and the newly added metadata is determined as metadata. Similar to the reason for adding the new component mentioned above, the newly added metadata and the newly added resource mapping information of the newly added metadata can be determined according to the preset metadata rules as needed.

[0082] The newly added resource mapping information and the resource mapping information of the metadata in the preset knowledge base include, but are not limited to, the mapping relationship between the subject, the address, and the storage method in the metadata. For example, the field in the database table corresponding to a certain subject. The storage method of the data corresponding to the subject can be local storage or accessed through a third-party interface. In this case, the address mapped to the subject includes, but is not limited to, the interface host address, port, format, path, etc.

[0083] Step S102 : constructing a directed acyclic application execution graph according to the plurality of metadata and the plurality of logical components.

[0084] In one embodiment, constructing a directed acyclic application execution graph according to a plurality of metadata and a plurality of logical components includes:

[0085] Concatenate metadata with a logic component and determine the metadata as input data of one or more logic components, or determine the metadata as output data of a logic component;

[0086] Multiple logic components are connected in series or in parallel to obtain a directed acyclic application execution graph, in which a directed connection line exists between two connected logic components and no loop is formed between the logic components.

[0087] Metadata is connected in series with logical components, with metadata acting as input or output data resources for the logical components. If metadata is used as an input data resource, it can be used as an input resource for multiple logical components simultaneously; if metadata is used as an output data resource, it can only be used as an output resource for a specific logical component. Multiple logical components can be connected in series or in parallel, with only one directed connection allowed between any two logical components, and no loops. This allows multiple metadata and multiple logical components to form a directed acyclic application execution graph.

[0088] The specific metadata and logical component to be connected in series can be set by those skilled in the art as needed.

[0089] The construction of an application relies on a preset component library and a preset knowledge base. The preset component library provides various display components and logic components, and the preset knowledge base provides all metadata information. An application can contain multiple metadata and multiple logic components. Logical components can be connected in series or parallel. Metadata can only be connected in series with logic components as input or output of the logic components, forming a directed acyclic execution graph. The output nodes in the execution graph must be configured with corresponding display components.

[0090] In one embodiment, the metadata includes subject data, the resource mapping information includes a mapping relationship between the metadata and a target resource library, the target resource library stores a data flow, and after constructing a directed acyclic application execution graph based on the plurality of metadata and the plurality of logical components, the method further includes:

[0091] Determine the target resource repository for the metadata on each node based on the resource mapping information of the metadata on each node in the directed acyclic application execution graph;

[0092] Based on the preset analysis statement and the target resource library type of the target resource library, the preset analysis statement is converted into a conversion analysis statement of the target resource library, so that the logic component is converted according to the preset analysis syntax and the subject data of the metadata corresponding to the logic component to obtain output data.

[0093] By converting the subject metadata used by the logical component into the data resources of the target resource library according to the mapping relationship configured with the target resource library, and converting the logical component analysis process into the analysis syntax of the target resource library (conversion analysis statement), the data resources (data flow) of the target resource library include but are not limited to solidified tables, temporary tables, memory tables, and view tables. The logical component analysis process conversion includes but is not limited to DQL / DML / DDL / DCL syntax, data type and resource library function conversion.

[0094] Step S103 , configuring the execution parameters of the first target component, and configuring the output component of the output node of the directed acyclic application execution graph.

[0095] The first target component is a logic component including multiple preset parameters in the directed acyclic application execution graph, and the output component is used to display output data.

[0096] Some of the logic components in the preset component library have one or more pre-set execution parameters. After completing the directed acyclic application execution graph, one of the multiple pre-set execution parameters must be selected for the logic component as the execution parameter of the first target component (logic component). The implementation of this process can be customized by those skilled in the art as needed.

[0097] In one embodiment, configuring an output component of an output node of a directed acyclic application execution graph includes:

[0098] Get the number of data labels of the output data of the output node;

[0099] Based on the number of data tags, at least one of the cluster display component and the output parameter component is determined as the output component. The cluster display component is used to display multiple output data in a preset graphic display format when there are multiple data tags. The output parameter component is used to present one output data.

[0100] The preset graphic display format can be in the form of the above-mentioned cluster display component, including but not limited to at least one of a line chart, a bar chart, a donut chart, a column chart, a pie chart, a bubble chart, a radar chart, a distribution chart, a funnel chart, a scatter chart, a surface chart, a list, a card, a map, a track, etc.

[0101] Output data can also be displayed directly through the output parameter component instead of through graphical display. Alternatively, the output data can be displayed through both the cluster display component and the output parameter component.

[0102] Step S104: configuring the execution parameters of the second target component and configuring the parameter input component of the execution parameters of the second target component.

[0103] The second target component is a logic component including zero preset parameters in the directed acyclic application execution graph, and the parameter input component is used to input the execution parameters of the second target component.

[0104] In other words, some logical components in the pre-set component library do not have pre-set execution parameters. In this case, after completing the DAA execution graph, execution parameters need to be manually set for these logical components. In this case, the input parameter component can be configured to allow users to enter customized execution parameters for certain logical components. The input parameter component can be any component known to those skilled in the art, and the specific format and rules of the input parameter component are not limited here.

[0105] Step S105 , constructing a target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component.

[0106] During the process of building the target application, you need to configure and debug the target application, and then release the target application after successful debugging.

[0107] The specific application scenario of the target application can be determined based on the selected metadata and logical components. For example, the data stream corresponding to the obtained metadata is trajectory data, and the logical component includes a trajectory determination component, then the target application is a trajectory tracking application. For another example, the data stream corresponding to the obtained metadata is the trajectory data of the vehicle, and the logical component includes an illegal operation determination component, then the target application is an illegal operation determination application. It should be noted that the data streams corresponding to the metadata in the embodiments of the present application are all data obtained by those skilled in the art after obtaining corresponding authorization (such as authorization from relevant departments, authorization from relevant personnel, etc.).

[0108] See also Figure 2 , Figure 2 This is a specific flow chart of an application construction method shown in an exemplary embodiment of the present application. Figure 2 As shown, the specific method includes:

[0109] Step S201: Select one or more metadata from a knowledge base.

[0110] Select one or more metadata that meet your business needs from the preset knowledge base. If the preset knowledge base does not have metadata that meets your business needs, you can customize and add relevant metadata and resource mapping information.

[0111] Step S202: Select one or more logical components from the component library.

[0112] Select one or more logic components that meet your business requirements from the preset component library. If the preset component library does not have a logic component that meets your business requirements, you can add a custom logic component according to the component specifications.

[0113] Step S203: construct a directed acyclic application execution graph.

[0114] Build a directed acyclic application execution graph using the selected metadata and logical components, including:

[0115] Metadata is connected in series with logical components as the input or output data resource of the logical components. If metadata is used as an input data resource, it can be used as an input resource for multiple logical components at the same time; if metadata is used as an output data resource, it can only be used as an output resource for a certain logical component.

[0116] Connect multiple logic components in series or in parallel. Only one directed connection line is allowed between any two logic components, and no loop is allowed.

[0117] Step S204: configuring the execution parameters of the logic components in the application execution graph and the cluster display component and output parameter component of the output logic component nodes.

[0118] Step S205: configure application input parameters and parameter input components.

[0119] Obtain execution parameters of all logical components from the application execution graph (ie, the directed acyclic application execution graph described above), select execution parameters that require manual user input as input parameters of the application, and configure corresponding input parameter components.

[0120] Step S206: Check the configuration and debug the big data application. After successful debugging, release the big data application.

[0121] Big data applications are also target applications.

[0122] See also Figure 3 , Figure 3 This is a flow chart of an application conversion method in an application construction process shown in an exemplary embodiment of the present application. Figure 3 When the application is debugged and executed or the user triggers the application execution after the application is released, the application conversion method process is as follows:

[0123] Step S301: Obtain a big data application execution graph.

[0124] That is, the directed acyclic application execution graph mentioned in the above embodiment is obtained.

[0125] Step S302 traverses each node in the graph, starting from the node with in-degree zero, to obtain the target repository type and standard analysis statement mapped to the metadata of each node, and converts the standard analysis statement into an analysis statement corresponding to the target repository based on the target repository type. The aforementioned graph is a directed acyclic application execution graph, the standard analysis statement can be a standard statement in the relevant technology, or a statement customized by a person skilled in the art, the analysis statement is the aforementioned conversion analysis statement, and the standard analysis statement is the aforementioned preset analysis grammar.

[0126] The target resource library corresponding to the metadata can be an actual resource library or a virtual resource library pre-set by technical personnel in this field. Through the above method, even if the specific address and other information of the target resource library corresponding to the metadata are not obtained, and the access rights to the target resource library have not been obtained, the metadata can still be created first, and the target application can be built based on the metadata. After the access rights and other conditions of the target resource library corresponding to the metadata are obtained later, the target application can be debugged and used. There is no need to extract business data stored in different locations to a unified database before developing the application, which reduces the cost and threshold of application development, improves the efficiency of application development, and effectively shortens the time of application development.

[0127] In one embodiment, the resource mapping information includes a mapping relationship between metadata and a target resource library. After constructing the target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component, the method further includes:

[0128] According to the directed acyclic application execution graph, reversely traverse each logical component in the directed acyclic application execution graph from the node with out-degree zero;

[0129] Get the target repository of metadata for each logical component;

[0130] If the input data of the logic component corresponds to at least two metadata, and the target resource repositories of at least two metadata are different, the node where the logic component is located is determined as the data flow transfer node;

[0131] Split the directed acyclic application execution graph into fragment execution graphs with the same resource library based on the data flow transfer nodes;

[0132] Execute each logical component of the directed acyclic application execution graph starting from the node with in-degree zero according to the directed acyclic application execution graph;

[0133] If the node where the logic component is located is determined to be a data flow transfer node, the upstream input data of the data flow transfer node is collected to perform logic operations, and the results of the logic operations are written into the target resource library of the data flow transfer node;

[0134] If the same resource library fragment execution graph is encountered during the execution process, the conversion analysis statements of each logical component are submitted to the target resource library of the data flow transfer node for execution, and the execution results are written to the first preset space (such as a memory table, etc.) of the target resource library of the data flow transfer node;

[0135] If the execution reaches the output logic component node, the final calculation result is stored in the second preset space (such as the result table) and returned to the preset interface for display. At this time, the output result can be displayed through the output component.

[0136] See also Figure 9 , Figure 9 This is an execution diagram of an application execution method in the application construction process shown in an exemplary embodiment of the present application. Figure 9 As shown, F1 and F2 represent two data resources in resource library F; G1 and G2 represent two data resources in resource library G. Figure 9There are five logical component nodes, ABCDE. A, B, and C, connected in series, all use metadata belonging to data source F (F1, F2). D uses metadata from C's output and requires metadata from data source G (G1). E is connected in series with D, using D's output as its data source and other metadata from G (G2). Therefore, A, B, and C are the execution graph for the same resource library fragment. C is the output node of this fragment, and the execution result is still stored in the data source F library. D is a data flow transit node, so D's final output result will be stored in G. E will use D's calculation result and other metadata from G to calculate the final output result, which is still stored in G.

[0137] See also Figure 4 , Figure 4 This is a flow chart of an exemplary embodiment of the present application showing an application execution method during the application construction process. When the application is debugged and executed or the user triggers the application execution after the application is released, such as Figure 4 The application execution method flow is as follows:

[0138] Step S401: Obtain a big data application execution graph.

[0139] For example, obtain application execution graph configuration information.

[0140] Step S402: traverse the logical component nodes in the graph backward from the node with out-degree zero, mark the data flow transfer nodes, and split the execution graph according to the data flow transfer nodes to obtain the same resource library fragment execution graph.

[0141] According to the execution graph, reversely traverse the logical component nodes in the graph from the node with zero out-degree, obtain the target resource library corresponding to the input / output data flow of the logical component node, and if the input data flow comes from different target resource libraries, use the logical component node as the data flow transfer node, and split the execution graph into the same resource library fragment execution graph according to the data flow transfer node;

[0142] Step S403: Starting from the node with in-degree zero, each logic component node in the graph is executed in parallel or serially. When the output logic component node is executed, the final calculation result is stored in the result table and returned to the user interface for display.

[0143] According to the execution graph, each logical component node in the graph is executed in parallel or serially starting from the node with zero in-degree. When a data flow transfer node is encountered during the execution process, the upstream input data flow is collected for logical operation and the calculation results are written to the target resource library. When the same resource library fragment execution graph is encountered, all logical component node analysis statements are combined and optimized and submitted to the target resource library for execution, and the execution results are written to the target resource library's memory table. The above process can be referred to Figure 9The process shown in the figure is as follows: when the output logic component node is executed, the final calculation result is stored in the result table and returned to the user interface for display.

[0144] In one embodiment, the resource mapping information includes a mapping relationship between metadata and a target resource library. After constructing the target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component, the method further includes:

[0145] Rendering the application input area based on the execution parameters of the parameter input component and the execution parameters of the second target component;

[0146] If the output component includes a cluster display component, obtain the metadata of the output node and the cluster display component, and render the application output area;

[0147] If the output component includes an output parameter component, obtain the metadata and output parameter component of the output node, and render the application execution result in the application output area.

[0148] See also Figure 5 , Figure 5 This is a flow chart of an exemplary embodiment of the present application showing an application execution method during the application construction process. When the application is debugged and executed or the user triggers the application execution after the application is released, such as Figure 5 The application data rendering method process is as follows:

[0149] Step S501: Obtain application execution graph configuration information.

[0150] Step S502: Obtain input parameters and input parameter components of the big data application, and render the application input area according to the input parameters and input parameter components;

[0151] Step S503: Obtain the application output logic component node cluster display component and node output metadata, and render the application output area.

[0152] The application output logic component node cluster display component and node output metadata can be obtained from the application execution graph configuration information to render the application output area.

[0153] Step S504 : obtaining the output parameter components of the output node metadata and metadata indicators, and rendering the application execution result in the application output area according to the output node metadata and metadata indicator output parameter components.

[0154] The output node metadata and output parameter components of the metadata indicators are obtained from the application execution graph configuration information, and the application execution results are rendered in the application output area according to the output node metadata and output parameter components of the metadata indicators.

[0155] The application construction method provided in the above embodiment obtains multiple logical components and multiple metadata from a preset knowledge base, constructs a directed acyclic application execution graph based on the multiple metadata and multiple logical components, configures the execution parameters of the first target component, and configures the output component of the output node of the directed acyclic application execution graph, configures the execution parameters of the second target component, and configures the parameter input component of the execution parameters of the second target component. The target application is constructed based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component. Through metadata standardization and application componentization, the application construction is separated from the actual data resources, so that the application can switch smoothly under different data resources, which can effectively reduce the application development cost and maintenance cost.

[0156] Even if you do not have the specific address and other information of the target resource library corresponding to the metadata, and have not yet obtained access rights to the target resource library, you can still create metadata first, build the target application based on the metadata, and then debug and use the target application after obtaining access rights and other conditions for the target resource library corresponding to the metadata. There is no need to extract business data stored in different locations to a unified database before developing the application, which reduces the cost and threshold of application development, improves the efficiency of application development, and effectively shortens the time of application development. The method provided by the above embodiment solves the problem of the same application needing to be repeatedly developed in different data resource scenarios, and also solves the problem of the strong coupling between application and data resources, which makes it difficult to commercialize the application.

[0157] Currently, the common practice for application implementation is to use data extraction, transformation, and load (ETL) tools to periodically extract or synchronize heterogeneous data to a unified OLAP database in real time, and then rely on a specific OLAP database or big data computing engine to perform specific business analysis. This is undoubtedly the most costly, requiring at least double the amount of storage facilities already in an enterprise / institution / organization. Development and implementation from data to application must rely on specific projects, and it is impossible to abstract metadata based on industry characteristics and build industry-wide applications for productization.

[0158] To solve the above problems, this application proposes an application construction system based on a virtual data warehouse, which divides the application construction system into three modules: knowledge base, component library, and application construction. The application construction is further divided vertically into construction layer, conversion layer, execution layer, and resource layer. The construction layer directly depends on the knowledge base and component library, thereby avoiding the coupling between the application and resource layers and enhancing the application's adaptability to different data sources.

[0159] See also Figure 6 , Figure 6FIG. 1 is a block diagram of an application construction system based on a virtual data warehouse, as shown in an exemplary embodiment of the present application. Figure 6 As shown, this embodiment provides an application construction system 600 based on a virtual data warehouse. The virtual data warehouse includes a preset knowledge base. The system includes a construction layer, which includes:

[0160] An acquisition module 601 is configured to acquire multiple metadata from a preset knowledge base and multiple logical components from a preset component library. The metadata is used to represent data structure information of a data stream. The preset knowledge base is configured with resource mapping information between the metadata and the data stream. The logical components are used to convert the metadata by executing parameters to obtain output data.

[0161] An execution graph construction module 602 is configured to construct a directed acyclic application execution graph based on multiple metadata and multiple logical components;

[0162] A first configuration module 603 is used to configure execution parameters of a first target component and an output component of an output node of a directed acyclic application execution graph. The first target component is a logical component in the directed acyclic application execution graph that includes a plurality of preset parameters. The output component is used to display output data.

[0163] A second configuration module 604 is configured to configure execution parameters of a second target component and a parameter input component for configuring the execution parameters of the second target component, wherein the second target component is a logical component in a directed acyclic application execution graph that includes zero preset parameters, and the parameter input component is used to input the execution parameters of the second target component;

[0164] The target application construction module 605 is configured to construct a target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component.

[0165] Among them, the preset knowledge base stores multiple metadata, the metadata is configured with resource mapping information, the metadata is used to characterize the data structure information of the data flow, the resource mapping information is used to characterize the mapping relationship between the metadata and the data flow, and the preset component library stores multiple preset components, which are used to convert the metadata through execution parameters to obtain output data.

[0166] In one embodiment, the metadata includes subject data, the resource mapping information includes a mapping relationship between the metadata and a target resource library, and the target resource library stores a data flow. The apparatus further includes an application conversion module configured to, after constructing a directed acyclic application execution graph based on the plurality of metadata and the plurality of logical components:

[0167] Determine the target resource repository for the metadata on each node based on the resource mapping information of the metadata on each node in the directed acyclic application execution graph;

[0168] Based on the preset analysis statement and the target resource library type of the target resource library, the preset analysis statement is converted into a conversion analysis statement of the target resource library, so that the logic component is converted according to the preset analysis syntax and the subject data of the metadata corresponding to the logic component to obtain output data.

[0169] In one embodiment, the resource mapping information includes a mapping relationship between metadata and a target resource library. The apparatus further includes an application execution module, wherein the application execution module is configured to construct a target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component.

[0170] According to the directed acyclic application execution graph, reversely traverse each logical component in the directed acyclic application execution graph from the node with out-degree zero;

[0171] Get the target repository of metadata for each logical component;

[0172] If the input data of the logic component corresponds to at least two metadata, and the target resource repositories of at least two metadata are different, the node where the logic component is located is determined as the data flow transfer node;

[0173] Split the directed acyclic application execution graph into fragment execution graphs with the same resource library based on the data flow transfer nodes;

[0174] Execute each logical component of the directed acyclic application execution graph starting from the node with in-degree zero according to the directed acyclic application execution graph;

[0175] If the node where the logic component is located is determined to be a data flow transfer node, the upstream input data of the data flow transfer node is collected to perform logic operations, and the results of the logic operations are written into the target resource library of the data flow transfer node;

[0176] If the same resource library fragment execution graph is encountered during the execution process, the conversion analysis statements of each logical component are submitted to the target resource library of the data flow transfer node for execution, and the execution results are written to the first preset space of the target resource library of the data flow transfer node;

[0177] If the execution reaches the output logic component node, the final calculation result is stored in the second preset space and returned to the preset interface for display.

[0178] In one embodiment, the apparatus further includes an application rendering module, configured to construct the target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component:

[0179] Rendering the application input area based on the execution parameters of the parameter input component and the execution parameters of the second target component;

[0180] If the output component includes a cluster display component, obtain the metadata of the output node and the cluster display component, and render the application output area;

[0181] If the output component includes an output parameter component, obtain the metadata and output parameter component of the output node, and render the application execution result in the application output area.

[0182] See also Figure 7 , Figure 7 FIG. 1 is a block diagram of an application construction system based on a virtual data warehouse, as shown in another exemplary embodiment of the present application. Figure 7 As shown in the figure, the application construction system based on the virtual data warehouse includes, in addition to the construction layer, the following:

[0183] The conversion layer is used to determine the target resource library of the metadata on each node in the directed acyclic application execution graph based on the resource mapping information of the metadata on the node, and based on the preset analysis statement and the target resource library type of the target resource library, convert the preset analysis statement into a conversion analysis statement of the target resource library, so that the logic component converts the subject data of the metadata corresponding to the preset analysis syntax and the logic component to obtain output data;

[0184] The execution layer is used to reversely traverse each logical component in the directed acyclic application execution graph from the node with zero out-degree according to the directed acyclic application execution graph, obtain the target resource library of the metadata of each logical component, and if the input data of the logical component corresponds to at least two metadata, and the target resource libraries of at least two metadata are different, determine the node where the logical component is located as the data flow transfer node, divide the directed acyclic application execution graph into the same resource library fragment execution graph according to the data flow transfer node, and execute each logical component of the directed acyclic application execution graph starting from the node with zero in-degree according to the directed acyclic application execution graph. If the node where the logical component is located is determined to be the data flow transfer node, collect the upstream input data of the data flow transfer node for logical operation, and write the logical operation result to the target resource library of the data flow transfer node. If the same resource library fragment execution graph is encountered during the execution process, the conversion analysis statement of each logical component is submitted to the target resource library of the data flow transfer node for execution, and the execution result is written to the first preset space of the target resource library of the data flow transfer node. If the execution reaches the output logical component node, the final calculation result is stored in the second preset space and returned to the preset interface for display;

[0185] The resource layer includes multiple target resource libraries, which are used to provide data streams for target applications, that is, to provide raw data for target applications.

[0186] like Figure 7 As shown in the figure, the application module includes the construction layer, conversion layer, execution layer, and resource layer. The application module is mainly used for application development and execution, including:

[0187] The construction layer is primarily used for building applications. Theoretically, there's no limit to the number of applications. Application construction relies on a component library (pre-set component library) and a knowledge base (pre-set knowledge base). The component library provides various display components and logic components, while the knowledge base provides all metadata information. An application can contain multiple metadata and multiple logic components. Logical components can be connected in series or parallel. Metadata can only be connected in series with logic components as input or output, forming a directed acyclic execution graph (the aforementioned application execution graph, directed acyclic application execution graph, and execution graph). Output nodes in the execution graph must be configured with corresponding display components.

[0188] The conversion layer primarily converts the subject metadata used by logical components into the target repository's data resources based on the mapping relationship configured with the target repository. It also converts the logical component analysis process into the target repository's analysis syntax. The target repository's data resources include, but are not limited to, fixed tables, temporary tables, in-memory tables, and view tables. The logical component analysis process conversion includes, but is not limited to, DQL / DML / DDL / DCL syntax, data types, and repository function conversion.

[0189] The execution layer is mainly used to execute applications. Based on the application execution graph, the reverse analysis method is used to analyze the target resource library where the input data flow of each logical component node on the execution graph is located. Starting from the output node in the execution graph, the execution graph is traversed in reverse. If the input data flow comes from a different target resource library, the logical component node is used as a data flow transfer node, and the execution graph is divided into the same resource library fragment execution graph according to the data flow transfer node. When executing a specific application, the execution layer starts from the node with zero in-degree in the execution graph and executes each logical component node in the graph in parallel or serially. When encountering a data flow transfer node during the execution process, the upstream input data flow is collected for logical operation and the calculation results are written to the target resource library. When encountering the same resource library fragment execution graph, all logical component node analysis statements are combined and optimized and submitted to the target resource library for execution, and the execution results are written to the target resource library's memory table. When the output logical component node is executed, the final calculation result is stored in the result table or returned to the user interface for display.

[0190] The above-mentioned conversion layer and execution layer can be secondary developed based on the open source components Spark and Flink, and the above-mentioned method of this application can be encapsulated and run on their open source components.

[0191] The resource layer primarily provides applications with raw data (data streams corresponding to metadata), including local and remote resources. Local resources include, but are not limited to, text, CSV, Excel, and cached data (to improve application data analysis efficiency, infrequently changing data resources are cached locally to avoid having to retrieve them from the resource library each time). Remote resources include, but are not limited to, relational and non-relational databases. Raw data can span data centers and network segments.

[0192] Continue to see Figure 7 , the system further comprises:

[0193] Preset knowledge base (i.e. Figure 7 The knowledge base module in the data flow stores multiple metadata, and the metadata is configured with resource mapping information. The metadata is used to represent the data structure information of the data flow, and the resource mapping information is used to represent the mapping relationship between the metadata and the data flow;

[0194] Preset component library (also known as Figure 7 The component library module in the module stores multiple preset components, which are used to convert metadata by executing parameters to obtain output data.

[0195] The system provided in this embodiment utilizes layered construction, metadata standardization, and application componentization to separate application construction from actual data resources. This allows applications to smoothly switch between different data resources, effectively reducing application development and maintenance costs. This separation of application construction from actual data resources avoids coupling between the application and resource layers, enhances the application's adaptability to different data sources, and enables smooth switching based on the data resource storage methods of different sites. This effectively reduces the development and maintenance costs associated with adapting the same application to different data resources in different projects.

[0196] In this embodiment, the system is essentially provided with multiple modules for executing the method in any of the above embodiments. The specific functions and technical effects can be referred to the above embodiments and will not be described in detail here.

[0197] See also Figure 8 , an embodiment of the present invention further provides an electronic device 800, including a processor 801, a memory 802 and a communication bus 803;

[0198] The communication bus 803 is used to connect the processor 801 and the memory 802;

[0199] The processor 801 is configured to execute the computer program stored in the memory 802 to implement one or more of the methods described in the above embodiments.

[0200] The embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon.

[0201] The computer program is used to enable a computer to execute the method described in any one of the above-mentioned embodiments.

[0202] An embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute the instructions (instructions) of the steps included in embodiment 1 of the embodiment of the present application.

[0203] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0204] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0205] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0206] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0207] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A method for building an application based on a virtual data warehouse, characterized in that: The virtual data warehouse includes a preset knowledge base, and the method includes: Acquire multiple metadata from the preset knowledge base and multiple logical components from the preset component library, wherein the metadata is used to represent data structure information of the data stream, the preset knowledge base is configured with resource mapping information between the metadata and the data stream, and the logical components are used to convert the metadata by executing parameters to obtain output data; Constructing a directed acyclic application execution graph based on the plurality of metadata and the plurality of logic components, including connecting one metadata and one logic component in series, and determining the metadata as input data of one or more logic components, or determining the metadata as output data of one logic component; connecting the plurality of logic components in series or in parallel to obtain the directed acyclic application execution graph, wherein a directed connection line exists between two connected logic components, and no loop is formed between the logic components; Configuring execution parameters of a first target component and configuring an output component of an output node in the directed acyclic application execution graph, wherein the first target component is the logic component in the directed acyclic application execution graph including a plurality of preset parameters, and the output component is used to display the output data; Configuring execution parameters of a second target component, and configuring a parameter input component for the execution parameters of the second target component, wherein the second target component is the logic component in the directed acyclic application execution graph including zero preset parameters, and the parameter input component is used to input the execution parameters of the second target component; A target application is constructed based on the directed acyclic application execution graph, the execution parameter of the first target component, the output component, the execution parameter of the second target component, and the parameter input component.

2. The application construction method according to claim 1, wherein: The logic component is a preset component in a preset component library. Before acquiring multiple logic components and multiple metadata in a preset knowledge base, the method further includes: Obtaining the logical component information and metadata information to be obtained; If the preset component library does not include the preset component of the logical component information to be obtained, a new component is determined based on the logical component information to be obtained and the preset component rule, the new component is added to the preset component library, and the new component is determined as the logical component.

3. The application construction method according to claim 1, wherein: Configuring the output component of the output node of the directed acyclic application execution graph includes: Obtain the number of data labels of the output data of the output node; Based on the number of data tags, at least one of a cluster display component and an output parameter component is determined as the output component. The cluster display component is used to display multiple output data in a preset graphic display format when the number of data tags is multiple. The output parameter component is used for data presentation of one output data.

4. The application construction method according to any one of claims 1 to 3, characterized in that: The metadata includes subject data, the resource mapping information includes a mapping relationship between the metadata and a target resource library, the target resource library stores the data flow, and after constructing a directed acyclic application execution graph based on the plurality of metadata and the plurality of logical components, the method further includes: Determining a target resource repository for the metadata on each node according to resource mapping information of the metadata on each node in the directed acyclic application execution graph; Based on the preset analysis statement and the target resource library type of the target resource library, the preset analysis statement is converted into a conversion analysis statement of the target resource library, so that the logical component is converted according to the preset analysis syntax and the subject data of the metadata corresponding to the logical component to obtain the output data.

5. The application construction method according to any one of claims 1 to 3, characterized in that: The resource mapping information includes a mapping relationship between the metadata and the target resource library. After constructing the target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component, the method further includes: According to the directed acyclic application execution graph, traverse the logical components in the directed acyclic application execution graph in reverse from the node with out-degree zero; Obtaining a target repository of metadata of each of the logical components; If the input data of the logic component corresponds to at least two metadata, and the target resource repositories of at least two metadata are different, determining the node where the logic component is located as a data flow transfer node; Splitting the directed acyclic application execution graph into same-resource library fragment execution graphs according to the data flow transfer nodes; Executing each logical component of the directed acyclic application execution graph starting from a node with an in-degree of zero according to the directed acyclic application execution graph; If the node where the logic component is located is determined to be the data flow transfer node, the upstream input data of the data flow transfer node is collected to perform logic operations, and the logic operation results are written into the target resource library of the data flow transfer node; If the same resource library fragment execution graph is encountered during execution, the conversion analysis statements of each of the logic components are submitted to the target resource library of the data flow transfer node for execution, and the execution results are written into the first preset space of the target resource library of the data flow transfer node; If the execution reaches the output logic component node, the final calculation result is stored in the second preset space and returned to the preset interface for display.

6. The application construction method according to any one of claims 1 to 3, characterized in that: The resource mapping information includes a mapping relationship between the metadata and the target resource library. After constructing the target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component, the method further includes: Rendering an application input area based on the execution parameters of the parameter input component and the execution parameters of the second target component; If the output component includes a cluster display component, obtaining metadata of the output node and the cluster display component, and rendering the application output area; If the output component includes an output parameter component, metadata of the output node and the output parameter component are obtained, and the application execution result is rendered in the application output area.

7. An application construction system based on a virtual data warehouse, characterized in that: The virtual data warehouse includes a preset knowledge base. The system includes a construction layer, which includes: An acquisition module is provided for acquiring a plurality of metadata from the preset knowledge base and a plurality of logical components from the preset component library, wherein the metadata is used to characterize data structure information of a data flow, the preset knowledge base is configured with resource mapping information of the metadata and the data flow, and the logical component is used to convert the metadata through execution parameters to obtain output data; an execution graph construction module is provided for constructing a directed acyclic application execution graph based on the plurality of metadata and the plurality of logical components, and constructing the directed acyclic application execution graph includes connecting one metadata and one logical component in series, and determining the metadata as input data of one or more logical components, or determining the metadata as output data of one logical component; connecting a plurality of logical components in series or in parallel to obtain the directed acyclic application execution graph, wherein a directed connection line exists between two connected logical components, and no loop is formed between the logical components; a first configuration module, configured to configure execution parameters of a first target component and an output component of an output node of the directed acyclic application execution graph, wherein the first target component is the logic component in the directed acyclic application execution graph including a plurality of preset parameters, and the output component is configured to display the output data; a second configuration module, configured to configure execution parameters of a second target component, and a parameter input component for configuring the execution parameters of the second target component, wherein the second target component is the logic component in the directed acyclic application execution graph including zero preset parameters, and the parameter input component is used to input the execution parameters of the second target component; A target application construction module is used to construct a target application based on the directed acyclic application execution graph, the execution parameters of the first target component, the output component, the execution parameters of the second target component, and the parameter input component.

8. The application construction system according to claim 7, wherein: The system further comprises: A conversion layer is used to determine the target resource library of the metadata on each node in the directed acyclic application execution graph based on the resource mapping information of the metadata on the node, and convert the preset analysis statement into a conversion analysis statement of the target resource library based on a preset analysis statement and a target resource library type of the target resource library, so that the logic component performs conversion according to the preset analysis syntax and the subject data of the metadata corresponding to the logic component to obtain the output data; The execution layer is used to reversely traverse each of the logical components in the directed acyclic application execution graph from the node with an out-degree of zero according to the directed acyclic application execution graph, obtain the target resource library of the metadata of each of the logical components, and if the input data of the logical component corresponds to at least two metadata, and the target resource libraries of at least two metadata are different, determine the node where the logical component is located as the data flow transfer node, divide the directed acyclic application execution graph into the same resource library fragment execution graph according to the data flow transfer node, and execute the directed acyclic application execution graph from the node with an in-degree of zero according to the directed acyclic application execution graph. For each logic component in the row graph, if the node where the logic component is located is determined to be the data flow transfer node, the upstream input data of the data flow transfer node is collected for logical operation, and the logical operation result is written into the target resource library of the data flow transfer node. If the same resource library fragment execution graph is encountered during execution, the conversion analysis statement of each logic component is submitted to the target resource library of the data flow transfer node for execution, and the execution result is written into the first preset space of the target resource library of the data flow transfer node. If the execution reaches the output logic component node, the final calculation result is stored in the second preset space, and returned to the preset interface for display; The resource layer includes a plurality of target resource libraries, and is used to provide the data stream for the target application.

9. The application construction system according to any one of claims 7 or 8, wherein: The system further comprises: A preset knowledge base stores a plurality of metadata, wherein the metadata is configured with resource mapping information, the metadata is used to represent data structure information of the data stream, and the resource mapping information is used to represent a mapping relationship between the metadata and the data stream; The preset component library stores a plurality of preset components, and the preset components are used to convert the metadata by executing parameters to obtain output data.

10. An electronic device, characterized in that: Includes processor, memory and communication bus; The communication bus is used to connect the processor and the memory; The processor is configured to execute the computer program stored in the memory to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that A computer program is stored thereon, The computer program is used to cause the computer to execute the method according to any one of claims 1 to 7.

Images