Construction method, device, equipment, medium and product

Abstract

The invention discloses a construction method and device, equipment, a medium and a product in the technical field of computers, and the method comprises the steps: responding to a configuration operation triggered in a configuration page, constructing metadata of each layer in a data warehouse according to configuration information indicated by the configuration operation, the metadata can describe characteristics presented by each layer in the data warehouse when the target data is acquired from the data warehouse; constructing a first precursor graph according to the metadata, so that the first precursor graph can indicate a dependency relationship between the metadata of different layers in the data warehouse; a first node is constructed according to the first precursor graph, so that the first node can describe a working process of obtaining the target data from the data warehouse, and data meeting the current analysis requirement can be automatically extracted from the data warehouse by using the first node subsequently; in this way, the data production node meeting the configuration can be automatically constructed based on the user configuration, manual participation of other personnel is not needed, and therefore efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a construction method, apparatus, device, medium, and product. Background Art

[0002] For some scenarios, these scenarios may have the following requirements: perform a certain analysis based on the stored data in the data warehouse (such as the touchpoint data of a certain product) to obtain an analysis result, so that the analysis result can represent the characteristics indicated by the stored data (such as the usage characteristics presented by a certain user under this product). Among them, the touchpoint refers to an object that exists on the interaction interface of the product, is created by means of data embedding, and can obtain interaction data (such as behavior data, etc.), so that the touchpoint can be received by the user through the senses and transmit some information. Touchpoint data refers to the data obtained through the touchpoint and stored in the data warehouse (such as interaction data, etc.), so that the touchpoint data can represent the usage characteristics presented by the user under this product. Summary of the Invention

[0003] To better meet the above requirements, this application provides a construction method, apparatus, device, medium, and product.

[0004] To achieve the above object, the technical solutions provided in this application are as follows:

[0005] This application provides a construction method, and the method includes: in response to a configuration operation triggered on a configuration page, construct metadata for each layer in the data warehouse according to the configuration information indicated by the configuration operation. The functions implemented by different layers in the data warehouse are different, and the configuration information describes the target data stored in the data warehouse; construct a first precursor graph according to the metadata of each layer in the data warehouse, and the first precursor graph indicates the dependency relationship between the metadata of different layers in the data warehouse; construct a first node according to the first precursor graph, and the first node is used to obtain the target data from the data warehouse.

[0006] In a possible implementation manner, the step of constructing metadata for each layer in the data warehouse according to the configuration information indicated by the configuration operation includes: verifying the configuration information according to a preset configuration rule; in response to the configuration information passing the verification, construct metadata for each layer in the data warehouse according to the configuration information.

[0007] In a possible implementation manner, the method further includes: testing the first node; in response to the first node passing the test, publishing the first node to a pre-production environment for verification; in response to the first node passing the verification, publishing the first node to a production environment.

[0008] In a possible implementation, before testing the first node, the method further includes: running the first node to obtain the running state of the first node; the testing of the first node includes: in response to the running state indicating that the first node runs successfully, testing the first node.

[0009] In a possible implementation, the method further includes: constructing metadata for each layer in at least one analysis layer according to the metadata of each layer in the data warehouse, where different analysis layers implement different data analysis methods; constructing a second precursor graph according to the metadata of each layer in the at least one analysis layer, the second precursor graph indicating the dependency relationship between the metadata of different layers in the at least one analysis layer; constructing a second node according to the second precursor graph, the second node being used to perform data analysis on the target data obtained through the first node.

[0010] In a possible implementation, after constructing the second node according to the second precursor graph, the method further includes: running the second node to obtain the running state of the second node; in response to the running state indicating that the second node executes successfully, publishing the second node to the production environment.

[0011] In a possible implementation, the configuration information includes at least one of the attributes of the target data and the screening conditions of the target data; and / or, the input data of the first node includes the metadata of each layer in the data warehouse, and the output data of the first node includes the target data.

[0012] The present application provides a construction device, including: a first construction unit, configured to construct metadata for each layer in a data warehouse according to configuration information indicated by a configuration operation in response to a configuration operation triggered on a configuration page, where different layers in the data warehouse implement different functions, and the configuration information describes target data stored in the data warehouse; a second construction unit, configured to construct a first precursor graph according to the metadata of each layer in the data warehouse, the first precursor graph indicating the dependency relationship between the metadata of different layers in the data warehouse; a third construction unit, configured to construct a first node according to the first precursor graph, the first node being used to obtain the target data from the data warehouse.

[0013] The present application provides an electronic device, the device including: a processor and a memory; the memory is configured to store instructions or computer programs; the processor is configured to execute the instructions or computer programs in the memory so that the electronic device executes the construction method provided by the present application.

[0014] The present application provides a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs run on a device, the device is caused to execute the construction method provided by the present application.

[0015] The present application provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium. The computer program includes program codes for executing the construction method provided by the present application.

[0016] Compared with the related art, the present application has at least the following advantages:

[0017] In the technical solution provided by the present application, first, in response to a configuration operation triggered on a configuration page, metadata of each layer in the data warehouse is constructed according to the configuration information indicated by the configuration operation (such as information such as the filtering conditions of the target data stored in the data warehouse), so that the metadata can describe the characteristics presented by each layer in the data warehouse when the target data is obtained from the data warehouse (such as tasks to be executed, rules to be followed, processing means to be adopted, etc.), so that the metadata can to a certain extent describe the role played by each layer in the data warehouse in the process of obtaining the target data; then, a first precedence graph (Directed Acyclic Graph, DAG) is constructed based on the metadata, so that the first precedence graph can indicate the dependency relationship between the metadata of different layers in the data warehouse, so that the first precedence graph can to a certain extent describe the usage order presented by each layer in the data warehouse in the process of obtaining the target data; then, a first node is constructed based on the first precedence graph, so that the first node can describe the workflow of obtaining the target data from the data warehouse, so that the first node can describe the processing tasks to be executed by each layer in the data warehouse in the process of obtaining the target data, so that the first node can be used later to automatically extract data (such as the target data) that meets the current analysis requirements from the data warehouse. In this way, it is possible to automatically construct a data production node that meets the user configuration based on the user configuration without manual participation of other personnel, which is beneficial to improving efficiency. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings required for the description of the embodiments or the related art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0019] Figure 1 It is a flowchart of a construction method provided by an embodiment of the present application;

[0020] Figure 2 A schematic diagram of an automation solution implemented based on configuration provided by an embodiment of the present application;

[0021] Figure 3 A schematic diagram of an automation solution provided by an embodiment of the present application;

[0022] Figure 4 A schematic diagram of the release process of a first node provided by an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the release process of a second node provided by an embodiment of the present application;

[0024] Figure 6 A schematic diagram of the structure of a construction device provided by an embodiment of the present application;

[0025] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0026] It has been found through research that the analysis requirements described in the background art can be realized through a multi-party coordination method, which specifically includes: first, the first party (such as business personnel) manually provides data requirements so that the data requirements can describe which data in the data warehouse are needed for the current analysis; then, the second party (such as the R & D personnel of the data warehouse) manually develops each layer in the data warehouse according to the relevant specifications of the data warehouse to obtain metadata so that the metadata can describe the characteristics of the data existing in the data warehouse that meet the requirements (such as what the acquisition process of the data is); then, the third party (such as the R & D personnel of the front end) manually identifies data screening conditions based on the metadata, so that on the one hand, the front-end display page is manually developed so that the page can not only describe the conditions but also display the data that meet the conditions obtained from the data warehouse, and on the other hand, the code document (such as a json document) is manually developed and provided to the fourth party (such as the R & D personnel of the back end) so that the fourth party can manually convert the code document into Structured Query Language (SQL) so that the SQL can represent information such as which table in the data warehouse to read which data, what the data screening conditions are, and finally which table to summarize the data that meet the conditions to; finally, the fifth party (such as testers) manually configures relevant tests for the SQL, such as performing tests on the primary key and whether the field is 0, etc.

[0027] It is also found through research that the solution manually implemented through multi-party coordination shown above has the following defects: Since this solution is implemented manually, a large amount of manpower is required for this solution, resulting in too low return on investment (ROI); and since each link in this solution is manually developed by different personnel, the same data has different names in different links, making the data lineage involved in this solution unclear and difficult to trace, and thus making it difficult for this solution to present a clear and understandable full-linkage lineage, resulting in inefficient problem troubleshooting; also, manual development is prone to bugs, so some hidden problems may occur when publishing the output of this solution to the production environment (such as the online environment), affecting the user experience.

[0028] Based on the above research, in order to better overcome the above defects, the present application provides an automated execution construction method, which includes: First, in response to a configuration operation triggered on the configuration page, according to the configuration information indicated by the configuration operation (such as information such as the screening conditions of the target data stored in the data warehouse), construct the metadata of each layer in the data warehouse, so that these metadata can describe the characteristics presented by each layer in the data warehouse when obtaining the target data from the data warehouse (such as tasks to be executed, rules to be followed, processing means to be adopted, etc.), so that these metadata can to a certain extent describe the role played by each layer in the data warehouse in the process of obtaining the target data; then, construct a first precursor graph based on these metadata, so that the first precursor graph can indicate the dependency relationship between the metadata of different layers in the data warehouse, so that the first precursor graph can to a certain extent describe the usage order presented by each layer in the data warehouse in the process of obtaining the target data; then, construct a first node based on the first precursor graph, so that the first node can describe the workflow of obtaining the target data from the data warehouse, so that the first node can describe the processing tasks to be executed by each layer in the data warehouse in the process of obtaining the target data, so that the first node can be used later to automatically extract data that meets the current analysis requirements (such as the target data) from the data warehouse. In this way, it is possible to automatically construct a data production node that meets the user configuration based on the user configuration without the manual participation of other personnel, which not only helps to improve efficiency but also effectively overcomes the defects of the solution implemented manually through multi-party coordination.

[0029] In addition, the execution subject of the construction method is not limited in this application. For example, the method can be applied to a terminal device or a server. For another example, the method can also be implemented by means of data interaction between the terminal device and the server. Among them, the terminal device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server can be an independent server, a cluster server or a cloud server.

[0030] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. For example, when responding to an active request from the user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information (such as operation behavior information, etc.). Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that executes the operation of the technical solution of the present disclosure according to the prompt message. As an optional but non-limiting implementation manner, for example, when responding to an active request from the user, the manner of sending a prompt message to the user can be in the form of a pop-up window, and the prompt message can be presented in text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It can be understood that the above process of notifying and obtaining the user's authorization is only illustrative and does not limit the implementation manner of the present disclosure, and other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0031] It should be noted that in the embodiments of this application, information such as the stored data in the data warehouse (such as the above-mentioned contact data, target data, or behavior data) does not involve sensitive information of the user. The stored data and other information are obtained and used after the user's authorization, and the analysis results and other information of the stored data are also determined after the user's authorization. In one example, before obtaining the target data or the analysis result of the target data, a prompt message related to obtaining data usage authorization is displayed on the corresponding interface. The prompt message informs the user of the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure in an appropriate manner according to relevant laws and regulations, and the user determines whether to agree to the authorization based on the prompt message. It can be understood that the above process of notifying and obtaining the user's authorization is only illustrative and does not limit the implementation manner of the present disclosure, and other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0032] To enable those skilled in the art to better understand the solution of this application, the following will clearly and completely describe the technical solution in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0033] To better understand the technical solution provided by this application, the following will first describe the construction method provided by this application with reference to some drawings. As Figure 1 shown, the construction method provided by the embodiments of this application includes S1 - S3 below.

[0034] S1: In response to a configuration operation triggered on the configuration page, construct the metadata of each layer in the data warehouse according to the configuration information indicated by the configuration operation. Different layers in the data warehouse implement different functions, and the configuration information describes the target data stored in the data warehouse.

[0035] Among them, the configuration page refers to a page that has been pre - developed and can provide configuration services for users (such as Figure 2 shown business parties), so that the user can configure the data required for current analysis by triggering some operations on this page. It should be noted that this application does not limit the implementation manner of the configuration page. For example, it can be implemented using the Figure 2 shown configuration page.

[0036] The configuration operation refers to an operation triggered by the user on the configuration page that can describe the data acquisition requirements specified by the user, so that the configuration operation can indicate the configuration information provided by the user through this configuration page (such as Figure 2 shown information of "configured behavioral data" or Figure 3 shown information of "behavioral data"), so that the configuration information can describe what characteristics the data required for current analysis has, such as which table and column in the data warehouse it is located in, and what filtering conditions it meets.

[0037] It can be seen that in a possible implementation manner, the configuration information provided by the user through the configuration page can at least meet the following constraints: The configuration information describes the target data (such as behavioral data) stored in the data warehouse, so that the configuration information can indicate the data acquisition requirements presented by the user in the current analysis. Among them, the behavioral data refers to the data obtained through touch points that can describe the interaction operations triggered by the user (such as browsing operations, purchase operations, collection operations, etc.), so that the behavioral data can describe the state of the user on the current interaction interface to a certain extent.

[0038] Furthermore, this application does not limit the implementation of the above configuration information. For example, it can be implemented using any information that can describe the user's data acquisition requirements. For another example, in some scenarios, the configuration information can include metadata of the target data to enable the configuration information to better describe the characteristics of the target data in the data warehouse.

[0039] In addition, this application does not limit the method for obtaining the above-mentioned configuration information. For example, it may specifically include: after the user obtains the data caliber through preliminary research on the data acquisition requirements under the current analysis, the user triggers some configuration operations on the configuration page based on the data caliber, so that the configuration information indicated by these configuration operations includes part or all of the data caliber. Among them, because the data caliber can describe the characteristics of the data required for the current analysis (such as which table and column in the data warehouse is located, what screening conditions are met, etc.), the configuration information determined based on the data caliber can describe the characteristics of the data required for the current analysis as comprehensively and accurately as possible.

[0040] The above-mentioned data warehouse (DW) is a data storage system, and the data warehouse can integrate and store some scattered and heterogeneous data according to certain themes and models, so that the data warehouse can provide support for data analysis, report generation, etc.

[0041] In addition, for the above-mentioned data warehouse, the data warehouse may include multiple layers; and this application does not limit the implementation methods of the multiple layers. For example, in some scenarios, the multiple layers in the data warehouse may include a contact layer, a behavior layer, a batch layer, a product line layer, a contact unit layer, a data flow analysis (DFA) layer, a mart layer, a feature layer, a behavior dimension layer, and part or all of the service address-binding-contract (ABC) layer, so that the multiple layers can describe the specifications followed by the data warehouse in the current application scenario.

[0042] It can be seen that, under one possible implementation, the multiple layers in the above-mentioned data warehouse can be determined based on the data warehouse requirements in the actual application scenario. It should be noted that this application does not limit the implementation method of the data warehouse. For example, it can be implemented using any data storage system. For another example, the data warehouse can be determined based on the actual application scenario.

[0043] In addition, for any layer in the data warehouse, the metadata of that layer describes the layer so that the metadata can describe the characteristics presented by that layer when obtaining target data from the data warehouse (such as tasks to be executed, rules to be followed, processing means to be adopted, etc.), so that the metadata can, to a certain extent, describe what role that layer plays in the process of obtaining the target data. It can be seen that the metadata of each layer in the data warehouse can describe the characteristics of obtaining the target data. It should be noted that this application does not limit the implementation manner of the metadata.

[0044] The target data refers to the data stored in the data warehouse and needed in the current analysis (such as the above-mentioned contact data, behavior data, etc.); moreover, this application does not limit the implementation manner of the target data. For example, the target data can refer to the data stored in the data warehouse that can describe the behavior of triggering the browsing of a target page (such as a shopping page) within 30 days.

[0045] In addition, this application does not limit the implementation manner of S1 above. For example, it can be implemented by means of any method capable of generating the metadata of each layer in the data warehouse based on configuration information, such as pre-constructing a script with this generation function or a pre-trained machine learning model with this generation function, etc. It should be noted that this application does not limit the implementation manner of the script, nor the implementation manner of the machine learning model.

[0046] S2: Construct a first precedence graph according to the metadata of each layer in the data warehouse, and the first precedence graph indicates the dependency relationship between the metadata of different layers in the data warehouse.

[0047] Among them, the first precedence graph refers to a DAG constructed according to the metadata of each layer in the data warehouse (such as the DAG shown in Figure 4 ), so that the DAG can describe these metadata in the form of a workflow, so that the DAG can not only describe what the metadata of each layer in the data warehouse is, but also describe the dependency relationship between the metadata of different layers in the data warehouse (such as the sequence relationship presented when using these metadata), and further enable the DAG to not only describe what role each layer in the data warehouse plays in the process of obtaining the target data, but also to a certain extent describe when each layer in the data warehouse plays a role in the process of obtaining the target data, so that the DAG can better describe how each layer in the data warehouse cooperates to achieve the process of obtaining the target data.

[0048] In addition, the dependency relationship between the metadata of different layers in the data warehouse is determined based on the dependency relationship between different layers in the data warehouse, so that the "dependency relationship between the metadata of different layers in the data warehouse" can describe to a certain extent the framework presented by the data warehouse. It should be noted that this application does not limit the determination process of the "dependency relationship between the metadata of different layers in the data warehouse".

[0049] In addition, this application does not limit the implementation manner of the above S2. For example, it can adopt any method that can construct a predecessor graph based on some data with a dependency relationship, such as the construction method of a first-in-first-out queue for implementation. It can be seen that in a possible implementation manner, the S2 can specifically be: according to the dependency relationship between different layers in the data warehouse, convert the metadata of each layer in the data warehouse into a first-in-first-out queue, so that the queue can describe the front-to-back order indicated by the dependency relationship, so that the queue can be used as the first predecessor graph in the future, so that the first predecessor graph can indicate the dependency relationship.

[0050] S3: Construct a first node according to the first predecessor graph, and the first node is used to obtain target data from the data warehouse.

[0051] Among them, the first node (also known as the behavior data node) is a node constructed according to the first predecessor graph and can describe the implementation process of obtaining target data from the data warehouse, so that the target data can be obtained from the data warehouse with the help of the first node in the future. It can be seen that the first node can describe the acquisition process of the target data, so that the first node can describe the tasks performed by each layer in the data warehouse in the acquisition process of the target data.

[0052] In addition, this application does not limit the implementation manner of S3. For example, it can adopt any method that can construct a node based on a DAG for implementation.

[0053] Based on the relevant content of S1 to S3 above, the technical solution provided by this application that can achieve the automated access of target data (such as behavior data) includes: first, in response to the configuration operation triggered on the configuration page, according to the configuration information indicated by the configuration operation (such as information such as the screening conditions of the target data stored in the data warehouse), construct the metadata of each layer in the data warehouse, so that these metadata can describe the characteristics presented by each layer in the data warehouse when obtaining the target data from the data warehouse (such as tasks to be executed, rules to be followed, processing means to be adopted, etc.), so that these metadata can, to a certain extent, describe the role played by each layer in the data warehouse in the process of obtaining the target data; then, construct a first precursor graph based on these metadata, so that the first precursor graph can indicate the dependency relationship between the metadata of different layers in the data warehouse, so that the first precursor graph can, to a certain extent, describe the usage order presented by each layer in the data warehouse in the process of obtaining the target data; then, construct a first node based on the first precursor graph, so that the first node can describe the workflow of obtaining the target data from the data warehouse, so that the first node can describe the processing tasks required to be executed by each layer in the data warehouse in the process of obtaining the target data, so that the first node can be used later to automatically extract data (such as the target data) that meets the current analysis needs from the data warehouse. In this way, it is possible to automatically construct a data production node that meets the user configuration based on the user configuration without the need for other personnel to participate manually, which not only helps to improve efficiency but also effectively overcomes the defects of the above-mentioned manually implemented solution through multi-party coordination.

[0054] In addition, in a possible implementation manner, in order to improve accuracy, the configuration information provided by the user through the configuration page may include at least one of the attributes of the target data and the screening conditions of the target data, so that the configuration information can more accurately describe the characteristics of the target data stored in the data warehouse, so that the first node constructed based on the configuration information can better obtain the target data from the data warehouse, which is beneficial to better improving the user experience.

[0055] Furthermore, in a possible implementation manner, in order to better improve the data acquisition effect, the above-mentioned first node may at least meet the following constraints: the input data (that is, the input parameters) of the first node includes the metadata of each layer in the data warehouse, and the output data of the first node includes the above-mentioned target data, so that the first node can realize the function of obtaining the target data from the data warehouse under the guidance of the metadata, so that the first node can be used later to better obtain the target data from the data warehouse.

[0056] It has been found through research that since the configuration information is manually provided by the user, there may be some problems with the configuration information (such as problems caused by user carelessness or mistakes), which may cause the first node generated based on the configuration information to be unable to accurately obtain the target data from the data warehouse.

[0057] Based on the above research, in order to better overcome the above problems, the present application also provides a possible implementation manner of the above S1. In this manner, the S1 may specifically include: in response to a configuration operation triggered on the configuration page, verifying the configuration information indicated by the configuration operation according to a preset configuration rule, in order to determine whether the configuration information meets the configuration rule by means of this verification; in response to the configuration information passing the verification, it can be determined that the configuration information meets the configuration rule, so it can be determined that there are no defects such as configuration errors in the configuration information itself. Therefore, the metadata of each layer in the above data warehouse can be constructed based on the configuration information, which can effectively overcome the problems caused by manual configuration errors, thus being beneficial to better improving the data acquisition effect. Among them, the configuration rule describes the constraints that need to be met when the user configures on the configuration page, and the present application does not limit the configuration rule. For example, the configuration rule can be determined according to the actual application scenario.

[0058] It can be seen that in some scenarios, for any user (such as Figure 2 the business party shown), if it is detected that the user has triggered a configuration operation on the configuration page, the configuration information indicated by these configuration operations (such as information such as the metadata of a certain behavior data) can be obtained first; then, according to the configuration rule preset for the configuration page, the configuration information is verified to confirm whether the configuration information meets the configuration rule; if it is detected that the configuration information fails to pass the verification, it can be determined that the configuration information does not meet the configuration rule, so it can be determined that there are some problems with the configuration information (such as missing some information, syntax errors, etc.). Therefore, the result that "the configuration information does not meet the configuration rule" can be informed to the user by displaying corresponding prompt information (such as a pop-up window) on the configuration page, so that the user can adjust the configuration information to overcome the foregoing problems; however, if it is detected that the configuration information passes the verification, it can be determined that the configuration information meets the configuration rule, so it can be determined that there are no problems with the configuration information. Therefore, the metadata of each layer in the above data warehouse can be directly constructed based on the configuration information, so that these metadata can describe the characteristics presented when obtaining the target data from the data warehouse, which can effectively overcome the problems caused by manual configuration errors, thus being beneficial to better meeting the user experience.

[0059] It has been found through research that since the first node is automatically generated, there may be some problems with the first node, which affects the user experience.

[0060] Based on the above research, in order to better improve the user experience, the present application also provides a possible implementation manner of the construction method. In this manner, the construction method may at least include the following steps: testing the above-mentioned first node (such as the data testing shown in Figure 2 or the data testing shown in Figure 3 ), in order to determine whether there is a first error (such as errors like missing fields, data loss, etc.) in the first node by means of this test; in response to the first node passing this test, it can be determined that the first node does not have this first error, so the first node is published to the pre-production environment for verification, in order to determine whether there is a second error (such as errors like being unable to implement the data acquisition function expected by the user, being unable to cooperate with other modules in the project integrated with the first node, etc.) in the first node by means of this verification; in response to the first node passing this verification, it can be determined that the first node does not have this second error, so the first node can be published to the production environment (such as the online environment), so that the target data can be obtained by periodically running the first node in this production environment subsequently, in order to accumulate and precipitate the target data (such as the behavior data existing in the data assets shown in Figure 3 ). In this way, it is possible to implement a variety of verification methods for the first node to ensure the availability of the first node, thereby effectively improving the data acquisition effect.

[0061] It should be noted that the present application does not limit the implementation manner of the above-mentioned test. For example, the test may include at least one test, such as Quality Assurance (QA) test, engineering test, etc., so that the test can confirm as comprehensively as possible whether there is a first error in the first node. Another example is that the test can be determined according to the error troubleshooting requirements in the actual application scenario, in order to determine whether an error indicated by the error troubleshooting requirements appears in the first node by means of this test.

[0062] It should also be noted that the above-mentioned production environment refers to an environment that can provide services to users, and the present application does not limit this production environment. For example, it can be implemented using the online environment.

[0063] It should be noted again that the above-mentioned pre-production environment refers to an environment constructed by simulating the above-mentioned production environment (such as the environment used for pre-production verification shown in Figure 2 , the environment used for data verification shown in Figure 3 , or Figure 4The pre-release environment shown, etc., so that subsequently, the functions of the first node (such as the function of "obtaining target data from the data warehouse" and the function of "cooperating with other modules in the project integrated with the first node", etc.) can be verified with the help of this pre-release environment to confirm whether the functions implemented by the first node can meet the user's expectations.

[0064] It should be further noted that the present application does not limit the implementation manner of the above verification. For example, the verification may include at least one verification, such as checking whether the first node can meet the data acquisition requirements by simulating an interaction interface, and confirming whether the first node can cooperate with other modules in the current project through engineering tests (such as Figure 3 the engineering tests shown), so that the verification can comprehensively confirm whether the functions implemented by the first node meet the expectations as much as possible. Another example is that the verification can be determined according to the functional requirements in the actual application scenario, in order to determine whether the first node meets the functional requirements with the help of this verification.

[0065] Based on the above five paragraphs, for some scenarios, after constructing the first node (such as the node constructed by the control center of the behavior data publishing shown by Figure 3 calling the service management) according to the configuration information provided by the user through the configuration page, some tests (such as QA tests, etc.) are performed on the first node, in order to check whether the first node has a first error (such as missing fields, data loss, etc.) with the help of these tests; if it is detected that the first node fails these tests, it can be determined that the first node has the first error, so the result of "the first node has the first error" can be informed to the user by displaying corresponding prompt information (such as a pop-up window) on the configuration page, so that the user can adjust the configuration information to overcome the first error; however, if it is detected that the first node passes these tests, it can be determined that the first node does not have the first error, so the first node can be released to the pre-release environment, in order to verify whether the first node has a second error in this pre-release environment; if it is detected that the first node fails this verification, it can be determined that the first node has the second error, so the result of "the first node has the second error" can be informed to the user by displaying corresponding prompt information (such as a pop-up window) on the configuration page, so that the user can adjust the configuration information to overcome the second error; however, if it is detected that the first node passes this verification, it can be determined that the first node does not have the second error, so the first node can be directly released to the production environment to realize pushing the first node to the online environment, so that subsequently, the first node can periodically obtain target data from the data warehouse as data assets (such as Figure 3Store, accumulate, and precipitate the data assets shown for future use.

[0066] It has been found through research that for the automatically constructed first node, some tasks in the workflow indicated by the first node may have problems, resulting in the abnormal operation of the first node.

[0067] Based on the above research, in order to better overcome the defects shown in the previous paragraph, the present application also provides a possible implementation manner of the construction method. In this manner, the construction method may at least include the following steps: Run the first node to obtain the running state of the first node (such as the node execution result stored in the message middleware shown), so that the running state can describe whether each task in the workflow indicated by the first node can be successfully executed, thereby enabling the running state to indicate whether the first node can run successfully; In response to the running state indicating that the first node runs successfully, it can be determined that the first node can work normally, so the first node can be tested (such as QA testing, etc.). This is beneficial for timely discovering problems existing in the first node itself, thus better overcoming the defects caused by problems in some tasks in the workflow indicated by the first node, and further beneficial for better improving the automatic construction effect. Figure 4 It should be noted that for the running state of the first node, the running state refers to the execution result obtained by executing the first node, so that the running state includes the execution states of each task in the workflow indicated by the first node (such as states of successful execution or failed execution, etc.), thereby enabling the running state to describe whether each task in the workflow indicated by the first node can be successfully executed, and further enabling the running state to indicate whether the first node can work normally. In addition, the present application does not limit the implementation manner of the "each task". For example, the "each task" may refer to the tasks executed by each layer in the data warehouse.

[0068] It should also be noted that the present application does not limit the acquisition method of the above running state. For example, in some scenarios, it may specifically be: During the running of the first node, write the execution states of each task in the workflow indicated by the first node into the message middleware (RocketMQ), and use the message detector (MetaqListener) to detect the messages in the message middleware, so as to determine that the first node can work normally when the message detector (such as the message detector shown) detects that all tasks in the workflow indicated by the first node are successfully executed.

[0069] Figure 4

[0070] ​​Based on the above three paragraphs, for some scenarios, after constructing the first node according to the configuration information provided by the user through the configuration page (such as the node constructed by the control center for publishing behavior data shown in Figure 3 by calling service management), run the first node (such as the data operation shown in Figure 3 or the execution of the DAG node shown in Figure 4 ), store the running state of the first node in RocketMQ (such as the message middleware shown in Figure 4 ), so that the messages stored in the RocketMQ can describe whether the tasks indicated by the metadata of each layer in the data warehouse are executed successfully, and use MetaqListener to detect the messages in the RocketMQ, so that when the MetaqListener detects that all the tasks indicated by the metadata of each layer in the data warehouse are executed successfully, it can be determined that the first node can run successfully, and thus it can be determined that the first node can work properly. Therefore, in order to further determine the availability of the first node, some tests can be performed on the first node (such as the QA test shown in Figure 4 ), in order to confirm whether there are other problems with the first node by means of these tests. This is conducive to discovering as many problems as possible in the first node before pushing the first node to the online environment, thus facilitating better avoidance of affecting the user experience due to problems in the first node after the first node is launched, and further improving the user experience.

[0071] It has been found through research that in some scenarios, not only is it necessary to obtain data that meets the user's needs, but also some analysis needs to be performed on the data to determine the characteristics of the user who generated the data (such as characteristics in the browsing stage, in the multi-party commodity comparison stage, in the purchase stage, etc.).

[0072] Based on the above research, in order to better meet the analysis requirements shown in the previous paragraph, the present application also provides a possible implementation manner of the construction method. In this manner, the construction method can at least include the following steps: construct the metadata of each layer in at least one analysis layer (such as the model layer and the metric layer shown in Figure 5 ) according to the metadata of each layer in the data warehouse. The data analysis methods implemented by different analysis layers are different; construct a second precursor graph (such as the DAG shown in Figure 5 ) according to the metadata of each layer in the at least one analysis layer. The second precursor graph indicates the dependency relationship between the metadata of different layers in the at least one analysis layer; construct a second node (such as the one shown in Figure 5The DAG node shown), and the second node is used to perform data analysis on the target data obtained through the first node. In this way, not only can the target data be automatically obtained, but also some analyses can be performed based on the target data, which is beneficial to better improving the user experience.

[0073] It should be noted that the above at least one analysis layer can indicate the data analysis methods required when analyzing based on the target data, such as methods implemented by means of models or index calculation methods, etc., so that the at least one analysis layer can describe the analysis requirements for the target data under the current analysis. In addition, for any layer in the at least one analysis layer, the metadata of this layer can describe the characteristics of this layer (such as what analysis means are adopted and what the analysis purpose is, etc.), so that the metadata can describe what kind of analysis can be performed on the target data by means of this layer.

[0074] The above second precursor graph refers to a DAG constructed based on the metadata of each layer in the at least one analysis layer, so that the DAG can describe these metadata in a workflow manner, so that the DAG can not only describe what the metadata of each layer in the at least one analysis layer is, but also describe the dependency relationships between the metadata of different layers in the at least one analysis layer (such as the sequential relationships such as the front-back order when using these metadata), and further enable the DAG to not only describe what roles each layer in the at least one analysis layer plays in the analysis process of the target data, but also enable the DAG to describe to a certain extent when each layer in the at least one analysis layer plays a role in the analysis process of the target data. In this way, the DAG can better describe how each layer in the at least one analysis layer cooperates to implement the analysis process of the target data.

[0075] In addition, the "dependency relationships between the metadata of different layers in the at least one analysis layer" mentioned above are determined based on the dependency relationships between different layers in the at least one analysis layer, so that the "dependency relationships between the metadata of different layers in the at least one analysis layer" can describe the framework presented by the at least one analysis layer to a certain extent. It should be noted that the present application does not limit the determination process of the "dependency relationships between the metadata of different layers in the at least one analysis layer".

[0076] In addition, the present application does not limit the construction method of the above-mentioned second precursor graph. For example, it can adopt any method capable of constructing a precursor graph based on some data with a dependency relationship, such as the construction method of a first-in-first-out queue for implementation. It can be seen that in a possible implementation manner, the process of obtaining the second precursor graph may include: according to the dependency relationship between different layers in the at least one analysis layer, converting the metadata of each layer in the at least one analysis layer into a first-in-first-out queue, so that the queue can describe the order indicated by the dependency relationship, so that the queue can be used as the second precursor graph subsequently, so that the second precursor graph can indicate the dependency relationship.

[0077] Based on the above five paragraphs, it can be known that for some scenarios, after generating the metadata of each layer in the data warehouse according to the configuration information provided by the user through the configuration page (such as the metadata shown by Figure 4 ), it is not only necessary to construct a first node capable of obtaining target data from the data warehouse according to the metadata of each layer in the data warehouse, but also necessary to construct the metadata of each layer in at least one analysis layer according to the metadata of each layer in the data warehouse (such as the metadata shown by Figure 5 ), so that the metadata of each layer in the at least one analysis layer can describe to a certain extent the characteristics of the analysis process of the target data, so that a second precursor graph (such as the DAG shown by Figure 5 ) can be constructed based on the metadata of each layer in the at least one analysis layer subsequently, so that the second precursor graph can describe the dependency relationship between the metadata of different layers in the at least one analysis layer in a workflow manner (such as the dependency relationship shown by Figure 5 ), so that the second node (such as the DAG node shown by Figure 5 ) constructed based on the second precursor graph can describe how to analyze the target data, and further enable the second node to describe how to perform data analysis on the target data obtained through the first node, and enable the second node to indicate the characteristics presented by each layer in the at least one analysis layer in the analysis process of the target data (such as what role to play and when to play the role). In this way, not only can a node with data production function (such as the first node) be automatically constructed, but also a node with data analysis function (such as the second node) can be automatically constructed, which is conducive to better improving the automatic construction effect.

[0078] It has been found through research that for the automatically constructed second node, some tasks in the workflow indicated by the second node may cause problems and lead to abnormal operation of the second node.

[0079] Based on the above research, in order to better overcome the defects shown in the previous paragraph, the present application also provides a possible implementation manner of the construction method. In this manner, the construction method may at least include the following steps: running a second node to obtain the running state of the second node (such as the node execution result stored in the message middleware shown by Figure 5 ), so that the running state can describe whether each analysis task in the workflow indicated by the second node can be successfully executed, thereby enabling the running state to indicate whether the second node can run successfully; in response to the running state indicating that the second node has been successfully executed, it can be determined that the second node can work properly, so the second node can be published to the production environment, so that in the production environment, the second node can perform some analysis on the target data obtained by the first node. This is beneficial to timely discover problems existing in the second node itself, thereby better overcoming the defects caused by problems in some tasks in the workflow indicated by the second node, and further facilitating better improvement of the automatic construction effect.

[0080] It should be noted that the relevant content of the running state of the above second node is similar to the relevant content of the running state of the first node above. For the sake of brevity, it will not be elaborated here.

[0081] Based on the above two paragraphs, for some scenarios, after generating the metadata of each layer in the data warehouse according to the configuration information provided by the user through the configuration page (such as the metadata provided by the analysis data publishing to the project shown by Figure 5 ), the second node can be constructed first according to the metadata of each layer in the data warehouse (such as the DAG node shown by Figure 5 ); then run the second node (such as the execution of the DAG node shown by Figure 5 ), and store the running state of the second node in RocketMQ (such as the message middleware shown by Figure 5 ), so that the message stored in the RocketMQ can describe whether the tasks indicated by the metadata of each layer in the above at least one analysis layer are executed successfully, and use MetaqListener to detect the messages in the RocketMQ, so that when the MetaqListener detects that the tasks indicated by the metadata of each layer in the above at least one analysis layer are all executed successfully, it can be determined that the second node can run successfully, thereby it can be determined that the second node can work properly, and further it can be determined that the second node is in an available state. Therefore, the second node can be directly published to the production environment to realize pushing the second node to the online environment, so that subsequently the second node can periodically perform analysis on the target data obtained by the first node (such as the data shown by Figure 3Analyze the data in the data assets shown to obtain some analysis results, so that these analysis results can be used to perform other tasks subsequently, such as presenting these analysis results to users and other tasks.

[0082] Based on the relevant content of the above construction method, this application provides an automated access solution for target data (such as behavioral data). In this solution, first, the user (such as Figure 2 the business party shown) conducts a data caliber research for a certain analysis requirement (such as the requirement of "triggering the shopping page within 30 days") to obtain a research result, so that the research result can describe the characteristics (such as attributes and / or filtering conditions) presented by the data that meets the analysis requirement in the data warehouse; then, the user performs some configuration operations on the configuration page (such as the configuration page shown by the Figure 2 client in it) according to the research result, so that the configuration information indicated by these configuration operations can describe the research result, so that the configuration information can be used as the metadata of the data that meets the analysis requirement, so that after the contact point is put on the shelf for the configuration information, a series of automated operations are performed by the background corresponding to the configuration page (such as the series of operations of "configured behavioral data → behavioral data production → data testing → pre-release verification" performed by the Figure 2 server in it, or a series of operations shown by Figure 3 etc.) are launched to the product, so that the product (such as the product with Figure 3 the processing flow shown or Figure 4 the product shown) can not only automatically precipitate the lineage relationship between different data, but also perform contact point value evaluation (such as the evaluation shown by Figure 2 ).

[0083] Based on the above content, the automated access solution for target data provided by this application (such as the Figure 2 product solution shown) can help users (such as the Figure 2 business party shown) to automatically produce data and automatically apply the produced data to certain analysis scenarios (such as scenarios of targeting people, generating analysis reports, etc.) by simply configuring on the page, and automatically generate the contact point lineage of the entire link to help relevant personnel (such as R & D personnel) quickly view the application value of some data and / or troubleshoot problematic data.

[0084] In addition, for the automated access solution for target data provided by this application (such as the Figure 2 product solution shown), this solution can be implemented by means of the Figure 3 processing flow shown; and this solution can include the following first step to fifth step.

[0085] Step 1: Based on the data required for the current analysis (such as behavioral data, or the above-mentioned target data, or Figure 3 the behavioral data shown), perform some configuration operations on the configuration page to register the relevant information of this data (such as metadata) according to dimensions and / or metrics through this configuration page (such as Figure 3 the registration shown), and after the registration is completed, obtain the configuration information indicated by these configuration operations (such as the metadata of this data), so that this configuration information can describe the characteristics of the data required for the current analysis (such as which data table it is located in, which column it belongs to, filtering conditions, etc.), so that subsequent interfaces (such as Figure 2 the interface where the client sends information to Figure 2 the server shown, etc.) can verify this configuration information, in order to confirm whether this configuration information meets the pre-set configuration rules by means of this verification (such as Figure 3 the configuration verification shown), so as to confirm whether there are defects such as configuration errors in this configuration information by means of this verification.

[0086] Step 2: After determining that the configuration information passes the verification, this configuration information can be used to call the service management (such as Figure 3 the service management shown or Figure 4 the service management shown) interface to publish the target data (such as Figure 3 the behavioral data publishing shown), to generate the first node of the daily routine (such as the behavioral node or Figure 4 the DAG node shown), so that subsequent data testing of this node can be performed by calling the data testing template of the QA test (such as Figure 3 the data testing shown or Figure 4 the QA test shown), and after determining that this node passes this data test, proceed to the third step below. It should be noted that Figure 3 the workflow engine in it is used to generate the DAG and DAG nodes; the Figure 3 data operation in it refers to asynchronously executing this first node after building this first node to confirm whether this first node can work properly; the Figure 3 project in it is a pre-developed product that can execute any implementation manner of the construction method provided in this application.

[0087] Step 3: After the above-mentioned first node is published, send a function confirmation notice to the user (such as Figure 2 the business party shown), so that this user can perform a mock test on this first node through the pre-production environment, in order to view whether the data acquisition function implemented by this first node can meet their own requirements (such as Figure 3 the function confirmation shown) through the interactive interface in this pre-production environment, so as to perform engineering testing on this first node after confirmation (such asFigure 3 the engineering test shown) to confirm whether the first node can work in cooperation with other modules in the project, so that after determining that the first node successfully passes the engineering test, the first node is released to the production environment to push the first node to go live (such as Figure 2 the official go-live shown or Figure 3 the engineering go-live shown or Figure 4 the engineering go-live shown).

[0088] Step 4: After the successful release of the above first node, the target data (such as behavior data) obtained through the first node can be written into some model layers or the report layer of the performance test (Rational Performance Tester, RPT) for the precipitation of assets (such as Figure 3 the data assets shown), so as to continuously accumulate some behavior data of users for a certain product (such as an application program), so that the product can be better optimized or the application scenario of the product can be extended or better services can be provided to these users based on the accumulated behavior data. In addition, to better implement this step, the release of the model layer and the report layer of the RPT (such as Figure 3 the analysis data release shown or Figure 5 the analysis data release shown) and go-live are also required.

[0089] Step 5: After determining the successful release of the first node and the successful release of the second node (such as the release of the model layer and the report layer of the RPT), conduct engineering tests on these nodes (such as Figure 3 the engineering test shown) so that after confirming that these nodes all pass the engineering test, these nodes are released to the production environment to push these nodes to go live (such as Figure 3 the engineering go-live shown).

[0090] In addition, for the automated access solution of the target data provided in this application (such as Figure 2 the product solution shown or the product solution implemented by Figure 3 ), this solution can at least include the production release process of the behavior node (such as the above first node) shown by Figure 4 and the production release process of the analysis node (such as the second node above) shown by Figure 5 .

[0091] For the production release process of the behavior node (such as the above first node) shown by Figure 4 , this process is implemented by means of the behavior data release interface; and the working principle of the behavior data release interface includes: converting the configuration information (such as the metadata of the above target data) provided by the user through the configuration page into the first object (such asFigure 4 the POJO object shown), to call a project that can implement any implementation of the construction method provided in this application (such as Figure 3 the project shown or Figure 4 the project shown), so that the project can perform encapsulation of the High-Speed Service Framework (HSF) interface for the target data (such as behavior data) indicated by the configuration information, and enable the called project to perform the following steps: first, perform processing such as POJO object format verification and contact behavior uniqueness processing; then generate metadata for each layer (such as the mart layer, feature layer, etc.) in the data warehouse based on the first object; then build a DAG (such as Figure 4 the DAG shown or the above-mentioned first predecessor graph) so that the DAG can describe the dependency relationship between these metadata in a workflow manner (such as Figure 4 the dependency relationship shown); then call the service management (such as Figure 4 the service management shown) interface to build the behavior node (such as the above-mentioned first node or Figure 4 the DAG node shown); then asynchronously execute the node, write the execution result of the node into RocketMQ (such as Figure 4 the message middleware shown), and set a MetaqListener (such as Figure 4 the message detector shown) to detect the messages in the RocketMQ, so that when it is detected that the metadata of each layer can be successfully executed, call the data test interface of the QA test for the node to implement the QA test for the node, and after passing the QA test, the release information of the node (such as Figure 4 the behavior information, node information, etc. shown) can be encapsulated, register the first node in the distributed database management system (DataStax Enterprise, DSE), and ABC enable the creation and release of the metadata, so as to subsequently implement the feedback of the successful release of the behavior node through the interface of the project in a callback manner.

[0092] For the production release process of the analysis node (such as the second node above) shown by Figure 5 , this process is implemented by means of the analysis data release interface; and the working principle of the analysis data release interface is similar to the working principle of the behavior data release interface shown in the previous paragraph. It can be seen that the working principle of the analysis data release interface can at least include: converting the metadata (such as the metadata of each layer in the data warehouse) constructed through the behavior data release interface into a second object (such as Figure 5 the POJO object shown), to call a project that can implement any implementation of the construction method provided in this application (such as Figure 5, so that the project can analyze the target data pointed to by the metadata in the data warehouse (such as Figure 5 The analysis data shown in FIG) is published HSF interface encapsulation, and the project after the call executes the following steps: first, at least one analysis layer (such as FIG) is generated based on the second object Figure 5 The metadata of each layer in the model layer and indicator layer shown in Figure 2 are used to construct a DAG based on these metadata (such as Figure 5 The DAG shown in FIG or the second predecessor diagram above) can be used to describe the dependencies between these metadata in a workflow manner (such as Figure 5 Dependency relationship shown in ); then call the service management according to the dependency relationship (such as Figure 5 The service management) interface shown in FIG is used to build the analysis node (such as the second node mentioned above); the node is then executed asynchronously, and the execution result of the node is written to RocketMQ (such as FIG. Figure 5 Message middleware as shown), and set a MetaqListener (such as Figure 5 The message detector shown in the figure detects the messages in the RocketMQ, so that when it detects that the metadata of each layer can be successfully executed, it can encapsulate the release information of the node, such as the task identifier (taskIdentitydocument, task_ID), behavior identifier (touchIdentity document, touch_ID), status (status), information (Message) and other information, so that the interface of the project can feedback the success of the analysis node release through a callback method.

[0093] It should be noted that Figure 4 or Figure 5 The Decision Check Point (DCP) backend shown is a management tool that performs a structured evaluation on certain information (such as the metadata) through some preset review mechanisms; and this application does not limit the implementation method of the DCP backend.

[0094] It can be seen that this application provides a method for back-end automated production based on configuration information, connecting multiple environments such as data warehouses, engineering, and testing to improve work efficiency.

[0095] In addition, this application defines a unified implementation solution for the behavioral data and analytical data publishing process, namely, from POJO object call → verification → encapsulation → DAG workflow dependency → service management asynchronous call → RocketMQ message middleware → message detector return → asynchronous callback other interfaces, which can better meet the needs of most application scenarios of user contact behavior data.

[0096] In addition, since the technical solution provided by this application is an automated solution, this solution can effectively overcome the defect of too low ROI caused by manual development.

[0097] Furthermore, since the technical solution provided by this application uses the same set of mechanisms to generate metadata for different layers, so that the names of these metadata meet the naming rules described by the mechanisms, so that the same data has the same name in different layers, and then it is possible to trace the lineage relationship between different data based on the name subsequently. In this way, not only can the accurate and comprehensive lineage link be effectively precipitated, but also the efficiency of problem troubleshooting can be effectively improved.

[0098] Moreover, since the technical solution provided by this application is implemented through an automated method, this solution can effectively overcome the problems caused by the mistakes of relevant personnel themselves during manual development; and because there are many problem detection means involved in this solution (such as configuration verification, QA detection, engineering detection, node self-detection, etc.), so that through this solution, problems can be discovered as timely and comprehensively as possible before going online, so as to better overcome the impact on user experience caused by problems after going online, and thus contribute to better improving user experience.

[0099] Based on the construction method provided by the embodiments of this application, the embodiments of this application also provide a construction device. The following will be combined with Figure 6 for explanation and illustration. Among them, Figure 6 is a schematic structural diagram of a construction device provided by the embodiments of this application. It should be noted that for the technical details of the construction device provided by the embodiments of this application, please refer to the relevant content of the above construction method.

[0100] As Figure 6 shown, the construction device 600 provided by the embodiments of this application includes:

[0101] A first construction unit 601, configured to respond to a configuration operation triggered on a configuration page, and construct metadata for each layer in a data warehouse according to the configuration information indicated by the configuration operation. The functions implemented by different layers in the data warehouse are different, and the configuration information describes the target data stored in the data warehouse;

[0102] A second construction unit 602, configured to construct a first precursor graph according to the metadata of each layer in the data warehouse, where the first precursor graph indicates the dependency relationship between the metadata of different layers in the data warehouse;

[0103] A third construction unit 603, configured to construct a first node according to the first precursor graph, where the first node is used to obtain the target data from the data warehouse.

[0104] In a possible implementation manner, the first construction unit 601 is specifically configured to: verify the configuration information according to a preset configuration rule; in response to the configuration information passing the verification, construct the metadata of each layer in the data warehouse according to the configuration information.

[0105] In a possible implementation manner, the construction device 600 further includes:

[0106] A test unit, configured to test the first node;

[0107] A verification unit, configured to, in response to the first node passing the test, publish the first node to a pre-release environment for verification;

[0108] A first publishing unit, configured to, in response to the first node passing the verification, publish the first node to a production environment.

[0109] In a possible implementation manner, the construction device 600 further includes:

[0110] A first running unit, configured to run the first node to obtain the running state of the first node;

[0111] The test unit is specifically configured to: in response to the running state indicating that the first node runs successfully, test the first node.

[0112] In a possible implementation manner, the construction device 600 further includes:

[0113] A fourth construction unit, configured to construct the metadata of each layer in at least one analysis layer according to the metadata of each layer in the data warehouse, where the data analysis methods implemented by different analysis layers are different;

[0114] A fifth construction unit, configured to construct a second precursor graph according to the metadata of each layer in the at least one analysis layer, where the second precursor graph indicates the dependency relationship between the metadata of different layers in the at least one analysis layer;

[0115] A sixth construction unit, configured to construct a second node according to the second precursor graph, where the second node is used to perform data analysis on the target data obtained through the first node.

[0116] In a possible implementation manner, the construction device 600 further includes:

[0117] A second running unit, configured to run the second node to obtain the running state of the second node;

[0118] A second publishing unit, configured to publish the second node to a production environment in response to the running state indicating that the execution of the second node is successful.

[0119] In a possible implementation, the configuration information includes at least one of the attributes of the target data and the filtering conditions of the target data; and / or, the input data of the first node includes the metadata of each layer in the data warehouse, and the output data of the first node includes the target data.

[0120] Based on the relevant content of the above-mentioned construction device 600, the working principle of the device 600 includes: first, in response to a configuration operation triggered on a configuration page, according to the configuration information indicated by the configuration operation (such as information such as the filtering conditions of the target data stored in the data warehouse), construct the metadata of each layer in the data warehouse, so that these metadata can describe the characteristics presented by each layer in the data warehouse when obtaining the target data from the data warehouse (such as tasks to be executed, rules to be followed, processing means to be adopted, etc.), so that these metadata can to a certain extent describe the role played by each layer in the data warehouse in the process of obtaining the target data; then, construct a first precursor graph based on these metadata, so that the first precursor graph can indicate the dependency relationship between the metadata of different layers in the data warehouse, so that the first precursor graph can to a certain extent describe the usage order presented by each layer in the data warehouse in the process of obtaining the target data; then, construct a first node based on the first precursor graph, so that the first node can describe the workflow of obtaining the target data from the data warehouse, so that the first node can describe the processing tasks required to be executed by each layer in the data warehouse in the process of obtaining the target data, so as to be able to use the first node to automatically extract data that meets the current analysis requirements (such as the target data) from the data warehouse in the future. In this way, it is possible to realize the automatic construction of a data production node that meets the user configuration based on the user configuration without the need for other personnel to participate manually, which is beneficial to improving efficiency.

[0121] In addition, an embodiment of the present application further provides an electronic device, where the device includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation manner of the construction method provided by the embodiment of the present application.

[0122] See Figure 7, which shows a schematic structural diagram of an electronic device 700 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0123] As Figure 7 shown, the electronic device 700 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 701, which may perform various appropriate actions and processes according to the programs stored in the read-only memory (ROM) 702 or the programs loaded from the storage device 708 into the random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The input / output (I / O) interface 705 is also connected to the bus 704.

[0124] Generally, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 7 the electronic device 700 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be implemented or had alternatively.

[0125] Specifically, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above functions defined in the methods of the embodiments of the present disclosure are executed.

[0126] The electronic device provided by the embodiments of the present disclosure and the method provided by the above embodiments belong to the same inventive concept. For technical details not described in detail in this embodiment, reference may be made to the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0127] An embodiment of the present application also provides a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs run on a device, the device is caused to execute any implementation manner of the construction method provided by the embodiments of the present application.

[0128] It should be noted that the computer-readable medium in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0129] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0130] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0131] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can perform the method.

[0132] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0134] The units involved in the embodiments described in the present disclosure can be implemented in software or in hardware. Among them, the name of the unit / module does not, in some cases, constitute a limitation on the unit itself.

[0135] The functions described above herein can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), System on a Chip (SOC), Complex Programmable Logic Devices (CPLD), and the like.

[0136] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] It should be noted that the embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For related parts, reference can be made to the description in the method section.

[0138] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or its similar expression means any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0139] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0140] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0141] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A construction method, characterized in that, The method includes: In response to a configuration operation triggered on a configuration page, constructing metadata for each layer in a data warehouse according to the configuration information indicated by the configuration operation, where different layers in the data warehouse implement different functions, and the configuration information describes the target data stored in the data warehouse; Constructing a first precursor graph according to the metadata for each layer in the data warehouse, where the first precursor graph indicates the dependency relationships between the metadata for different layers in the data warehouse; Constructing a first node according to the first precursor graph, where the first node is used to obtain the target data from the data warehouse.

2. The method according to claim 1, wherein The constructing metadata for each layer in the data warehouse according to the configuration information indicated by the configuration operation includes: Verifying the configuration information according to a preset configuration rule; In response to the configuration information passing the verification, constructing metadata for each layer in the data warehouse according to the configuration information.

3. The method according to claim 1, wherein The method further includes: Testing the first node; In response to the first node passing the test, publishing the first node to a pre-production environment for verification; In response to the first node passing the verification, publishing the first node to a production environment.

4. The method according to claim 3, wherein Before testing the first node, the method further includes: Running the first node to obtain the running state of the first node; The testing the first node includes: In response to the running state indicating that the first node runs successfully, testing the first node.

5. The method according to claim 1, characterized in that, The method further includes: Constructing metadata for each layer in at least one analysis layer according to the metadata for each layer in the data warehouse, where different analysis layers implement different data analysis methods; Constructing a second precursor graph according to the metadata for each layer in the at least one analysis layer, where the second precursor graph indicates the dependency relationships between the metadata for different layers in the at least one analysis layer; Constructing a second node according to the second precursor graph, where the second node is used to perform data analysis on the target data obtained through the first node.

6. The method according to claim 5, wherein After constructing the second node according to the second precursor graph, the method further includes: Running the second node to obtain the running state of the second node; In response to the running state indicating that the second node executes successfully, publishing the second node to a production environment.

7. The method according to any one of claims 1 to 6, characterized in that, The configuration information includes at least one of the attributes of the target data and the filtering conditions of the target data; And / or The input data of the first node includes the metadata for each layer in the data warehouse, and the output data of the first node includes the target data.

8. A construction device, characterized in that, Includes: A first construction unit, configured to, in response to a configuration operation triggered on a configuration page, construct metadata for each layer in a data warehouse according to the configuration information indicated by the configuration operation, where different layers in the data warehouse implement different functions, and the configuration information describes the target data stored in the data warehouse; A second construction unit, configured to construct a first precursor graph according to the metadata for each layer in the data warehouse, where the first precursor graph indicates the dependency relationships between the metadata for different layers in the data warehouse; A third construction unit, configured to construct a first node according to the first precursor graph, where the first node is used to obtain the target data from the data warehouse.

9. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is configured to store instructions or computer programs; The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the method according to any one of claims 1-7.

10. A computer-readable medium, characterized in that, Instructions or computer programs are stored in the computer-readable medium, and when the instructions or computer programs run on the device, the device executes the method according to any one of claims 1-7.

11. A computer program product, characterized in that, It includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the method according to any one of claims 1-7.

Images