Data mart management method and device, equipment, storage medium and program product

Abstract

The invention discloses a data mart management method and device, equipment, a storage medium and a program product, and relates to the technical field of data processing. The method comprises the steps that under the condition that first data in a target project is received, data source information and flow path information of the first data are acquired and stored; a first list is generated according to the data source information and the flow path information, the first list comprises content which exists in the data mart and is not needed by the target project, and the content comprises at least one of a model, a program and a script; and cutting the data mart according to the first list. According to the embodiment of the invention, the development efficiency of the data mart can be improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a data mart management method, device, equipment, storage medium and program product. Background Art

[0002] In today's financial technology field, the construction and application of big data are becoming increasingly important. The banking industry needs a flexible and constantly improving data mart to support different fields, different business components, and different business modules to efficiently process and utilize data and maximize the value of data.

[0003] Since large commercial banks can have multiple branches, the procedures for handling financial projects between branches are both unified and different. Therefore, there are large differences between the solutions of each bank, and developers need a lot of time to write programs and modify data marts. The existing customized data mart development method requires a lot of manpower costs and a long implementation cycle, so the development efficiency of data marts is low. Summary of the invention

[0004] The embodiments of the present application provide a data mart management method, apparatus, device, storage medium and program product, which can solve the problem of low development efficiency of existing data marts.

[0005] In a first aspect, an embodiment of the present application provides a data mart management method, the method comprising:

[0006] Upon receiving first data in the target project, acquiring and storing data source information and flow path information of the first data;

[0007] Generate a first list according to the data source information and the flow path information, wherein the first list includes content existing in the data mart and not required by the target project, and the content includes at least one of a model, a program, and a script;

[0008] The data mart is tailored according to the first list.

[0009] In some embodiments, before obtaining the data source information and flow path information of the first data, the method further includes:

[0010] generating a second list according to the data source information and the flow path information, wherein the second list includes content that does not exist in the data mart and is required by the target project;

[0011] The second list is output.

[0012] In some embodiments, the acquiring and storing the data source information of the first data includes:

[0013] When the first data is received, the data source information is obtained, where the data source information includes a business system name and a component name that generate the first data, and a generation time of the first data;

[0014] The data source information is stored in a relational database corresponding to the first data according to preset rules.

[0015] In some embodiments, the relational database includes an API interface, and after storing the data source information in the relational database corresponding to the first data according to a preset rule, the method further includes:

[0016] In case of receiving a data query request sent by a calling module through the API interface, determining a data query condition in the data query request;

[0017] Determine the data query condition to determine the corresponding target source information;

[0018] The target source information is sent to the calling module through the API interface.

[0019] In some embodiments, the acquiring and storing the flow path information of the first data includes:

[0020] Performing a target operation on the first data through a script in the data mart to obtain the first data after the target operation, wherein the target operation includes at least one of data cleaning, data aggregation, and data conversion;

[0021] Acquire operation information associated with the target operation;

[0022] The flow path information is generated and stored according to the operation information.

[0023] In some embodiments, after acquiring and storing the flow path information of the first data, the method further includes:

[0024] generating a flow path map based on the flow path information;

[0025] The flow path diagram is displayed.

[0026] In a second aspect, an embodiment of the present application provides a data mart management device, the device comprising:

[0027] An acquisition module, configured to acquire and store data source information and flow path information of the first data upon receiving the first data in the target project;

[0028] A first generating module, configured to generate a first list according to the data source information and the flow path information, wherein the first list includes content existing in the data mart and not required by the target project, and the content includes at least one of a model, a program, and a script;

[0029] A tailoring module is used to tailor the data mart according to the first list.

[0030] In a third aspect, an embodiment of the present application provides a data mart management device, the device comprising: a processor and a memory storing computer program instructions;

[0031] The above data mart management method is implemented when the processor executes the computer program instructions.

[0032] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the above-mentioned data mart management method is implemented.

[0033] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, the above data mart management method is implemented.

[0034] In the present application, when the first data in the target project is received, the data source information and flow path information of the first data are obtained and stored; a first list is generated based on the data source information and the flow path information, the first list includes the content that exists in the data mart and is not needed by the target project, and the content includes at least one of the models, programs, and scripts; the data mart is trimmed according to the first list. In this way, by obtaining and analyzing the data source and flow path information in the target project, it is possible to accurately identify and generate a list of models, programs, and scripts that are no longer needed, thereby effectively trimming the data mart. In this way, it is possible to automatically reduce redundant content in the data mart, optimize system performance, ensure that the data mart accurately meets the actual needs of the project, and there is no need to manually write code to modify the data mart, thereby improving the development efficiency of the data mart. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1It is a flowchart of a data mart management method provided by an embodiment of the present application;

[0037] Figure 2 It is a structural diagram of a data mart management device provided in an embodiment of the present application;

[0038] Figure 3 is a structural diagram of a data mart management device provided by another embodiment of the present application;

[0039] Figure 4 is a structural diagram of a data mart management device provided in another embodiment of the present application;

[0040] Figure 5 is a structural diagram of a data mart management device provided in another embodiment of the present application;

[0041] Figure 6 is a structural diagram of a data mart management device provided in yet another embodiment of the present application;

[0042] Figure 7 It is a structural diagram of a data mart management device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0044] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0045] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations.

[0046] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0047] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0048] Specifically, in order to solve the problems of the prior art, the embodiments of the present application provide a data mart management method, device, equipment, storage medium and program product. The data mart management method provided by the embodiments of the present application is first introduced below.

[0049] Figure 1 A flow chart of a data mart management method provided by an embodiment of the present application is shown. The method comprises the following steps:

[0050] S110, when first data in the target project is received, acquiring and storing data source information and flow path information of the first data.

[0051] The embodiments of the present application may include Figure 2 The data architecture shown in the figure includes a data warehouse, a data mart, and a business system. The data warehouse is the foundation layer of the entire data architecture. The data warehouse stores all raw data in a centralized manner, including integrated data from multiple business systems or external data sources.

[0052] The data mart is located between the data warehouse and the business system. It can be a small data warehouse dedicated to a specific business field or department. The data mart can extract data from the data warehouse, perform specific processing on the extracted data, and then provide the processed data to the business system for use.

[0053] The business system is located at the top layer of the data architecture, which may include specific applications or system modules, such as customer information components, customer relationship management (CRM) components, and customer marketing components. The business system is directly user-oriented and provides users with various business functions, such as customer information management, customer relationship management, and marketing activity support. In addition, the business system can obtain data from the business mart according to its needs, or send data to the business mart for processing and storage.

[0054] In this embodiment, Figure 2 The data architecture shown is used to implement and complete the target project. Then, when the data mart receives the first data, the data source information and flow path information of the first data can be obtained and stored, wherein the first data can be data obtained from the business system, data obtained from the data warehouse, or data converted based on the data stored in the data warehouse. Among them, the data source information is information associated with the source of the data, and the flow path information is information that characterizes the path that the data passes through in the data mart. Therefore, the data source information and the flow path information include information on all steps of how the data is collected, processed, stored, and used.

[0055] S120, generating a first list according to the data source information and the flow path information, wherein the first list includes content existing in the data mart and not required by the target project, and the content includes at least one of a model, a program, and a script.

[0056] In this embodiment, the first list includes content that is no longer needed in the target project but originally existed in the data mart. In this case, these contents can be cut or moved without affecting the realization of the target project.

[0057] Specifically, the data source information and flow path information of all first data can be obtained, wherein the data source information and flow path information can include data unique identifier, data name, script path, script name, program version and other information. Based on the obtained data source information and flow path information, the dependency and actual usage of each model, program and script in the data mart can be analyzed, and which content is not called or used in the current project can be identified. These unused or called contents are regarded as redundant, so a first list can be generated based on the above unused or called contents, and it is considered that the contents in the first list can be cut.

[0058] S130: Tailor the data mart according to the first list.

[0059] In this embodiment, after the content of the first list is determined, the content such as the data model, program, and script in the first list may be trimmed to remove or simplify this part of the content from the data mart.

[0060] For example, Figure 2 As shown, the data mart may include a metadata management module and a program tailoring module, wherein the metadata management module is used to record and manage information such as the source, processing and path of the data to ensure the consistency and traceability of the data; the program tailoring module is used to generate a first list that needs to be tailored based on the data provided by the metadata management module, and optimize and tailor the content in the data mart to remove unnecessary data and functional modules.

[0061] For example, the business system originally had Figure 2 The customer information component, customer relationship management (CRM) component and customer marketing component shown in the figure, the customer relationship management (CRM) component includes the desktop customer relationship management system (CRM-PC) and the mobile relationship management system (mobile CRM). If during the implementation of the target project, Figure 3 As shown, the customer marketing component and the mobile relationship management system (mobile CRM) in the customer relationship management (CRM) component have been cancelled, so the models, programs and script lists corresponding to the customer marketing component and the mobile relationship management system (mobile CRM) in the customer relationship management (CRM) component in the data mart also need to be tailored.

[0062] That is, a first list can be generated based on the data source information and flow path information of the first data transmitted from the business system to the data mart, and the content in the first list is the model, program and script corresponding to the mobile relationship management system (mobile CRM) in the customer marketing component and the customer relationship management (CRM) component. Afterwards, the content in the first list can be tailored to meet the personalized needs of the target project.

[0063] In an embodiment of the present application, when the first data in the target project is received, the data source information and flow path information of the first data are obtained and stored; a first list is generated based on the data source information and the flow path information, the first list includes the content that exists in the data mart and is not needed by the target project, and the content includes at least one of the models, programs, and scripts; the data mart is trimmed according to the first list. In this way, by obtaining and analyzing the data source and flow path information in the target project, it is possible to accurately identify and generate a list of models, programs, and scripts that are no longer needed, thereby effectively trimming the data mart. In this way, it is possible to automatically reduce redundant content in the data mart, optimize system performance, ensure that the data mart accurately meets the actual needs of the project, and there is no need to manually write code to modify the data mart, thereby improving the development efficiency of the data mart.

[0064] As an optional embodiment, before obtaining the data source information and flow path information of the first data, the method further includes:

[0065] generating a second list according to the data source information and the flow path information, wherein the second list includes content that does not exist in the data mart and is required by the target project;

[0066] The second list is output.

[0067] In this embodiment, after obtaining the data source information and flow path information of all the first data, a second list can also be generated based on the data source information and the flow path information. The second list includes data models, programs and scripts that meet the needs of the target project. If these contents are not originally deployed in the data mart, they need to be added to the second list and the second list is output to the user, who will determine whether the contents in the second list need to be deployed in the data mart.

[0068] Specifically, the data mart can analyze the data source information and flow path information of the first data to determine which data models, programs and scripts are necessary for the project. Then, a second list is generated based on these necessary contents, and the second list is output to the user through voice output or image display output. The user further determines whether the contents in the second list need to be added to the data mart.

[0069] For example, the business system originally had Figure 2 The customer information component, customer relationship management (CRM) component and customer marketing component shown in the figure, the customer relationship management (CRM) component includes the desktop customer relationship management system (CRM-PC) and the mobile relationship management system (mobile CRM). If during the implementation of the target project, Figure 4 As shown, some new functions are added to the mobile relationship management system (mobile CRM) in the customer information component and the customer relationship management (CRM) component. If the data mart lacks models, programs and scripts corresponding to the new functions, a second list can be generated based on the models, programs and scripts corresponding to the new functions.

[0070] That is, a second list can be generated based on the data source information and flow path information of the first data transmitted from the business system to the data mart, and the content in the second list is the models, programs and scripts corresponding to some newly added functions in the customer information component and the mobile relationship management system (mobile CRM) in the customer relationship management (CRM) component. Afterwards, the second list can be output to the user, who decides whether to add the models, programs and scripts corresponding to the newly added functions in the data mart.

[0071] This method analyzes metadata, extracts data models, programs and scripts that meet project requirements, generates a second list and outputs it to users, providing data support for the smooth implementation of the target project.

[0072] As an optional embodiment, the acquiring and storing the data source information of the first data includes:

[0073] When the first data is received, the data source information is obtained, where the data source information includes a business system name and a component name that generate the first data, and a generation time of the first data;

[0074] The data source information is stored in a relational database corresponding to the first data according to preset rules.

[0075] In this embodiment, when the data mart receives the first data sent from the outside, it is necessary to first register the data source information of the first data, and the registered content includes at least the business system name, component name and generation time of the first data of the first data. Then, the registered data source information can be stored in a relational database in the data mart. The relational database contains multiple tables, which organize and manage data through a relational model. That is, data is stored in these tables in the form of rows and columns.

[0076] The data source information storage table shown in Table 1 consists of rows and columns, each row represents a record, and each column represents a data field or attribute.

[0077] Table 1

[0078]

[0079] By using the above method, the data source information is stored in a relational database in a tabular form, which can ensure structured management and efficient query of the data and facilitate the maintenance of data consistency and integrity.

[0080] As an optional embodiment, the relational database includes an API interface, and after storing the data source information in the relational database corresponding to the first data according to a preset rule, the method further includes:

[0081] In case of receiving a data query request sent by a calling module through the API interface, determining a data query condition in the data query request;

[0082] Determine the data query condition to determine the corresponding target source information;

[0083] The target source information is sent to the calling module through the API interface.

[0084] In this embodiment, the relational database can also provide an internal application programming interface (Application Programming Interface, API) interface, which is an application program interface that allows other modules in the data mart to interact with the relational database. Through this interface, other modules can issue data query requests to retrieve the required target source information from the relational database.

[0085] For example, the data query request sent by the calling module in the data mart includes data query conditions, which may include one or more of the data ID, generation time, business system, etc. The relational database can determine the target source information that meets the data query conditions and return the target source information to the API interface. The API interface can format the target source information returned by the database and send the formatted target source information back to the calling module.

[0086] Through the above method, based on the internal API interface, each module can flexibly and efficiently query and obtain the required target source information, thereby supporting data interaction and management within the data mart system.

[0087] As an optional embodiment, the acquiring and storing the flow path information of the first data includes:

[0088] Performing a target operation on the first data through a script in the data mart to obtain the first data after the target operation, wherein the target operation includes at least one of data cleaning, data aggregation, and data conversion;

[0089] Acquire operation information associated with the target operation;

[0090] The flow path information is generated and stored according to the operation information.

[0091] In this embodiment, the data mart also includes program scripts, which are used to perform target operations on the received first data based on pre-set rules to obtain the first data after the target operations. For example, operations such as cleaning, aggregation, and format conversion of the first data are all completed through these scripts. In addition, the processes of these target operations not only need to be executed, but also need to be recorded as operation information associated with the target operations.

[0092] Specifically, the operation information may include necessary information such as the script path, script name, program version, etc. associated with the script that performs the target operation. This information can be obtained by the program script or by inputting. The operation information may also include the frequency and time point of performing the target operation on the first data, so as to analyze the timeliness and reliability of data processing.

[0093] For example, the operation information may be as shown in Table 2, including an identifier of the operation process in the target operation, an operation type, an identifier of input data representing the target operation, an identifier of output data representing the target operation, a frequency of the operation, and a time point when the operation is executed.

[0094] Table 2

[0095]

[0096]

[0097] After performing a target operation on the first data, the first data after the target operation can be obtained. That is, the first data flows within the data mart. After recording the operation information associated with the target operation during the target operation, the flow path information of the first data within the data mart can be obtained and recorded based on the operation information to ensure that the data flow is clear and recordable.

[0098] Specifically, the flow path information may include the data identifier, source model, target model, and transmission mode of the first data. Similarly, the flow path information may also be recorded in a relational database. For example, the flow path information may be as shown in Table 3:

[0099]

[0100] In this way, data quality and consistency can be ensured by cleaning, aggregating and transforming data, while detailed flow path information is generated to facilitate tracking and auditing of data processing. This can improve data accuracy and traceability and support more accurate decision-making and analysis.

[0101] As an optional embodiment, after acquiring and storing the flow path information of the first data, the method further includes:

[0102] generating a flow path map based on the flow path information;

[0103] The flow path diagram is displayed.

[0104] In this embodiment, after the flow path information of the first data in the data mart is obtained, a flow path diagram may be generated based on the flow path information. The flow path diagram may intuitively and graphically display the flow process of the first data in the data mart.

[0105] Through this flow path diagram, users can intuitively understand the entire flow process of data, including the various steps and processing links. It helps users quickly identify and analyze bottlenecks or anomalies in data flow, optimize data processing procedures, and improve system transparency and data management efficiency.

[0106] As an optional embodiment, the data mart may include: Figure 5 The metadata management module and program tailoring module shown. The metadata management module is used to record and manage the source, processing and path of data to ensure the consistency and traceability of data; the program tailoring module is used to generate the first list to be tailored based on the data provided by the metadata management module, and optimize and tailor the content in the data mart to remove unnecessary data and functional modules.

[0107] Specifically, the metadata management module can implement data source component management, transformation process management, and flow path management. Among them, source component management can register the data source and store source information of the data entered into the data mart, and support other modules to query the source information of the data. Transformation process management can include management of transformation rules for data transformation and management of the specific process of transformation. Flow path management can record the flow path of data and generate a path map based on the recorded information. The recorded flow path and path map also support other modules to query information.

[0108] The metadata management module service API serves as an interface between the metadata management module and the program tailoring module, providing unified access to metadata services. The program tailoring module can obtain data source information and flow path information to tailor the content in the data mart.

[0109] Based on the data mart management method provided in the above embodiment, the present application also provides a specific implementation of the data mart management device. Please refer to the following embodiment.

[0110] See first Figure 6 The data mart management device 600 provided in the embodiment of the present application includes the following modules:

[0111] An acquisition module 601 is used to acquire and store data source information and flow path information of the first data when receiving the first data in the target project;

[0112] A first generating module 602, configured to generate a first list according to the data source information and the flow path information, wherein the first list includes content existing in the data mart and not required by the target project, and the content includes at least one of a model, a program, and a script;

[0113] The trimming module 603 is configured to trim the data mart according to the first list.

[0114] When the device receives the first data in the target project, it obtains and stores the data source information and flow path information of the first data; generates a first list based on the data source information and flow path information, the first list includes the content that exists in the data mart and is not needed by the target project, and the content includes at least one of the models, programs and scripts; and trims the data mart according to the first list. In this way, by obtaining and analyzing the data source and flow path information in the target project, it is possible to accurately identify and generate a list of models, programs and scripts that are no longer needed, thereby effectively trimming the data mart. In this way, it is possible to automatically reduce redundant content in the data mart, optimize system performance, ensure that the data mart accurately meets the actual needs of the project, and there is no need to manually write code to modify the data mart, thereby improving the development efficiency of the data mart.

[0115] As an implementation of the present application, the data mart management device 600 may further include:

[0116] A second generating module, configured to generate a second list according to the data source information and the flow path information, wherein the second list includes content that does not exist in the data mart and is required by the target project;

[0117] An output module is used to output the second list.

[0118] As an implementation of the present application, the acquisition module 601 may further include:

[0119] A first acquisition unit is used to acquire the data source information when receiving the first data, wherein the data source information includes a business system name and a component name that generate the first data and a generation time of the first data;

[0120] The first storage unit is used to store the data source information in a relational database corresponding to the first data according to a preset rule.

[0121] As an implementation of the present application, the data mart management device 600 may further include:

[0122] A first determining module, configured to determine a data query condition in the data query request when receiving a data query request sent by a calling module through the API interface;

[0123] A second determination module is used to determine the target source information corresponding to the data query condition;

[0124] A sending module is used to send the target source information to the calling module through the API interface.

[0125] As an implementation of the present application, the acquisition module 601 may further include:

[0126] an execution unit, configured to execute a target operation on the first data through a script in the data mart to obtain the first data after the target operation, wherein the target operation includes at least one of data cleaning, data aggregation and data conversion;

[0127] A second acquisition unit, used to acquire operation information associated with the target operation;

[0128] The second storage unit is used to generate and store the flow path information according to the operation information.

[0129] As an implementation of the present application, the data mart management device 600 may further include:

[0130] A third generating module, configured to generate a flow path diagram based on the flow path information;

[0131] A display module is used to display the flow path diagram.

[0132] The data mart management device provided in the embodiment of the present invention can implement each step in the above method embodiment, and will not be described again here to avoid repetition.

[0133] Figure 7 A schematic diagram of the hardware structure of a data mart management device provided in an embodiment of the present application is shown.

[0134] The management device in the data mart may include a processor 1001 and a memory 1002 storing computer program instructions.

[0135] Specifically, the processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0136] The memory 1002 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1002 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1002 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1002 is a non-volatile solid-state memory.

[0137] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0138] The processor 1001 implements any one of the data mart management methods in the above embodiments by reading and executing computer program instructions stored in the memory 1002 .

[0139] In one example, the management device of the data mart may further include a communication interface 1003 and a bus 1010. Figure 7 As shown, the processor 1001, the memory 1002, and the communication interface 1003 are connected via a bus 1010 and communicate with each other.

[0140] The communication interface 1003 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0141] Bus 1010 includes hardware, software or both, and the parts of the management equipment of the data mart are coupled to each other. For example, but not limitation, the bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In suitable cases, bus 1010 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0142] The management device of the data mart may be based on the above embodiment, thereby realizing the management method and apparatus in combination with the above data mart.

[0143] In addition, in combination with the data mart management method in the above-mentioned embodiment, the embodiment of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the data mart management methods in the above-mentioned embodiments is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here. Among them, the above-mentioned computer-readable storage medium may include non-transitory computer-readable storage media, such as read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), a disk or an optical disk, etc., which are not limited here.

[0144] In addition, an embodiment of the present application also provides a computer program product, including computer program instructions, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0145] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0146] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0147] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0148] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0149] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A data mart management method, characterized in that: The method is applied to a data mart, and the method includes: Upon receiving first data in the target project, acquiring and storing data source information and flow path information of the first data; Generate a first list according to the data source information and the flow path information, wherein the first list includes content existing in the data mart and not required by the target project, and the content includes at least one of a model, a program, and a script; The data mart is tailored according to the first list.

2. The data mart management method according to claim 1, characterized in that: Before obtaining the data source information and flow path information of the first data, the method further includes: generating a second list according to the data source information and the flow path information, wherein the second list includes content that does not exist in the data mart and is required by the target project; The second list is output.

3. The data mart management method according to claim 1, characterized in that: The acquiring and storing the data source information of the first data includes: When the first data is received, the data source information is obtained, where the data source information includes a business system name and a component name that generate the first data, and a generation time of the first data; The data source information is stored in a relational database corresponding to the first data according to preset rules.

4. The data mart management method according to claim 3, characterized in that: The relational database includes an API interface. After storing the data source information in the relational database corresponding to the first data according to a preset rule, the method further includes: In case of receiving a data query request sent by a calling module through the API interface, determining a data query condition in the data query request; Determine the data query condition to determine the corresponding target source information; The target source information is sent to the calling module through the API interface.

5. The data mart management method according to claim 1, characterized in that: The acquiring and storing the flow path information of the first data includes: Performing a target operation on the first data through a script in the data mart to obtain the first data after the target operation, wherein the target operation includes at least one of data cleaning, data aggregation, and data conversion; Acquire operation information associated with the target operation; The flow path information is generated and stored according to the operation information.

6. The data mart management method according to claim 1, characterized in that: After acquiring and storing the flow path information of the first data, the method further includes: generating a flow path map based on the flow path information; The flow path diagram is displayed.

7. A data mart management device, characterized in that: The device comprises: An acquisition module, configured to acquire and store data source information and flow path information of the first data upon receiving the first data in the target project; A first generating module, configured to generate a first list according to the data source information and the flow path information, wherein the first list includes content existing in the data mart and not required by the target project, and the content includes at least one of a model, a program, and a script; A tailoring module is used to tailor the data mart according to the first list.

8. A data mart management device, characterized in that: The data mart management device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the data mart management method according to any one of claims 1 to 6 is implemented.

9. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data mart management method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises computer program instructions, and when the computer program instructions are executed by a processor, the data mart management method according to any one of claims 1 to 6 is implemented.

Images