Data providing method and device, processor and electronic device

By acquiring and constructing a set of target variables, the problem of the data warehouse needing to load data in batches at regular intervals is solved, and real-time interaction and efficient data transmission between the data warehouse and the business system are achieved, meeting the high timeliness requirements of the business system.

CN115017240BActive Publication Date: 2025-09-12INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210770079.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-01
Publication Date
2025-09-12
Estimated Expiration
2042-07-01

AI Technical Summary

Technical Problem

In the existing technology, data warehouses need to load data in batches at regular intervals before providing data to business systems. This results in low efficiency of data warehouses in providing data to business systems and cannot meet the high timeliness requirements of current business systems.

Method used

By acquiring N raw data, building a target variable set, and building a target report based on the target variable set and the corresponding data, and providing it directly to the target business system, real-time interaction and efficient transmission of data are achieved.

Benefits of technology

It improves the efficiency of data warehouses in providing data to business systems and meets the high requirements of business systems for data timeliness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115017240B_ABST
    Figure CN115017240B_ABST
Patent Text Reader

Abstract

The present application discloses a data providing method and device, a processor and an electronic device, and relates to the field of big data. The method comprises: obtaining N raw data, wherein the raw data is the data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1; based on the N raw data, obtaining a target variable set, wherein the target variables in the target variable set are used to represent the field information of the raw data, and any target variable in the target variable set is determined by at least two raw data; constructing a target report based on the target variable set and the data corresponding to the target variables, and providing the target data in the target report to the target business system. Through this application, the problem in the related art that the data warehouse needs to load data in batches at regular intervals before providing data to the business system is solved, resulting in low efficiency of the data warehouse in providing data to the business system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of big data, and more specifically, to a method and device for providing data, a processor, and an electronic device. Background Art

[0002] Currently, data warehouses are the unified, shared foundational data platform for financial services institutions. Furthermore, with the development of various business applications based on data warehouses, the amount of data required to be processed in business systems is increasing, and the requirements for data timeliness in various business systems are becoming increasingly stringent.

[0003] like Figure 1 As shown, in current related technologies, flexible query platforms only support viewing data in the data warehouse and cannot perform any operations. Therefore, when the data warehouse provides data to the business system, the business system application must first view the relevant data through the business system database. The business system database must wait for the data warehouse's scheduled batch load to complete the data update. Therefore, the data warehouse in current related technologies lacks real-time interactive capabilities. In other words, the data warehouse management model that relies on daily scheduled batch loads to provide data to downstream systems has low timeliness and can no longer meet the high timeliness requirements of current business systems.

[0004] Currently, no effective solution has been proposed to the problem that data warehouses in related technologies need to load data in batches at regular intervals before providing data to business systems, resulting in low efficiency in providing data to business systems. Summary of the Invention

[0005] The main purpose of this application is to provide a data providing method and device, processor and electronic device to solve the problem in related technologies that data warehouses need to load data in batches at regular intervals before providing data to business systems, resulting in low efficiency of data warehouses in providing data to business systems.

[0006] To achieve the above-mentioned objectives, according to one aspect of the present application, a data provision method is provided. The method comprises: obtaining N raw data, wherein the raw data is data to be provided to a target business system by a target data warehouse, and N is a positive integer greater than 1; obtaining a target variable set based on the N raw data, wherein the target variables in the target variable set are used to represent field information of the raw data, and any target variable in the target variable set is determined by at least two of the raw data; constructing a target report based on the target variable set and the data corresponding to the target variables, and providing the target data in the target report to the target business system.

[0007] Furthermore, obtaining N pieces of original data includes: obtaining the data source of the original data; determining whether the data required by the data source is real-time data; if the data required by the data source is real-time data, obtaining the N pieces of original data from a target device, wherein the target device is at least one of the following: the target data warehouse, a target database in the target data warehouse, the target database being a database corresponding to a first business system, and the first business system being a business system other than the target business system; if the data required by the data source is not real-time data, obtaining the N pieces of original data from a target file, wherein the target file is a file obtained based on the first data in the target data warehouse.

[0008] Furthermore, the target variable set includes at least one of the following: original variables, derived variables. Based on the N original data, obtaining the target variable set includes: judging whether each original data is processed into the derived variable; if each original data is processed into the derived variable, processing each original data to obtain multiple derived variables; if each original data is not processed into the derived variable, obtaining the original variable corresponding to each original data from the target device or the target file to obtain multiple original variables; summarizing the multiple derived variables or the multiple original variables to obtain the target variable set.

[0009] Furthermore, each original data is processed to obtain multiple derived variables in at least one of the following ways: performing a target operation on each original data to obtain the multiple derived variables; using a target algorithm to process each original data to obtain the multiple derived variables; processing each original data according to a target script to obtain the multiple derived variables.

[0010] Furthermore, constructing a target report based on the target variable set and the data corresponding to the target variables includes: determining the format of the target report based on the target variables in the target variable set; and constructing the target report based on the format of the target report and the data corresponding to the target variables.

[0011] Furthermore, obtaining the original data from the target device includes: determining the type of the target device; determining a first file based on the type of the target device, wherein the first file is at least one of the following: a table file or a view file; obtaining multiple fields in the first file; and filtering the data in the first file based on the multiple fields to obtain the original data.

[0012] Furthermore, before obtaining the original data from the target file, the method also includes: obtaining the first data in the target data warehouse; determining multiple second data based on the first data in the target data warehouse; obtaining the target file based on the multiple second data and the target field, or obtaining the target file based on the multiple second data and the target table structure.

[0013] Furthermore, after constructing a target report based on the target variable set and the data corresponding to the target variables, the method also includes: obtaining target information, wherein the target information is at least one of the following: login information of the target user, operation information of performing target operations on the target data warehouse, and the target user is a user who accesses the target data warehouse; generating a target log based on the target information.

[0014] Furthermore, after constructing a target report based on the target variable set and the data corresponding to the target variables, the method also includes: obtaining a target code, wherein the target code is used to represent the process of performing the target operation on the target data warehouse; and providing the target code to a target object, wherein the target object is an object for developing the target data warehouse.

[0015] To achieve the above-mentioned purpose, according to another aspect of the present application, a data providing device is provided. The device includes: a first acquisition unit, configured to acquire N raw data, wherein the raw data are data to be provided to a target business system by a target data warehouse, and N is a positive integer greater than 1; a first determination unit, configured to obtain a target variable set based on the N raw data, wherein the target variables in the target variable set are used to represent field information of the raw data, and any target variable in the target variable set is determined by at least two of the raw data; and a first processing unit, configured to construct a target report based on the target variable set and the data corresponding to the target variables, and provide the target data in the target report to the target business system.

[0016] Furthermore, the first acquisition unit includes: a first acquisition module, used to obtain the data source of the N original data; a first judgment module, used to judge whether the data required by the data source is real-time data; a second acquisition module, used to obtain the N original data from a target device if the data required by the data source is real-time data, wherein the target device is at least one of the following: the target data warehouse, the target database in the target data warehouse, the target database is a database corresponding to the first business system, and the first business system is a business system other than the target business system; a third acquisition module, used to obtain the N original data from a target file if the data required by the data source is not real-time data, wherein the target file is a file obtained based on the first data in the target data warehouse.

[0017] Furthermore, the target variable set includes at least one of the following: original variables, derived variables, and the first determination unit includes: a second judgment module, used to judge whether each original data is processed into the derived variable; a first processing module, used to process each original data if each original data is processed into the derived variable, to obtain multiple derived variables; a fourth acquisition module, used to obtain the original variable corresponding to each original data from the target device or the target file if each original data is not processed into the derived variable, to obtain multiple original variables; the first determination module is used to summarize the multiple derived variables or the multiple original variables to obtain the target variable set.

[0018] Furthermore, the first processing module is at least one of the following: a first operation module, used to perform target operation on each original data to obtain the multiple derived variables; a second processing module, used to process each original data using a target algorithm to obtain the multiple derived variables; a third processing module, used to process each original data according to a target script to obtain the multiple derived variables.

[0019] Furthermore, the first processing unit includes: a second determination module, used to determine the format of the target report based on the target variables in the target variable set; and a first construction module, used to construct the target report based on the format of the target report and the data corresponding to the target variables.

[0020] Furthermore, the second acquisition module includes: a first determination submodule, used to determine the type of the target device; a second determination submodule, used to determine the first file based on the type of the target device, wherein the first file is at least one of the following: a table file or a view file; a first acquisition submodule, used to obtain multiple fields in the first file; and a first screening submodule, used to screen the data in the first file based on the multiple fields to obtain the original data.

[0021] Furthermore, the device also includes: a second acquisition unit, used to acquire the first data in the target data warehouse before acquiring the original data from the target file; a second determination unit, used to determine multiple second data based on the first data in the target data warehouse; a third determination unit, used to obtain the target file based on the multiple second data and the target field, or, to obtain the target file based on the multiple second data and the target table structure.

[0022] Furthermore, the device also includes: a third acquisition unit, used to obtain target information after constructing a target report based on the target variable set and the data corresponding to the target variables, wherein the target information is at least one of the following: login information of the target user, operation information of performing target operations on the target data warehouse, and the target user is a user who accesses the target data warehouse; a first generation unit, used to generate a target log based on the target information.

[0023] Furthermore, the device also includes: a fourth acquisition unit, used to obtain a target code after constructing a target report based on the target variable set and the data corresponding to the target variables, wherein the target code is used to represent the process of performing the target operation on the target data warehouse; a first providing unit, used to provide the target code to a target object, wherein the target object is an object for developing the target data warehouse.

[0024] In order to achieve the above-mentioned object, according to another aspect of the present application, a processor is provided, wherein the processor is configured to run a program, wherein the program executes any one of the above-mentioned methods for providing data when running.

[0025] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an electronic device is provided, which includes one or more processors and a memory, and the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned data providing methods.

[0026] Through this application, the following steps are adopted: obtaining N raw data, wherein the raw data is the data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1; obtaining a target variable set based on the N raw data, wherein the target variables in the target variable set are used to represent the field information of the raw data, and any target variable in the target variable set is determined by at least two raw data; constructing a target report based on the target variable set and the data corresponding to the target variables, and providing the target data in the target report to the target business system, thereby solving the problem in the related art that the data warehouse needs to load data in batches at regular intervals before providing data to the business system, resulting in low efficiency of the data warehouse in providing data to the business system. By obtaining a variable set based on the N raw data obtained from the data warehouse, and constructing a report based on the variable set and the data corresponding to each variable, and then providing the data in the report to the business system, the efficiency of the data warehouse in providing data to the business system is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0028] Figure 1 It is a schematic diagram of a data warehouse providing data to a business system according to existing technology;

[0029] Figure 2 is a flow chart of a method for providing data according to an embodiment of the present application;

[0030] Figure 3 This is a schematic diagram of a data warehouse providing data to a business system in an embodiment of the present application;

[0031] Figure 4 is a schematic diagram of a data warehouse real-time service platform used in an embodiment of the present application;

[0032] Figure 5 This is a flow chart of the data access management module (interface management) used in an embodiment of the present application;

[0033] Figure 6 This is a flowchart of using the variable management module in an embodiment of the present application;

[0034] Figure 7 is a flow chart of the use data output management module in an embodiment of the present application;

[0035] Figure 8 is a flowchart of an optional data providing method provided according to an embodiment of the present application;

[0036] Figure 9 is a schematic diagram of a data providing device according to an embodiment of the present application;

[0037] Figure 10 is a schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display and analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0042] The present invention will be described below in conjunction with preferred implementation steps. Figure 2 is a flow chart of a method for providing data according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0043] Step S201: Obtain N original data, wherein the original data is data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1.

[0044] For example, Figure 3 As shown, by building a data warehouse real-time service platform (hereinafter referred to as the "real-time service platform") that provides direct access, derivative processing, and real-time docking, we can achieve real-time transmission and management of data access, derivative processing, and data output. Specifically, the data warehouse real-time service platform acts as a bridge between the data warehouse and the business system application end through real-time interface calls. Furthermore, the real-time service platform primarily consists of a user management module, a data access management module, a variable management module, a data output management module, a log management module, an interface call monitoring module, and other modules.

[0045] in addition, Figure 4 Schematic diagram of the use of a data warehouse real-time service platform in an embodiment of the present application, such as Figure 4 As shown, business system A and business system B respectively send data requests to the real-time service platform, and then the real-time service platform distributes the requests to the data warehouse and applies for data. The data warehouse responds to the request and provides data to the real-time service platform, and then the real-time service platform returns the results to business system A and business system B respectively.

[0046] Therefore, in this embodiment, when the business system sends a data request, it first obtains data from the data warehouse through the data access management module in the real-time service platform.

[0047] Step S202 : obtaining a target variable set based on N original data, wherein the target variables in the target variable set are used to represent field information of the original data, and any target variable in the target variable set is determined by at least two original data.

[0048] For example, the data acquired through the data access management module is sent to the variable management module, and multiple variables corresponding to the data can be obtained, that is, multiple fields corresponding to the data can be obtained.

[0049] Step S203 : constructing a target report based on the target variable set and the data corresponding to the target variables, and providing the target data in the target report to the target business system.

[0050] For example, the obtained variables are sent to the data output management module, which can build a report based on the obtained variables and the data corresponding to the variables, and then provide the data in the report to the business system that sent the data request.

[0051] Through the above steps S201 to S203, a variable set is obtained based on the N original data obtained from the data warehouse, and a report is constructed based on the variable set and the data corresponding to each variable, and then the data in the report is provided to the business system, thereby achieving the effect of improving the efficiency of the data warehouse in providing data to the business system.

[0052] In order to quickly and accurately obtain N original data, in the data providing method provided in the embodiment of the present application, N original data can also be obtained through the following steps: obtaining the data source of the N original data; determining whether the data required by the data source is real-time data; if the data required by the data source is real-time data, obtaining N original data from the target device, wherein the target device is at least one of the following: a target data warehouse, a target database in the target data warehouse, the target database is a database corresponding to the first business system, and the first business system is a business system other than the target business system; if the data required by the data source is not real-time data, obtaining N original data from the target file, wherein the target file is a file obtained based on the first data in the target data warehouse.

[0053] For example, the real-time service platform can support two types of data access: API (Application Programming Interface) access (interface management) and file data import (file management). In addition, the data access management module includes centralized management of data sources accessed through the interface and data sources imported through files, including functions such as query, add, edit, enable, disable, approve, delete, and copy.

[0054] When entering the Data Access Management module, first select or import the required data as a data source. If the data source requires real-time data, access it through the data warehouse interface. This also allows integration of real-time data accessed through other interfaces. If the data source requires local user input, it can be uploaded via file import as a data source. Furthermore, the above-mentioned real-time data is the raw data in the data warehouse, i.e., data that has not been processed in any way. If the data requires processing, the raw data in the data warehouse can be processed and stored in a local file.

[0055] In summary, through interface management and file management, the data to be provided to the business system can be accurately obtained.

[0056] In order to enable the data warehouse to quickly provide data to the business system, how to obtain the original data from the target device is also one of the key points. In the data providing method provided in the embodiment of the present application, how to obtain the original data from the target device is further limited. The following technical features can be used to obtain the original data from the target device: determine the type of the target device; determine the first file based on the type of the target device, wherein the first file is at least one of the following: a table file or a view file; obtain multiple fields in the first file; and filter the data in the first file based on the multiple fields to obtain the original data.

[0057] For example, the interface management module can realize the access to the data source in the form of front-end configuration of the interface. The interface management module is used for the centralized management of the data source accessed in the form of the interface. Figure 5 This is a flow chart of the use of the data access management module (interface management) in the embodiment of the present application, such as Figure 5 As shown, the specific steps are:

[0058] Step 1: The user connects to the database through the API interface.

[0059] Step 2: Select an interface management function (using Add Interface as an example);

[0060] Step 3: Select the database type (common database types for data warehouses include mart layer, summary layer, model layer, etc.);

[0061] Step 4: Select the table or view under the database;

[0062] Step 5: Select the required fields in the table or view. The primary key field is a required field by default.

[0063] Step 6: Add the table or view query conditions for filtering control;

[0064] Step 7: Return real-time data;

[0065] Step 8: Enter the variable management module.

[0066] Among them, when the interface management module is in use, the function of the newly added interface can also be established on the existing interface, and operations such as reuse and editing can be performed on it to form a newly added interface.

[0067] In addition, the database type selection, table or view selection, field selection, etc. in the entire interface management module can all be operated through JSON (a drop-down box technology) or XML (another drop-down box technology).

[0068] Through the above solution, real-time data can be obtained from the data warehouse quickly and accurately.

[0069] In order to obtain the target file quickly and accurately, in the data providing method provided in the embodiment of the present application, the target file can also be obtained through the following steps: obtaining the first data in the target data warehouse; determining multiple second data based on the first data in the target data warehouse; obtaining the target file based on the multiple second data and the target field, or obtaining the target file based on the multiple second data and the target table structure.

[0070] For example, the real-time service platform supports directly creating tables through configuration on the front-end page and inserting data from files into the tables. The data in the tables can be integrated and exported for use on the real-time service platform. Data access can also be achieved by quickly importing file data. File data includes data imported into the real-time service platform through bin, txt, excel, csv, and other files. In addition, when data access is file data import, it can be achieved through the following two methods:

[0071] Method 1: You can define each field in the table to be created. That is, users can first customize the entry fields, create the fields, and then import the data (bin or txt format);

[0072] Method 2: You can directly import the entire table content, that is, import the table structure and data, and choose to upload a file (Excel or CSV format).

[0073] Moreover, the data imported by users through the file management module mainly includes real-time business data (real-time parameter information, etc.), data that needs to be customized, data for simulation tests, etc. In addition, these data imported by users can be integrated with the real-time data returned by the API interface.

[0074] In summary, through the visual and rapid configuration of the front-end page, efficient access to multiple data sources can be achieved, while reducing code development, thereby improving data access efficiency.

[0075] In order to enable the data warehouse to quickly provide data to the business system, how to obtain a target variable set based on N original data is also one of the key points. In the data providing method provided in the embodiment of the present application, the target variable set obtained based on N original data is further limited. The target variable set obtained based on N original data can be implemented by using the following technical features: judging whether each original data is processed into a derived variable; if each original data is processed into a derived variable, processing each original data to obtain multiple derived variables; if each original data is not processed into a derived variable, obtaining the original variable corresponding to each original data from the target device or target file to obtain multiple original variables; summarizing multiple derived variables or multiple original variables to obtain a target variable set.

[0076] For example, a variable is a standard data field that can be exported externally after the real-time service platform integrates data from multiple data sources. The variable management module is used for centralized management of variables, including functions such as query, add, edit, view, enable, disable, approve, call detail view, and delete. In addition, variables include original variables and derived variables. Among them, original variables are the original variables obtained in the data access management module (interface return or file import); derived variables are further processed and derived from variables, and original variables can be derived or derived multiple times.

[0077] Figure 6 This is a flow chart of using the variable management module in an embodiment of the present application, such as Figure 6 As shown, the specific steps are:

[0078] Step 1: The real-time data obtained by the data access management module is used as the data source;

[0079] Step 2: The data source is processed through the variable management module;

[0080] Step 3: Choose whether to process it as a derived variable. If "No", skip to step 5; if "Yes", go to step 4;

[0081] Step 4: Process the variables to obtain derived variables;

[0082] Step 5: After processing all variables, the required variable results are obtained;

[0083] Step 6: Enter the data output management module.

[0084] Through the above solution, multiple variables can be quickly obtained based on the acquired data, thus paving the way for the subsequent report construction.

[0085] In order to quickly and accurately obtain derived variables, in the data providing method provided in the embodiment of the present application, derived variables can also be obtained through the following steps: performing a target operation on each original data to obtain multiple derived variables; using a target algorithm to process each original data to obtain multiple derived variables; processing each original data according to a target script to obtain multiple derived variables.

[0086] For example, there are three ways to process derived variables: (1) directly performing simple operations on the original variables to obtain derived variables; (2) supporting the import of stored procedures and other algorithms to process derived variables; and (3) supporting the import of program scripts to process derived variables.

[0087] In summary, variables can be processed through a variety of methods to obtain derived variables, thereby improving the processing efficiency of variables.

[0088] In order to enable the data warehouse to quickly provide data to the business system, how to construct a target report based on the target variable set and the data corresponding to the target variables is also one of the key points. In the data providing method provided in the embodiment of the present application, the construction of the target report based on the target variable set and the data corresponding to the target variables is further limited. The construction of the target report based on the target variable set and the data corresponding to the target variables can be achieved by using the following technical features: determining the format of the target report based on the target variables in the target variable set; and constructing the target report based on the format of the target report and the data corresponding to the target variables.

[0089] For example, the data output management module can automatically generate reports based on custom output variables and display them directly on the front-end page. It can also automatically generate API documentation to connect to business systems and form an interface connection. In addition, based on the interface, the business system can call variables on the real-time service platform.

[0090] Figure 7 This is a flowchart of the data output management module in the embodiment of the present application. Figure 7 As shown, the specific steps are:

[0091] Step 1: The original variables obtained by the variable management module and the derived variables calculated are used as variable results;

[0092] Step 2: User-defined output report format: including uploading table structure, modifying variable names, etc.

[0093] Step 3: Automatically generate result reports through the data output management module;

[0094] Step 4: You can enter other modules to derive more functions, connect to other business systems through interfaces, and display them directly on the front page.

[0095] To sum up, by generating reports and presenting them visually, it is easy to provide data to the business system.

[0096] In order to quickly and accurately generate the target log, in the data providing method provided in the embodiment of the present application, the target log can also be generated by the following steps: obtaining target information, wherein the target information is at least one of the following: login information of the target user, operation information of performing target operations on the target data warehouse, and the target user is the user who accesses the target data warehouse; generating the target log based on the target information.

[0097] For example, the log management module can be used to record user logins and the operation and use of various modules in the real-time service platform, and generate logs. Specifically, the module first obtains the login information of users accessing the data warehouse and the operation information of the above-mentioned operations (operations in the user management module, data access management module, variable management module, and data output management module) on the data warehouse, and then records the obtained information to generate a log.

[0098] Through the above solution, the subsequent tracing can be facilitated based on the generated logs.

[0099] In order to manage the version of the data warehouse, how to quickly simulate the release of the data warehouse is also one of the key points. In the data providing method provided in the embodiment of the present application, after building the target report based on the target variable set and the data corresponding to the target variable, how to provide the code to the developer can be implemented using the following technical features: obtaining the target code, wherein the target code is used to represent the process of performing the target operation on the target data warehouse; providing the target code to the target object, wherein the target object is the object for developing the target data warehouse.

[0100] For example, as a further derivative module of the real-time service platform, it mainly includes the simulation release function, that is, the user integrates the entire process from the data acquisition process of the data access management module, to the variable processing method of the variable management module, and then to the customized result report content of the data output management module, and automatically generates a simulation release through code, and the code can be directly provided to developers (the target objects mentioned above).

[0101] Through the above solution, the R&D process can be accelerated, thereby improving R&D efficiency.

[0102] In addition, the real-time service platform also includes a user management module and an interface call monitoring module. The following describes these two modules separately.

[0103] Among them, the user management module can realize the management of various system users, permissions, resources and other modules, and provide a unified login entrance. In addition, the real-time service platform with unified login provides a portal homepage, through which users can enter different system interfaces. At the same time, other relevant report data of the platform can be directly viewed on the portal homepage. In addition, user permission management is implemented through the permission tools of the data warehouse itself. Each user is assigned to a different user group. Users and user groups have specific permissions. Each user group can also contain multiple sub-user groups. The permissions of users and user groups are the union of all permissions of the user group to which they belong and their own individual permissions. This permission includes constraints based on a certain attribute of the data warehouse, access rights to data content, usage rights, etc.

[0104] Interface call monitoring monitors the calls to various interfaces on the real-time service platform, providing visual monitoring graphs and reports across multiple dimensions. Users can switch between viewing monitoring graphs and reports, query existing interfaces on the real-time service platform, identify unique service identifiers, and perform invocation time periods. Users can also view an overview of the total number of calls, average response time, call failure rate, and number of failures under the query conditions. They can also generate statistics for interface calls, including the system exception rate, number of exceptions, and the top 5 distribution of call durations.

[0105] Through the method provided in the embodiments of the present application, for example, Figure 8 is a flow chart of an optional data providing method according to an embodiment of the present application, such as Figure 8 As shown, the process of the optional data provision method can be divided into the following steps:

[0106] Step 1: The user logs in to the real-time service platform and enters the user management module;

[0107] Step 2: Use permission control to limit the databases, tables, views, logs, and other information that users can access;

[0108] Step 3: Enter the data access management module and select or import the required data as the data source;

[0109] Step 4: If the data source requires real-time data, access it through the data warehouse database through the interface. It can also integrate real-time data accessed from other interfaces. If the data source requires users to provide it locally, it can be uploaded through file import as a data source.

[0110] Step 5: Enter the variable management module and generate original variables or derived variables based on the acquired data source through various algorithms;

[0111] Step 6: Enter the data output management module, and the user will get the final report based on the customized output variables. The report can be directly displayed on the front page;

[0112] Step 7: Record all operation processes in the log management module; record the interface calls used in this operation in the interface call monitoring module for subsequent statistics; users can directly view reports on the front-end display interface; and can also directly connect to other business systems through the interface for further real-time data transmission;

[0113] Furthermore, other modules, as derivatives of the data warehouse real-time service platform, can further expand upon this invention's functionality with more functionality. In particular, the simulated release function allows users to select data sources and import data independently in the data access management module, process various variables (including imported algorithms) in the variable management module, and finally obtain the desired report using a custom output format. This process automatically generates code, packages the generated version, and performs a simulated release. This can be used by business personnel for subsequent verification or directly provided to developers, accelerating R&D and significantly improving efficiency.

[0114] Furthermore, the real-time service platform utilizes a mature and advanced microservices architecture to support multiple functional modules, including data collection, interface invocation, data management, data export, and service monitoring. This component-based approach provides capabilities for microservice development and deployment, service registration and discovery, service governance, and service operations and maintenance. Because the real-time service platform needs to provide real-time data services to multiple business systems simultaneously, it requires a standard service interface, mature service governance capabilities, and efficient agile development techniques.

[0115] Furthermore, to ensure real-time service platform security, the platform utilizes a public key system based on asymmetric key encryption algorithms, establishing a certificate issuance, management, and usage system to support and implement identity authentication, information encryption, data integrity, and non-repudiation within the network system. Regarding data access, strict access control is implemented for all database entities, granting each database user the minimum permissions necessary for their specific functions. All operations performed by privileged users, such as DBA accounts, are auditable to prevent abuse of privileges. To prevent data loss, the database utilizes a variety of backup strategies and redundancy solutions, including database replica sets, full backups, incremental backups, and database log backups.

[0116] In summary, by building an easily scalable, intelligent, and personalized real-time service platform based on data warehouse applications and opening information interfaces with various business systems, we can ensure complete and real-time connectivity of information flows from the data warehouse system to the business systems. Furthermore, this solution can achieve the following:

[0117] (1) It provides a new service platform for the service diversification and data real-time of the data warehouse system, greatly improving the efficiency of data asset utilization.

[0118] The design of the data warehouse real-time service platform integrates the existing data warehouse system with the business system, breaking the traditional method of data development and service through batch loading operations. It features instant communication, convenient development, easy-to-understand interface, and practical functions, providing users with intuitive data processing and flexible report generation.

[0119] (2) The data warehouse real-time service platform is scalable and can derive more functions on the platform.

[0120] The development workload of a data warehouse system is enormous, requiring a comprehensive technical platform to support it and reduce unnecessary duplication of work for developers. Simulated release functionality, as a key derivative feature, will be a significant technological breakthrough for both data warehouse applications and other applications.

[0121] In summary, the data providing method provided by the embodiment of the present application obtains N original data, wherein the original data is the data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1; based on the N original data, a target variable set is obtained, wherein the target variables in the target variable set are used to represent the field information of the original data, and any target variable in the target variable set is determined by at least two original data; a target report is constructed based on the target variable set and the data corresponding to the target variables, and the target data in the target report is provided to the target business system, which solves the problem in the related art that the data warehouse needs to load data in batches at regular intervals before providing data to the business system, resulting in low efficiency of the data warehouse in providing data to the business system. By obtaining a variable set based on the N original data obtained from the data warehouse, and constructing a report based on the variable set and the data corresponding to each variable, and then providing the data in the report to the business system, the efficiency of the data warehouse in providing data to the business system is improved.

[0122] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0123] The present application also provides a data providing device. It should be noted that the data providing device of the present application can be used to execute the data providing method provided in the present application. The data providing device provided in the present application is introduced below.

[0124] Figure 9 Schematic diagram of a data providing device according to an embodiment of the present application. Figure 9 As shown, the device includes: a first acquiring unit 901 , a first determining unit 902 and a first processing unit 903 .

[0125] Specifically, the first acquisition unit 901 is configured to acquire N original data, wherein the original data is data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1;

[0126] A first determining unit 902 is configured to obtain a target variable set based on the N original data, wherein the target variables in the target variable set are used to represent field information of the original data, and any target variable in the target variable set is determined by at least two original data;

[0127] The first processing unit 903 is configured to construct a target report based on the target variable set and the data corresponding to the target variables, and provide the target data in the target report to the target business system.

[0128] In summary, the data providing device provided by the embodiment of the present application obtains N original data through the first acquisition unit 901, wherein the original data is the data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1; the first determination unit 902 obtains a target variable set based on the N original data, wherein the target variables in the target variable set are used to represent the field information of the original data, and any target variable in the target variable set is determined by at least two original data; the first processing unit 903 constructs a target report based on the target variable set and the data corresponding to the target variables, and provides the target data in the target report to the target business system, solving the problem in the related art that the data warehouse needs to load data in batches at regular intervals before providing data to the business system, resulting in low efficiency of the data warehouse in providing data to the business system. By obtaining a variable set based on the N original data obtained from the data warehouse, and constructing a report based on the variable set and the data corresponding to each variable, and then providing the data in the report to the business system, the efficiency of the data warehouse in providing data to the business system is improved.

[0129] Optionally, in the data providing device provided in the embodiment of the present application, the first acquisition unit includes: a first acquisition module, used to obtain the data source of N original data; a first judgment module, used to judge whether the data required by the data source is real-time data; a second acquisition module, used to obtain N original data from the target device if the data required by the data source is real-time data, wherein the target device is at least one of the following: a target data warehouse, a target database in the target data warehouse, the target database is a database corresponding to the first business system, and the first business system is a business system other than the target business system; a third acquisition module, used to obtain N original data from the target file if the data required by the data source is not real-time data, wherein the target file is a file obtained based on the first data in the target data warehouse.

[0130] Optionally, in the data providing device provided in the embodiment of the present application, the target variable set includes at least one of the following: original variables, derived variables, and the first determination unit includes: a second judgment module, used to determine whether each original data is processed into a derived variable; a first processing module, used to process each original data if each original data is processed into a derived variable, to obtain multiple derived variables; a fourth acquisition module, used to obtain the original variable corresponding to each original data from the target device or target file if each original data is not processed into a derived variable, to obtain multiple original variables; the first determination module is used to summarize multiple derived variables or multiple original variables to obtain the target variable set.

[0131] Optionally, in the data providing device provided in the embodiment of the present application, the first processing module is at least one of the following: a first operation module, used to perform target operation on each original data to obtain multiple derived variables; a second processing module, used to process each original data using a target algorithm to obtain multiple derived variables; a third processing module, used to process each original data according to a target script to obtain multiple derived variables.

[0132] Optionally, in the data providing device provided in the embodiment of the present application, the first processing unit includes: a second determination module, used to determine the format of the target report based on the target variables in the target variable set; and a first construction module, used to construct the target report based on the format of the target report and the data corresponding to the target variables.

[0133] Optionally, in the data providing device provided in the embodiment of the present application, the second acquisition module includes: a first determination submodule, used to determine the type of the target device; a second determination submodule, used to determine the first file based on the type of the target device, wherein the first file is at least one of the following: a table file or a view file; a first acquisition submodule, used to obtain multiple fields in the first file; and a first screening submodule, used to screen the data in the first file based on multiple fields to obtain the original data.

[0134] Optionally, in the data providing device provided in the embodiment of the present application, the device also includes: a second acquisition unit, used to obtain the first data in the target data warehouse before obtaining the original data from the target file; a second determination unit, used to determine multiple second data based on the first data in the target data warehouse; a third determination unit, used to obtain the target file based on the multiple second data and the target field, or, to obtain the target file based on the multiple second data and the target table structure.

[0135] Optionally, in the data providing device provided in the embodiment of the present application, the device also includes: a third acquisition unit, used to obtain target information after constructing a target report based on the target variable set and the data corresponding to the target variables, wherein the target information is at least one of the following: login information of the target user, operation information for performing target operations on the target data warehouse, and the target user is a user who accesses the target data warehouse; a first generation unit, used to generate a target log based on the target information.

[0136] Optionally, in the data providing device provided in the embodiment of the present application, the device also includes: a fourth acquisition unit, used to obtain the target code after constructing the target report based on the target variable set and the data corresponding to the target variables, wherein the target code is used to represent the process of performing the target operation on the target data warehouse; a first providing unit, used to provide the target code to the target object, wherein the target object is the object for developing the target data warehouse.

[0137] The data providing device includes a processor and a memory. The above-mentioned first acquisition unit 901, first determination unit 902 and first processing unit 903 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0138] The processor contains a kernel, which retrieves the corresponding program unit from memory. You can configure one or more kernels, and by adjusting kernel parameters, you can improve the efficiency of the data warehouse in providing data to the business system.

[0139] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0140] An embodiment of the present invention provides a storage medium on which a program is stored. When the program is executed by a processor, the data providing method is implemented.

[0141] An embodiment of the present invention provides a processor, which is used to run a program, wherein the data providing method is executed when the program is run.

[0142] like Figure 10As shown, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: obtaining N raw data, wherein the raw data are data to be provided to a target business system by a target data warehouse, and N is a positive integer greater than 1; obtaining a target variable set based on the N raw data, wherein the target variables in the target variable set are used to represent field information of the raw data, and any target variable in the target variable set is determined by at least two of the raw data; constructing a target report based on the target variable set and the data corresponding to the target variables, and providing the target data in the target report to the target business system.

[0143] When the processor executes the program, the following steps are also implemented: obtaining N original data, including: obtaining the data source of the N original data; determining whether the data required by the data source is real-time data; if the data required by the data source is real-time data, obtaining the N original data from a target device, wherein the target device is at least one of the following: the target data warehouse, the target database in the target data warehouse, the target database is a database corresponding to a first business system, and the first business system is a business system other than the target business system; if the data required by the data source is not real-time data, obtaining the N original data from a target file, wherein the target file is a file obtained based on the first data in the target data warehouse.

[0144] When the processor executes the program, the following steps are also implemented: the target variable set includes at least one of the following: original variables, derived variables, and based on the N original data, obtaining the target variable set includes: judging whether each original data is processed into the derived variable; if each original data is processed into the derived variable, processing each original data to obtain multiple derived variables; if each original data is not processed into the derived variable, obtaining the original variable corresponding to each original data from the target device or the target file to obtain multiple original variables; summarizing the multiple derived variables or the multiple original variables to obtain the target variable set.

[0145] When the processor executes the program, the following steps are also implemented: each original data is processed to obtain multiple derived variables in at least one of the following ways: performing a target operation on each original data to obtain the multiple derived variables; using a target algorithm to process each original data to obtain the multiple derived variables; processing each original data according to a target script to obtain the multiple derived variables.

[0146] When the processor executes the program, the following steps are also implemented: constructing a target report package based on the target variable set and the data corresponding to the target variables includes: determining the format of the target report based on the target variables in the target variable set; and constructing the target report based on the format of the target report and the data corresponding to the target variables.

[0147] When the processor executes the program, the following steps are also implemented: obtaining the original data from the target device includes: determining the type of the target device; determining a first file based on the type of the target device, wherein the first file is at least one of the following: a table file or a view file; obtaining multiple fields in the first file; and filtering the data in the first file based on the multiple fields to obtain the original data.

[0148] When the processor executes the program, the following steps are also implemented: before obtaining the original data from the target file, the method also includes: obtaining the first data in the target data warehouse; determining multiple second data based on the first data in the target data warehouse; obtaining the target file based on the multiple second data and the target field, or obtaining the target file based on the multiple second data and the target table structure.

[0149] When the processor executes the program, the following steps are also implemented: after constructing a target report based on the target variable set and the data corresponding to the target variables, the method also includes: obtaining target information, wherein the target information is at least one of the following: login information of the target user, operation information of performing target operations on the target data warehouse, and the target user is the user who accesses the target data warehouse; generating a target log based on the target information.

[0150] When the processor executes the program, the following steps are also implemented: after constructing a target report based on the target variable set and the data corresponding to the target variables, the method also includes: obtaining a target code, wherein the target code is used to represent the process of performing the target operation on the target data warehouse; providing the target code to a target object, wherein the target object is an object for developing the target data warehouse.

[0151] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0152] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: obtaining N original data, wherein the original data is data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1; based on the N original data, obtaining a target variable set, wherein the target variables in the target variable set are used to represent the field information of the original data, and any target variable in the target variable set is determined by at least two of the original data; constructing a target report based on the target variable set and the data corresponding to the target variables, and providing the target data in the target report to the target business system.

[0153] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: obtaining N original data including: obtaining the data source of the N original data; judging whether the data required by the data source is real-time data; if the data required by the data source is real-time data, obtaining the N original data from a target device, wherein the target device is at least one of the following: the target data warehouse, the target database in the target data warehouse, the target database is a database corresponding to a first business system, and the first business system is a business system other than the target business system; if the data required by the data source is not real-time data, obtaining the N original data from a target file, wherein the target file is a file obtained based on the first data in the target data warehouse.

[0154] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: the target variable set includes at least one of the following: original variables, derived variables, and based on the N original data, obtaining the target variable set includes: judging whether each original data is processed into the derived variable; if each original data is processed into the derived variable, processing each original data to obtain multiple derived variables; if each original data is not processed into the derived variable, obtaining the original variable corresponding to each original data from the target device or the target file to obtain multiple original variables; summarizing the multiple derived variables or the multiple original variables to obtain the target variable set.

[0155] When executed on a data processing device, it is also suitable for executing an initialized program having the following method steps: processing each original data to obtain multiple derived variables in at least one of the following ways: performing a target operation on each original data to obtain the multiple derived variables; using a target algorithm to process each original data to obtain the multiple derived variables; processing each original data according to a target script to obtain the multiple derived variables.

[0156] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: constructing a target report based on the target variable set and the data corresponding to the target variables, including: determining the format of the target report based on the target variables in the target variable set; and constructing the target report based on the format of the target report and the data corresponding to the target variables.

[0157] When executed on a data processing device, it is also suitable for executing an initialized program having the following method steps: obtaining the original data from the target device includes: determining the type of the target device; determining a first file based on the type of the target device, wherein the first file is at least one of the following: a table file or a view file; obtaining multiple fields in the first file; and filtering the data in the first file based on the multiple fields to obtain the original data.

[0158] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: before obtaining the original data from the target file, the method also includes: obtaining the first data in the target data warehouse; determining multiple second data based on the first data in the target data warehouse; obtaining the target file based on the multiple second data and the target field, or obtaining the target file based on the multiple second data and the target table structure.

[0159] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: after constructing a target report based on the target variable set and the data corresponding to the target variables, the method also includes: obtaining target information, wherein the target information is at least one of the following: login information of a target user, operation information for performing a target operation on the target data warehouse, and the target user is a user who accesses the target data warehouse; generating a target log based on the target information.

[0160] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: after constructing a target report based on the target variable set and the data corresponding to the target variables, the method also includes: obtaining a target code, wherein the target code is used to represent the process of performing the target operation on the target data warehouse; providing the target code to a target object, wherein the target object is an object for developing the target data warehouse.

[0161] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0162] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0165] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0166] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0167] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0168] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for providing data, characterized in that: include: Obtain N original data, wherein the original data is data to be provided to the target business system by the target data warehouse, and N is a positive integer greater than 1; Based on the N original data, a target variable set is obtained, wherein the target variables in the target variable set are used to represent field information of the original data, and any target variable in the target variable set is determined by at least two of the original data; Building a target report based on the target variable set and the data corresponding to the target variables, and providing the target data in the target report to the target business system; Obtaining N pieces of original data includes: obtaining a data source of the N pieces of original data; determining whether the data required by the data source is real-time data; if the data required by the data source is real-time data, obtaining the N pieces of original data from a target device; if the data required by the data source is not real-time data, obtaining the N pieces of original data from a target file, wherein the real-time data is accessed by accessing the target data warehouse through an interface, or integrating the real-time data accessed from an interface other than the interface, the real-time data being the original data in the target data warehouse, and if data needs to be processed, processing the original data in the target data warehouse and storing the processed data in a local file; The target device is at least one of the following: the target data warehouse, a target database in the target data warehouse, the target database is a database corresponding to a first business system, and the first business system is a business system other than the target business system; the target file is a file obtained based on the first data in the target data warehouse; Before obtaining the original data from the target file, the method further includes: Acquire first data in the target data warehouse; Determining a plurality of second data based on the first data in the target data warehouse; The target file is obtained based on the plurality of second data and the target field, or the target file is obtained based on the plurality of second data and the target table structure.

2. The method according to claim 1, characterized in that The target variable set includes at least one of the following: original variables and derived variables. Based on the N original data, the target variable set includes: Determining whether to process each raw data into the derived variable; If each original data is processed into the derived variable, each original data is processed to obtain multiple derived variables; If each original data is not processed into the derived variable, then obtaining the original variable corresponding to each original data from the target device or the target file to obtain multiple original variables; The multiple derived variables or the multiple original variables are aggregated to obtain the target variable set.

3. The method according to claim 2, characterized in that Each raw data is processed to obtain multiple derived variables in at least one of the following ways: Performing a target operation on each original data to obtain the plurality of derived variables; Processing each raw data using a target algorithm to obtain the multiple derived variables; Each raw data is processed according to the target script to obtain the multiple derived variables.

4. The method according to claim 1, wherein Constructing a target report based on the target variable set and the data corresponding to the target variables includes: Determining the format of the target report based on the target variables in the target variable set; The target report is constructed based on the format of the target report and the data corresponding to the target variable.

5. The method according to claim 1, wherein Acquiring the raw data from the target device includes: determining the type of the target device; Determining a first file according to the type of the target device, wherein the first file is at least one of the following: a table file or a view file; Obtain multiple fields in the first file; The data in the first file is screened according to the multiple fields to obtain the original data.

6. The method according to claim 1, characterized in that After constructing a target report based on the target variable set and the data corresponding to the target variables, the method further includes: Acquire target information, wherein the target information is at least one of the following: login information of a target user, and operation information of a target operation performed on the target data warehouse, wherein the target user is a user accessing the target data warehouse; Generate a target log based on the target information.

7. The method according to claim 6, characterized in that After constructing a target report based on the target variable set and the data corresponding to the target variables, the method further includes: Obtaining a target code, wherein the target code is used to represent a process of performing the target operation on the target data warehouse; The target code is provided to a target object, wherein the target object is an object for developing the target data warehouse.

8. A data providing device, characterized in that: include: A first acquisition unit is configured to acquire N original data, wherein the original data is data to be provided to a target business system by a target data warehouse, and N is a positive integer greater than 1; a first determining unit, configured to obtain a target variable set based on the N original data, wherein target variables in the target variable set are used to represent field information of the original data, and any target variable in the target variable set is determined by at least two of the original data; a first processing unit, configured to construct a target report based on the target variable set and the data corresponding to the target variables, and provide the target data in the target report to the target business system; The first acquisition unit includes: a first acquisition module, configured to acquire a data source of the N original data; A first judgment module is used to judge whether the data required by the data source is real-time data; a second acquisition module for acquiring the N raw data from a target device if the data required by the data source is real-time data; and a third acquisition module for acquiring the N raw data from a target file if the data required by the data source is not real-time data, wherein the real-time data is accessed by accessing the target data warehouse through an interface, or integrating the real-time data accessed through an interface other than the interface, and the real-time data is the raw data in the target data warehouse. If data needs to be processed, the raw data in the target data warehouse is processed and the processed data is stored in a local file. The target device is at least one of the following: the target data warehouse, a target database in the target data warehouse, the target database is a database corresponding to a first business system, and the first business system is a business system other than the target business system; the target file is a file obtained based on the first data in the target data warehouse; The device further includes: a second acquiring unit, configured to acquire first data in the target data warehouse before acquiring the original data from the target file; A second determining unit, configured to determine a plurality of second data based on the first data in the target data warehouse; The third determining unit is configured to obtain the target file according to the plurality of second data and the target field, or to obtain the target file according to the plurality of second data and the target table structure.

9. A processor, characterized in that: The processor is configured to run a program, wherein the program, when running, executes the data providing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The device comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data providing method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Report file generation method and device

    CN104615751A

  • Data warehousing method and device

    CN107463610A

  • Data pushing system

    CN113641644A