Index calculation method, platform, storage medium and program product

Abstract

The invention discloses an index calculation method and platform, a storage medium and a program product, and relates to the technical field of data processing.The index calculation platform responds to a data source configuration operation, obtains the format and address of a data source, and determines index calculation logic based on the format of the data source; it can be understood that the index calculation platform is provided with different index calculation logics for data sources of different formats; moreover, the index calculation platform can directly obtain the target data source based on the address of the data source, and a user does not need to manually import the data source; the index calculation platform can calculate a preset index of the target data source based on the index calculation logic to obtain an index calculation result; namely, for data sources of different formats, special tools or software do not need to be used for calculation, a user does not need to import the data sources of different formats when using different tools, and the convenience of index calculation can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an indicator calculation method, platform, storage medium and program product. Background Art

[0002] Currently, in the process of calculating indicators, for data sources in different formats, it may be necessary to use special tools or software to calculate them respectively, and users are required to manually import data sources in different formats when using different tools, resulting in low convenience in indicator calculation. Summary of the invention

[0003] The main purpose of this application is to provide an indicator calculation method, platform, storage medium and program product, aiming to solve the technical problem of low convenience of indicator calculation.

[0004] To achieve the above purpose, the present application proposes an indicator calculation method, which is applied to an indicator calculation platform, and the method comprises:

[0005] In response to the data source configuration operation, the format of the data source and the address of the data source are acquired, and based on the format of the data source, the indicator calculation logic is determined;

[0006] Based on the address of the data source, a target data source is obtained, and based on the indicator calculation logic, a preset indicator of the target data source is calculated to obtain an indicator calculation result.

[0007] In one embodiment, when the target data source is real-time data, the step of calculating the preset index of the target data source based on the index calculation logic to obtain the index calculation result includes:

[0008] Based on the indicator calculation logic, a calculation unit is determined, wherein the preset indicator includes an indicator to be calculated corresponding to the calculation unit;

[0009] The calculation unit calculates the to-be-calculated index of the real-time data to obtain the index calculation result.

[0010] In one embodiment, the step of determining a calculation unit based on the indicator calculation logic includes:

[0011] Obtain the business scenario to which the target data source belongs;

[0012] Based on the indicator calculation logic and the business scenario to which the target data source belongs, a calculation unit is determined.

[0013] In one embodiment, the preset indicator includes at least one target indicator. When the target data source is offline data, the step of calculating the preset indicator of the target data source based on the indicator calculation logic to obtain the indicator calculation result includes:

[0014] Based on the indicator calculation logic, generate a workflow corresponding to the target indicator;

[0015] Execute the workflow corresponding to the target indicator to obtain the calculation result of the indicator to be calculated, where the workflow is the process of calculating the target indicator of the offline data.

[0016] In one embodiment, the step of generating a workflow corresponding to a target indicator based on the indicator calculation logic includes:

[0017] When the data sources corresponding to at least two target indicators are the same, determining, based on the indicator calculation logic, a common flow on which at least two of the target indicators depend;

[0018] Based on the indicator calculation logic and the common flow, generating at least two workflows corresponding to the target indicators;

[0019] Wherein, when executing the workflows corresponding to at least two of the target indicators, the common flow is only executed once.

[0020] In one embodiment, before the step of calculating the preset index of the target data source based on the index calculation logic and obtaining the index calculation result, the step further includes:

[0021] Obtain the business scenario to which the target data source belongs;

[0022] Determine whether there is a corresponding first data table for the indicator to be stored in the business scenario;

[0023] If it does not exist, a second data table corresponding to the indicator to be stored is established based on the business scenario, and the first data table and the second data table are respectively used to store the indicator calculation results corresponding to the indicator to be stored.

[0024] In one embodiment, when establishing the second data table, an indicator code is generated, and after the step of calculating the preset indicator of the target data source based on the indicator calculation logic and obtaining the indicator calculation result, the step further includes:

[0025] In response to the indicator query operation, based on the target indicator code corresponding to the indicator to be queried and the target data table corresponding to the target indicator code, the indicator calculation result corresponding to the indicator to be queried is searched.

[0026] In addition, to achieve the above objectives, the present application also proposes an indicator calculation platform, which is used to:

[0027] In response to the data source configuration operation, the format of the data source and the address of the data source are acquired, and based on the format of the data source, the indicator calculation logic is determined;

[0028] Based on the address of the data source, a target data source is obtained, and based on the indicator calculation logic, a preset indicator of the target data source is calculated to obtain an indicator calculation result.

[0029] In addition, to achieve the above-mentioned purpose, the present application also proposes an indicator calculation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the indicator calculation method described above.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the indicator calculation method described above are implemented.

[0031] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the indicator calculation method described above are implemented.

[0032] One or more technical solutions proposed in this application have at least the following technical effects:

[0033] The indicator calculation platform of the present application responds to the data source configuration operation, obtains the format of the data source and the address of the data source, and determines the indicator calculation logic based on the format of the data source; it can be understood that the indicator calculation platform is designed with different indicator calculation logics for data sources of different formats; and, the indicator calculation platform of the present application can directly obtain the target data source based on the address of the data source, without the need for the user to manually import the data source; the indicator calculation platform can calculate the preset indicators of the target data source based on the indicator calculation logic to obtain the indicator calculation results; that is, for data sources of different formats, there is no need to use special tools or software for calculation respectively, and there is no need for users to import data sources of different formats when using different tools, which can improve the convenience of indicator calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1A flowchart of the first embodiment of the method for calculating the index of this application is provided;

[0037] Figure 2 A schematic diagram of the first scenario provided in Example 1 of the indicator calculation method of this application;

[0038] Figure 3 A schematic diagram of a second scenario provided in Example 1 of the indicator calculation method of this application;

[0039] Figure 4 A schematic diagram of the third scenario provided in Example 1 of the indicator calculation method of this application;

[0040] Figure 5 A flow chart of the second embodiment of the method for calculating the index of this application;

[0041] Figure 6 A schematic diagram of the fourth scenario provided in Example 2 of the indicator calculation method of this application;

[0042] Figure 7 A schematic diagram of the fifth scenario provided in Example 2 of the indicator calculation method of this application;

[0043] Figure 8 A flowchart of the third embodiment of the method for calculating the index of this application is provided;

[0044] Fig. 9 A schematic diagram of the sixth scenario provided in Example 3 of the index calculation method of this application;

[0045] Fig.10 A schematic diagram of the device structure of the hardware operating environment involved in the indicator calculation method in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an indicator calculation platform, etc. The following takes the indicator calculation platform as an example to illustrate this embodiment and the following embodiments.

[0050] Based on this, the present application embodiment provides an indicator calculation method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the indicator calculation method of the present application.

[0051] In this embodiment, the index calculation method includes steps S10 to S20:

[0052] Step S10, in response to the data source configuration operation, obtaining the format of the data source and the address of the data source, and determining the indicator calculation logic based on the format of the data source;

[0053] It should be noted that the indicator calculation method of this embodiment is applied to an indicator calculation platform, which includes modules such as indicator calculation, indicator storage and indicator query. Figure 1 .

[0054] Currently, in the process of calculating indicators, for data sources in different formats, it may be necessary to use special tools or software to calculate them respectively, and users are required to manually import data sources in different formats when using different tools, resulting in low convenience in indicator calculation.

[0055] In order to solve the above problem, this embodiment proposes an indicator calculation platform. When a user needs to perform indicator calculation, the indicator calculation can be automatically performed through the indicator calculation platform.

[0056] Specifically, the user can trigger the data source configuration operation in the interface of the indicator calculation platform, for example, by clicking the data source configuration button in the interface or clicking the location of the data source configuration, etc. It can be understood that after the user triggers the data source configuration operation, the data source configuration interface can pop up in the interface, and the user can enter the address of the data source to be calculated and the format of the data source based on the prompts in the data source configuration interface.

[0057] Furthermore, the indicator calculation module of the indicator calculation platform obtains the format of the data source and the address of the data source in response to the data source configuration operation, and determines the indicator calculation logic based on the format of the data source.

[0058] It should be noted that the format of the data source can be kafka (Apache Kafka), SFTP (Secure File Transfer Protocol), HIVE (Apache Hive), Mysql (a relational database management system), Oracle (Oracle Database), ES (Elasticsearch, an open source search and analysis engine based on Lucene), etc. The address of the data source is the location of the data source where the user needs to calculate the indicator, such as the corresponding address of the data table storing the data source.

[0059] Since data sources in different formats may need to be calculated using special tools or software respectively, this embodiment does not need to use special tools or software for calculation respectively. The indicator calculation module of the indicator calculation platform only needs to determine different indicator calculation logics based on the formats of different data sources. That is, the indicator calculation logic corresponding to data sources in different formats can be used in the indicator calculation platform, and the indicator calculation logic is integrated in the same indicator calculation platform to improve user convenience.

[0060] Step S20, based on the address of the data source, obtain the target data source, and based on the indicator calculation logic, calculate the preset indicator of the target data source to obtain the indicator calculation result.

[0061] Since users are required to manually import data sources of different formats when using different tools, this embodiment does not require users to manually import the data sources that need to be calculated. Instead, the indicator calculation module of the indicator calculation platform only needs to obtain the target data source based on the address of the data source, thereby improving user convenience.

[0062] Furthermore, the indicator calculation module of the indicator calculation platform calculates the preset indicators of the target data source based on the indicator calculation logic to obtain the indicator calculation results, thereby realizing the automation of indicator calculation.

[0063] It should be noted that the preset indicator can be an indicator that needs to be calculated and is input by the user through the indicator configuration interface, or it can be a pre-configured indicator that needs to be calculated for different data sources.

[0064] Specifically, different business scenarios correspond to different types of indicators, such as financial indicators, health indicators, project management indicators, etc. Among them, financial indicators can be profit margin, total asset turnover rate, debt-to-asset ratio, etc.

[0065] In this embodiment, before the step of calculating the preset index of the target data source based on the index calculation logic and obtaining the index calculation result, the following feasible implementation methods may also exist:

[0066] Obtain the business scenario to which the target data source belongs; determine whether there is a corresponding first data table for the indicator to be stored under the business scenario; if not, establish a second data table corresponding to the indicator to be stored based on the business scenario, wherein the first data table and the second data table are respectively used to store the indicator calculation results corresponding to the indicator to be stored.

[0067] It should be noted that the specific needs of users for indicator queries are taken into consideration: for minute-level data, users usually only need to query the data of the last half month or month, and the query rate for minute-level data in other time periods is lower; and for summary data at the hour, day, month and year levels, there are also different corresponding query needs.

[0068] Therefore, the index calculation platform of this embodiment adopts storage methods such as scene sub-library, data classification, data storage cycle classification, and regular adjustment to improve the utilization rate of the index calculation platform for index capacity. Figure 3 .

[0069] Specifically, regarding the storage method of scene sub-libraries:

[0070] After completing the data source configuration operation, the indicator calculation platform obtains the business scenario to which the target data source belongs; determines whether there is a corresponding first data table for the indicator to be stored under the business scenario; if not, then based on the business scenario, establishes a second data table corresponding to the indicator to be stored, the first data table and the second data table are respectively used to store the indicator calculation results of the corresponding indicator to be stored; if it exists, then after the indicator calculation results of the indicator to be stored are calculated, the indicator to be stored of the indicator to be stored is stored in the first data table.

[0071] Among them, business scenarios include customer service call scenarios, corporate financial scenarios, etc., and the indicator to be stored can be at least one indicator among the preset indicators.

[0072] It can be understood that the indicator calculation results corresponding to the customer service call scenario can be stored in data table 1, and the indicator calculation results corresponding to the customer service call scenario can be stored in data table 2; in order to facilitate standardized storage and improve query efficiency.

[0073] Furthermore, regarding the storage method of scene sub-libraries:

[0074] The indicator calculation results calculated by the indicator calculation platform are composed of minute-level data aggregated by real-time indicators and day-level data corresponding to offline indicator jobs; users can query indicators by minute, hour, day, and month; when creating a data table, the indicator calculation platform can store data corresponding to the same business scenario by minute, hour, day, and month respectively, and the summary calculation module of the indicator calculation platform can regularly complete the statistical summary of indicator calculation results by minute, hour, day, and month.

[0075] Furthermore, regarding the storage method of data retention period classification and regular adjustment:

[0076] The data table can be set with a data retention period, and the data table can automatically clean up the data outside the preset retention period, which can effectively save storage resources; because users have different requirements for query periods of minutes, hours, days, and months for different indicators, the preset retention period can be set based on different indicators, and because the amount of data corresponding to the indicator calculation results of different indicators is different, the capacity of the data table allocated to different indicators is also different.

[0077] Among them, when the preset data retention period of the data table does not meet user needs, the indicator calculation platform can dynamically adjust it. When the actual data volume of the indicator calculation result corresponding to the indicator is greater than the preset capacity of the corresponding data table, the indicator calculation platform can automatically detect it and realize dynamic expansion.

[0078] Through the above management measures, the indicator calculation platform realizes the automation, intelligence and precision of the storage management of indicator data (the indicator calculation results corresponding to the indicators), further improving the utilization rate of storage resources.

[0079] It should be noted that the indicator calculation results corresponding to the indicators can be stored in Elasticsearch (ES) or Timeseries Databases. Since Elasticsearch can process and retrieve large-scale data sets, this embodiment preferably stores the indicator calculation results corresponding to the indicators in Elasticsearch.

[0080] It should also be noted that the indicator calculation platform generates an indicator code when establishing the second data table. It can be understood that different data tables correspond to one indicator code.

[0081] Furthermore, after the step of calculating the preset index of the target data source based on the index calculation logic and obtaining the index calculation result, the following feasible implementation methods may also exist:

[0082] In response to the indicator query operation, based on the target indicator code corresponding to the indicator to be queried and the target data table corresponding to the target indicator code, the indicator calculation result corresponding to the indicator to be queried is searched.

[0083] It should be noted that in order to facilitate user queries, the indicator calculation platform can provide users with a general query API (interface). When querying indicators, the corresponding indicator calculation results can be obtained through a fixed message format.

[0084] Specifically, refer to Figure 4 , the indicator calculation platform responds to the indicator query operation triggered by the user through the query interface, and the indicator calculation platform obtains the data table name (storage index template name) corresponding to the indicator through the cache routing; the indicator calculation platform verifies the query conditions in the request message and determines whether it is necessary to perform an aggregated and downgraded query on this query, wherein the basis for the judgment is that when the data required by the query condition has been stored in a data table at a certain summary level, detailed data is not queried from a lower-level data table to improve query efficiency; it can be understood that the above method can save data aggregation time on the one hand, and avoid memory overflow due to large data volume during data aggregation on the other hand.

[0085] For example, when a user queries the full summary data of an indicator (launched in September 2023) from the time it was launched (configured) to the present, the data for 2023 can be obtained from the monthly statistical data table of the indicator. The data for January 2024 is not obtained directly from the minute-level statistical data table, but this part of the statistical data is downgraded and split into three parts: daily level (from the 1st to the 22nd), hourly level (0 o'clock to 17 o'clock), and minute level (17:00-17:30). It is then obtained in parallel through multiple threads and then summarized and returned to the front end after aggregation.

[0086] In addition, for business indicators with a high level of assurance, the indicator calculation platform can also use the master-slave query functions of Elasticsearch and TiDB (Titanium Database). Specifically, as the primary database, Elasticsearch is used to store real-time updated data; TiDB is used as a backup database to store historical data and backup data.

[0087] In this embodiment, the indicator calculation platform responds to the data source configuration operation, obtains the format of the data source and the address of the data source, and determines the indicator calculation logic based on the format of the data source; it can be understood that the indicator calculation platform is designed with different indicator calculation logics for data sources of different formats; and, the indicator calculation platform of the present application can directly obtain the target data source based on the address of the data source, without the need for the user to manually import the data source; the indicator calculation platform can calculate the preset indicators of the target data source based on the indicator calculation logic to obtain the indicator calculation results; that is, for data sources of different formats, there is no need to use special tools or software for calculation respectively, and there is no need for users to import data sources of different formats when using different tools, which can improve the convenience of indicator calculation.

[0088] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 5 When the target data source is real-time data, step S20 includes steps S01 to S02:

[0089] Step S01, based on the indicator calculation logic, determining a calculation unit, wherein the preset indicator includes an indicator to be calculated corresponding to the calculation unit;

[0090] It should be noted that, currently, in the process of indicator calculation, it is impossible to realize real-time calculation of indicators of data sources in different formats, which has certain limitations.

[0091] In order to solve the above problems, when the indicator calculation platform detects that the target data source is real-time data, it can determine the calculation unit based on the indicator calculation logic; it can be understood that since the calculation cycle of real-time data is relatively small, if the indicator calculation logic has a corresponding calculation unit, the corresponding calculation unit will be directly allocated to the target data source; if the indicator calculation logic has a corresponding calculation unit, it is necessary to generate the corresponding calculation unit based on the indicator calculation logic (create an application with indicator calculation logic).

[0092] It should also be noted that the preset indicators include the indicators to be calculated corresponding to the calculation units. It can be understood that the calculation units generated based on the indicator calculation logic correspond to the indicators to be calculated, that is, different calculation units correspond to different indicators to be calculated. It can be understood that when the calculation unit is determined, the indicator to be calculated is determined.

[0093] Step S02, calculating the to-be-calculated index of the real-time data through the calculation unit to obtain the index calculation result.

[0094] Furthermore, when it is necessary to calculate an indicator, the calculation unit pulls the indicator to be calculated under the unit, and broadcasts the indicator calculation logic to different instances under the calculation unit. Different instances calculate the indicators to be calculated of the real-time data respectively, thereby obtaining the indicator calculation results and completing the real-time indicator calculation. Figure 7 .

[0095] Specifically, the specific implementation of determining the calculation unit based on the indicator calculation logic may be:

[0096] Acquire the business scenario to which the target data source belongs; and determine a calculation unit based on the indicator calculation logic and the business scenario to which the target data source belongs.

[0097] The business scenario to which the target data source belongs may be input by the user or selected by the user from a drop-down option during data source configuration.

[0098] The business scenario may be a customer service call scenario, an enterprise financial scenario, etc. It can be understood that since different business scenarios require different indicators to be calculated, this embodiment determines the calculation unit based on the indicator calculation logic and the business scenario to which the target data source belongs. The corresponding indicator to be calculated can be calculated through the calculation unit.

[0099] It should be noted that the above real-time data can be Kafka data.

[0100] Based on the above method, the indicator calculation module can be purely configurable, that is, the indicator calculation can be defined through a configuration file or interface, and the indicator calculation logic of data sources in different formats can be adjusted without modifying the code; the modification can take effect in real time, that is, when the configuration changes, the indicator calculation module can immediately reflect these changes without restarting the service or redeploying.

[0101] For example, in the process of calculating real-time data in the indicator calculation module, Apache Spark Streaming can be used to implement data access, data screening, dimensionality reduction, data aggregation and other steps. Specifically, the Spark Streaming data source connector is used to access the real-time workflow. Figure 6 .

[0102] For example, you can read data sources from Kafka, Flume, or other message queues, use the filter function provided by Spark Streaming to filter data according to the configured filter conditions, use Spark Streaming's map function to select the required fields to achieve dimensionality reduction, and use Spark Streaming's aggregation functions, such as reduceByKey, aggregate, or fold, to aggregate the data sources and generate the required indicators. In this process, the configuration management module can store and manage configuration information such as filter conditions, dimensions, and aggregation methods. When these configuration information changes, the indicator calculation module can detect these changes and update the calculation logic in Spark Streaming, that is, the jobs in Spark Streaming can be dynamically updated instead of restarting the entire job, thereby realizing the functions of pure configuration and real-time effectiveness of modifications.

[0103] In this embodiment, the above method enables the indicator calculation platform to realize the indicator calculation of real-time data, enrich the use scenarios of the indicator calculation platform, and improve user convenience. Moreover, the user does not need to write code to realize the indicator calculation, but provides the indicator calculation platform for the user, and the user can determine the indicator calculation logic through interface configuration, drop-down selection or SQL language (writing filter conditions), etc., which reduces the user's use threshold, improves user convenience, and improves the efficiency of indicator calculation.

[0104] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 8 When the target data source is offline data, step S20 further includes steps A01 to A02:

[0105] Step A01, generating a workflow corresponding to a target indicator based on the indicator calculation logic;

[0106] Currently, in the process of indicator calculation, it is impossible to realize real-time calculation of indicators of data sources in different formats, which has certain limitations.

[0107] In order to solve the above problem, when the indicator calculation platform detects that the target data source is offline data, it can generate a workflow corresponding to the target indicator based on the indicator calculation logic.

[0108] It should be noted that the preset indicators include at least one target indicator, that is, the target indicator can be an indicator that needs to be calculated and input by the user through the indicator configuration interface, or it can be a pre-configured indicator that needs to be calculated for different data sources, etc.

[0109] Among them, one target indicator corresponds to one workflow, and the workflow is a process for calculating the target indicator of the offline data. One workflow includes multiple nodes, and each node corresponds to a script, which can be an HQL (Hive Query Language) script.

[0110] Reference Fig. 9 The specific implementation method of generating the workflow corresponding to the target indicator based on the indicator calculation logic may be:

[0111] When the data sources corresponding to at least two target indicators are the same, based on the indicator calculation logic, determine the common flow on which the at least two target indicators depend; based on the indicator calculation logic and the common flow, generate the workflows corresponding to the at least two target indicators.

[0112] It should be noted that when multiple target indicators depend on the same target data source, multiple workflows performing read and write operations on the same target data source are likely to affect each other, resulting in abnormal indicator calculation results or workflow execution failure. At the same time, repeated read and write operations by multiple workflows on the same target data source will also cause a waste of platform resources.

[0113] In order to solve the above problems, when the indicator calculation platform generates workflows for multiple target indicators that depend on the same target data source, it uniformly extracts nodes with the same operation process as the dependent workflows of each target indicator, that is, the common flow.

[0114] Specifically, if there are some similarities in the indicator calculation logic, the workflow parts it depends on are the same. Therefore, based on the indicator calculation logic, the common flows on which at least two of the target indicators depend can be determined, and then based on the indicator calculation logic and the common flows, at least two workflows corresponding to the target indicators can be generated, thereby reducing the number of script generation times during the workflow generation process and improving efficiency.

[0115] Further, refer to Fig. 9 ,For the same target data source, when generating a workflow, if there is a common stream, the workflow corresponding to the target indicator references the common stream as a dependency. If there is no common stream, a new common stream is created as a dependency.

[0116] Step A02, executing the workflow corresponding to the target indicator to obtain the calculation result of the indicator to be calculated, wherein the workflow is the process of calculating the target indicator of the offline data.

[0117] It should be noted that when executing at least two workflows corresponding to the target indicators, both can rely on the common flow. When the common flow is completed, each workflow corresponding to the target indicator can continue to execute other workflow nodes in parallel to obtain the calculation result of the indicator to be calculated. Fig. 9 .

[0118] It can be understood that the extraction of common flows can not only avoid the waste of resources caused by repeated calculations on the platform, but also reduce the mutual influence between workflows, ensure the quality of indicator calculation results, improve indicator calculation efficiency, and thus further improve the computing power of the platform.

[0119] The offline data may be HIVE data, SFTP data, ES data, Oracle data, MYSQL data, etc.

[0120] In this embodiment, the above method can enable the indicator calculation platform to realize indicator calculation for offline data and real-time data at the same time, enrich the use scenarios of the indicator calculation platform, and improve user convenience.

[0121] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the indicator calculation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0122] The present application provides an indicator calculation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the indicator calculation method in the above-mentioned embodiment one.

[0123] Reference below Fig.10 , which shows a schematic diagram of the structure of an indicator calculation device suitable for implementing the embodiment of the present application. The indicator calculation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The indicator calculation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0124] like Fig.10As shown, the indicator calculation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the indicator calculation device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the indicator calculation device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows an indicator calculation device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0125] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0126] The index calculation device provided by the present application adopts the index calculation method in the above embodiment, which can solve the technical problem of low convenience of index calculation. Compared with the prior art, the beneficial effects of the index calculation device provided by the present application are the same as the beneficial effects of the index calculation method provided by the above embodiment, and the other technical features in the index calculation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0127] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0129] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the indicator calculation method in the above-mentioned embodiment.

[0130] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0131] The computer-readable storage medium may be included in the indicator calculation device; or may exist independently without being assembled into the indicator calculation device.

[0132] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the indicator calculation device, the indicator calculation device is enabled to: execute the indicator calculation method.

[0133] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0136] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned index calculation method, and can solve the technical problem of low convenience of index calculation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the index calculation method provided in the above-mentioned embodiment, and will not be repeated here.

[0137] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned indicator calculation method when executed by a processor.

[0138] The computer program product provided by the present application can solve the technical problem of low convenience in index calculation. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the index calculation method provided by the above embodiment, which will not be repeated here.

[0139] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for calculating an indicator, characterized in that: Applied to the indicator calculation platform, the indicator calculation method includes: In response to the data source configuration operation, the format of the data source and the address of the data source are acquired, and based on the format of the data source, the indicator calculation logic is determined; Based on the address of the data source, a target data source is obtained, and based on the indicator calculation logic, a preset indicator of the target data source is calculated to obtain an indicator calculation result.

2. The method according to claim 1, characterized in that When the target data source is real-time data, the step of calculating the preset index of the target data source based on the index calculation logic to obtain the index calculation result includes: Based on the indicator calculation logic, a calculation unit is determined, wherein the preset indicator includes an indicator to be calculated corresponding to the calculation unit; The calculation unit calculates the to-be-calculated index of the real-time data to obtain the index calculation result.

3. The method according to claim 2, characterized in that The step of determining a calculation unit based on the indicator calculation logic includes: Obtain the business scenario to which the target data source belongs; Based on the indicator calculation logic and the business scenario to which the target data source belongs, a calculation unit is determined.

4. The method according to claim 1, characterized in that The preset indicator includes at least one target indicator. When the target data source is offline data, the step of calculating the preset indicator of the target data source based on the indicator calculation logic to obtain the indicator calculation result includes: Based on the indicator calculation logic, generate a workflow corresponding to the target indicator; Execute the workflow corresponding to the target indicator to obtain the calculation result of the indicator to be calculated, where the workflow is the process of calculating the target indicator of the offline data.

5. The method according to claim 4, characterized in that The step of generating a workflow corresponding to the target indicator based on the indicator calculation logic includes: When the data sources corresponding to at least two target indicators are the same, determining, based on the indicator calculation logic, a common flow on which at least two of the target indicators depend; Based on the indicator calculation logic and the common flow, generating at least two workflows corresponding to the target indicators; Wherein, when executing the workflows corresponding to at least two of the target indicators, the common flow is only executed once.

6. The method according to any one of claims 1 to 5, characterized in that: Before the step of calculating the preset index of the target data source based on the index calculation logic and obtaining the index calculation result, the method further includes: Obtain the business scenario to which the target data source belongs; Determine whether there is a corresponding first data table for the indicator to be stored in the business scenario; If it does not exist, a second data table corresponding to the indicator to be stored is established based on the business scenario, and the first data table and the second data table are respectively used to store the indicator calculation results corresponding to the indicator to be stored.

7. The method according to claim 1, characterized in that When establishing the second data table, generating an indicator code, and calculating the preset indicator of the target data source based on the indicator calculation logic, and obtaining the indicator calculation result, the step further includes: In response to the indicator query operation, based on the target indicator code corresponding to the indicator to be queried and the target data table corresponding to the target indicator code, the indicator calculation result corresponding to the indicator to be queried is searched.

8. An indicator calculation platform, characterized in that: The indicator calculation platform is used for: In response to the data source configuration operation, the format of the data source and the address of the data source are acquired, and based on the format of the data source, the indicator calculation logic is determined; Based on the address of the data source, a target data source is obtained, and based on the indicator calculation logic, a preset indicator of the target data source is calculated to obtain an indicator calculation result.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the indicator calculation method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the indicator calculation method according to any one of claims 1 to 7 are implemented.

Images