Intelligent index report calculation generation method and device and medium

Abstract

The invention discloses an intelligent index report calculation and generation method and device and a medium, mainly relates to the technical field of index reports, and aims to solve the problems that an existing index report is low in manual generation efficiency and prone to making mistakes, a commercial intelligent tool is high in generation cost, and function customization is difficult. Comprising the following steps: acquiring source data corresponding to a report to be counted through any computing node, and synchronizing a target table corresponding to the source data into an analysis database of the current computing node; the current computing node determines required main body data from the shared database according to the main body data name and the query condition, and generates an actual number of processing tasks based on the actual number of the required main body data and the unique identifier of the main body data; each computing node determines a required preset index item to obtain a computing result; notifying the calculation result to the calculation node generating the processing task; after the calculation nodes obtain calculation results of all the processing tasks, the calculation results are filled into the target table, and the target table is output and stored.

Description

Technical Field

[0001] The present application relates to the technical field of report generation, and in particular to a method, device and medium for calculating and generating an intelligent indicator report. Background Art

[0002] Given the importance of data-driven decision making, the demand for intelligent indicator calculation and report generation in various industries and fields is also growing. Users can automatically generate sales reports of different dimensions (such as by region, product, and time period) based on sales data, and predict future sales based on historical trends. Existing solutions usually include Excel and other spreadsheet statistics, dedicated business tools, etc.

[0003] For some small and medium-sized enterprises, if manual calculations and report generation are used, there are usually problems of inefficiency and poor scalability. Manual calculations and report generation are time-consuming and labor-intensive, especially when dealing with large amounts of data or complex calculations, and are prone to errors. When the amount of data or the types of indicators increase, manual methods are difficult to maintain, lack automation capabilities, and have poor scalability. If dedicated business intelligence tools are used, there are also the following problems: 1) They are usually expensive, especially for small and medium-sized enterprises, with high initial investment and ongoing usage costs; 2) The learning cost is high, and specialized training is required to fully utilize the functions of these tools, and non-technical personnel may find it difficult to get started; 3) It is difficult to customize functions. Although business intelligence tools provide rich functions, customization may be limited for very specific business needs.

[0004] Therefore, there is an urgent need for an intelligent indicator report calculation and generation method, device and medium to solve the above-mentioned problems of low efficiency and easy errors in manual generation, high cost of business intelligence tool generation, and difficulty in function customization. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present application provides a method, device and medium for calculating and generating an intelligent indicator report to solve the problems of low efficiency and easy errors in manual generation of existing indicator reports, high generation cost of business intelligence tools, and difficulty in function customization.

[0006] In a first aspect, the present application provides a method for calculating and generating an intelligent indicator report, the method comprising: When the computing node is started, it registers the node information on the preset redis and sends heartbeat data to redis regularly through the local heartbeat mechanism; obtains the source data corresponding to the report to be counted through any computing node, and then synchronizes the target table corresponding to the source data to the analysis database of the current computing node; obtains the report statistical dimension parameters, main data name, query conditions, and statistical items corresponding to the report to be counted; among them, the statistical items include several preset indicator items, and the preset indicator items include indicator ID, indicator attributes, indicator name, and preset operation expressions; generates the current task ID and stores the corresponding relationship between the current task ID and the statistical item in the shared database; the current computing node obtains the statistical items from the shared database according to the main data name and query conditions The required subject data is determined in the database, and based on the actual number of required subject data and the unique identifier of the subject data, an actual number of processing tasks are generated; wherein each processing task contains a unique identifier of the subject data; the current computing node determines all currently surviving computing nodes through the heartbeat data on redis, and notifies each computing node of the current task ID and the processing task; each computing node determines the required preset indicator items according to the received current task ID, and then uses the preset indicator items to process the subject data corresponding to the processing task to obtain the calculation result; the calculation result is notified to the computing node that generates the processing task; when the computing node obtains the calculation results of all processing tasks, it fills in the target table, and saves the target table out of the database.

[0007] Furthermore, when the computing node starts, the node information is registered on the preset redis, and heartbeat data is sent to redis regularly through the local heartbeat mechanism, specifically including: when the computing node starts, the node information is registered on the preset redis, and an expiration time is set; after successful registration, all computing nodes send heartbeat data to redis regularly through the local heartbeat mechanism within each preset time period to perform heartbeat keep-alive.

[0008] Furthermore, the source data includes a database and a target table name; through any computing node, the source data corresponding to the report to be counted is obtained, and then the target table corresponding to the source data is synchronized to the analysis database of the current computing node, specifically including: through any computing node, the database and the target table name corresponding to the report to be counted is obtained; then, using the target table name, the corresponding target table is searched from the database, and then the target table is synchronized to the analysis database of the current computing node.

[0009] Furthermore, the current task ID and processing task are distributed to each computing node, including: The actual number of processing tasks is modulo the number of computing nodes to calculate the allocated number of processing tasks that each computing node needs to calculate.

[0010] Further, the required preset indicator items are determined according to the received current task ID, and then the preset indicator items are used to process the main data corresponding to the task to obtain the calculation results, which specifically include: After receiving the processing task, the computing node creates a task thread for each processing task and puts it into the thread pool; Obtain the correspondence between the current task ID and the preset indicator item from the shared database, and then determine the preset indicator item required for the current task ID, and then create a calculation task for each preset indicator item and add it to the thread pool; Generate the md5 value corresponding to each preset indicator item according to the unique identifier of the subject data, indicator ID, current task ID and report statistical dimension parameters corresponding to the preset indicator item; Query the cached calculation result corresponding to the current md5 value from redis. When the cached calculation result exists, use it directly. When the cached calculation result does not exist, query the cached calculation result corresponding to the current md5 value from the preset indicator result correction and solidification table. When the cached calculation result exists, use it directly. When the cached calculation result does not exist, obtain the indicator ID of the current preset indicator item. Determine the sub-indicator ID corresponding to the current indicator ID, and then load the preset indicator item corresponding to the sub-indicator ID. When the preset indicator item does not have a sub-indicator ID and does not have a cached calculation result, obtain the indicator attribute corresponding to the current preset indicator item. When the indicator attribute is a non-scoring attribute, dynamically execute the preset operation expression in the current preset indicator item through the @SelectProvider annotation of mybatis to obtain the calculation result. When the indicator attribute is a scoring attribute, use the AviatorExpression method to execute the preset operation expression in the current preset indicator item to obtain the calculation result.

[0011] Furthermore, after obtaining the calculation result, the method further includes: The calculation result, md5 value, and indicator name are cached in redis as the assembly result.

[0012] Furthermore, the method further comprises: When it is determined that the cached calculation result corresponding to any md5 value in the shared database is an incorrect result, the data corresponding to the md5 value is deleted from the shared database; the correct calculation result corresponding to the md5 value is obtained, and then the correct calculation result corresponding to the md5 value is stored in the preset indicator result correction solidification table.

[0013] Furthermore, the current task ID and processing task are notified to each computing node, including: The current task ID and processing task are notified to each computing node through the redis message queue.

[0014] The calculation results are notified to the computing nodes that generate the processing tasks, including: The calculation results are written to the shared database and the computing nodes that generate the processing tasks are notified.

[0015] In a second aspect, the present application provides an intelligent indicator report calculation and generation device, the device comprising: processor; and a memory having executable codes stored thereon, which, when executed, causes the processor to execute a smart indicator report calculation and generation method as described in any one of the above items.

[0016] In a third aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, which when executed implement a method for calculating and generating an intelligent indicator report as described above.

[0017] Those skilled in the art can understand that the present application has at least the following beneficial effects: This application can encapsulate the calculation tasks of report indicator calculation according to the main information into a task list through any computing node that receives the report calculation request, and assign the tasks to the service nodes that have been registered and survived in redis for calculation, realizing decentralized distributed computing, which is conducive to the expansion and contraction of the computing host and fully utilizes the existing hardware resources. In addition, the use of each computing node for indicator calculation improves concurrency and improves computing efficiency; in summary, this application realizes the automatic generation of target tables without manual generation or the use of business intelligence tools, solving the problems of low efficiency, easy errors and high generation costs of business intelligence (BI) tools in the existing manual generation of indicator reports.

[0018] In addition, the present application can determine statistical items according to needs, and then determine specific preset operation expressions, thereby realizing the function of expression customization, thereby solving problems such as difficulty in function customization. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which: Figure 1 It is a flow chart of a method for calculating and generating an intelligent indicator report provided in an embodiment of the present application.

[0020] Figure 2 It is a schematic diagram of the internal structure of an intelligent indicator report calculation and generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be implemented through the preferred embodiments. The preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present disclosure.

[0022] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0023] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0024] The present application embodiment provides a method for calculating and generating an intelligent indicator report, such as Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps: Step 110: When the computing node is started, the node information is registered on the preset redis, and heartbeat data is sent to redis regularly through the local heartbeat mechanism.

[0025] It should be noted that the computing node may be a server, and redis may be a preset central platform that can support a Redis server.

[0026] This step can be specifically as follows: when the computing node is started, register the node information on the preset redis and set the expiration time; after successful registration, all computing nodes periodically send heartbeat data to redis through the local heartbeat mechanism within each preset time period to perform heartbeat keep-alive.

[0027] It should be noted that the expiration time can be 30 seconds, such as report:node1 ip:port 30s; the heartbeat data can be set to redis regularly through the local heartbeat mechanism (key:report:node1 value:ip:port ttl:30S), and the preset time period can be 20 seconds.

[0028] Step 120: Obtain source data corresponding to the report to be counted through any computing node, and then synchronize the target table corresponding to the source data to the analysis database of the current computing node.

[0029] It should be noted that the source data includes the database and target table names.

[0030] This application can be specifically: Through any computing node, obtain the database and target table name corresponding to the report to be counted; then use the target table name to find the corresponding target table from the database, and then synchronize the target table to the analysis database of the current computing node.

[0031] Step 130, obtain the report statistical dimension parameters, main data name, query conditions, and statistical items corresponding to the report to be counted; generate the current task ID and store the corresponding relationship between the current task ID and the statistical items in the shared database.

[0032] It should be noted that the statistical items include several preset indicator items, and the preset indicator items include indicator ID, indicator attributes, indicator name, and preset operation expression.

[0033] The preset operation expression may be a calculation formula or judgment logic.

[0034] The judgment logic can be: The total score is 10, the maximum score is 10, and the minimum score is 3; The product customer evaluation score expression is as follows: if (monthly favorable comment rate <= 50%) {total score - 7}; elsif(monthly favorable comment rate>50%&&monthly favorable comment rate<=70%){total score-5}; elsif(monthly favorable comment rate>70%&&monthly favorable comment rate<=90%){total score-3}; elsif(monthly favorable comment rate>90%&&monthly favorable comment rate<100%){total score-1}; else{total score}.

[0035] At the same time, it is stipulated that the maximum total score will not exceed 10 and the minimum will not be less than 3.

[0036] Step 140: The current computing node determines the required subject data from the shared database according to the subject data name and the query condition, and generates an actual number of processing tasks based on the actual number of required subject data and the unique identifier of the subject data.

[0037] It should be noted that each processing task contains a unique identifier of the subject data for easy distinction.

[0038] Step 150, the current computing node determines all currently surviving computing nodes through the heartbeat data on redis, and notifies each computing node of the current task ID and the processing task; each computing node determines the required preset indicator items based on the received current task ID, and then uses the preset indicator items to process the main data corresponding to the processing task to obtain the calculation result; the calculation result is notified to the computing node that generates the processing task.

[0039] As an example, the current task ID and the processing task are notified to each computing node, which may be: The current task ID and processing task are notified to each computing node through the redis message queue.

[0040] The current task ID and the processing task are distributed to each computing node, which can be specifically: The actual number of processing tasks is modulo the number of computing nodes to calculate the allocated number of processing tasks that each computing node needs to calculate.

[0041] Among them, the required preset index items are determined according to the received current task ID, and then the preset index items are used to process the main data corresponding to the task to obtain the calculation result, which can be specifically: S1. After receiving the processing task, the computing node creates a task thread for each processing task and puts it into the thread pool.

[0042] S2. Obtain the correspondence between the current task ID and the preset indicator items from the shared database, and then determine the preset indicator items required for the current task ID, and then create a calculation task for each preset indicator item and add it to the thread pool.

[0043] It should be noted that when each computing node performs indicator calculation, a thread pool is created for the processing task and the indicator calculation task respectively. At the same time, the indicator information that has been calculated by each computing node is subsequently cached in redis. While improving concurrency, it makes full use of the calculated results, thereby improving computing efficiency.

[0044] S3. Generate the md5 value corresponding to each preset indicator item according to the unique identifier of the subject data, indicator ID, current task ID and report statistical dimension parameters corresponding to the preset indicator item.

[0045] It should be noted that S3 can calculate the md5 value by the existing technology, and this application does not limit the calculation process.

[0046] S4. Query the cached calculation result corresponding to the current md5 value from redis. When the cached calculation result exists, use it directly. When the cached calculation result does not exist, query the cached calculation result corresponding to the current md5 value from the preset indicator result correction and solidification table. When the cached calculation result exists, use it directly. When the cached calculation result does not exist, obtain the indicator ID of the current preset indicator item. Determine the sub-indicator ID corresponding to the current indicator ID, and then load the preset indicator item corresponding to the sub-indicator ID. When the preset indicator item does not have a sub-indicator ID and does not have a cached calculation result, obtain the indicator attribute corresponding to the current preset indicator item. When the indicator attribute is a non-scoring attribute, dynamically execute the preset operation expression in the current preset indicator item through the @SelectProvider annotation of mybatis to obtain the calculation result. When the indicator attribute is a scoring attribute, use the AviatorExpression method to execute the preset operation expression in the current preset indicator item to obtain the calculation result.

[0047] It should be noted that the sub-indicator ID is an indicator ID in this application. The indicator ID itself can also be a sub-indicator ID of other indicator IDs. In addition, the correspondence between this indicator ID and the sub-indicator ID is preset by technical personnel in this field according to actual needs. In addition, the preset operation expression (mainly logic) of the above-mentioned scoring attribute cannot be dynamically executed through the @SelectProvider annotation of mybatis. This application is limited to using the AviatorExpression method to execute the preset operation expression in the current preset indicator item to obtain the calculation result (‌AviatorExpression‌ is a high-performance Java expression evaluation engine based on bytecode, focusing on providing fast expression calculation capabilities. It is mainly used for dynamic calculation expressions, especially in scenarios that require frequent calculations). Code Aviator is used here to assemble indicators and process calculation logic, and its dynamic expression capabilities are used to implement the definition of integral rules and the calculation of integral indicators, making user operations more intuitive.

[0048] In addition, in order to facilitate subsequent direct use and reduce computing consumption, after obtaining the calculation result, the method also includes: caching the calculation result, md5 value, and indicator name as an assembly result in redis.

[0049] In addition, the present application manually corrects and solidifies, and the solidification result of the current indicator can participate in the calculation of the integral indicator or other composite indicators, wherein the correction and solidification method can be: When it is determined that the cached calculation result corresponding to any md5 value in the shared database is an incorrect result, the data corresponding to the md5 value is deleted from the shared database; the correct calculation result corresponding to the md5 value is obtained, and then the correct calculation result corresponding to the md5 value is stored in the preset indicator result correction solidification table.

[0050] As an example, notifying the computing node that generates the processing task of the computing result may specifically include: writing the computing result into a shared database, and notifying the computing node that generates the processing task.

[0051] Based on the above steps, those skilled in the art can know that the present application assigns tasks to service nodes that have been registered and survived in redis for calculation, thereby realizing decentralized distributed computing, facilitating the expansion and reduction of computing hosts, and making full use of existing hardware resources.

[0052] Step 160: After the computing node obtains the computing results of all processing tasks, it fills in the target table and saves the target table out of the database.

[0053] The above is a method embodiment of the present application. Based on the same inventive concept, the present application embodiment also provides an intelligent indicator report calculation and generation device. Figure 2 As shown, the device includes: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes a smart indicator report calculation and generation method as described in the above embodiment.

[0054] Specifically, when the computing node on the server is started, it registers the node information on the preset redis and sends heartbeat data to redis regularly through the local heartbeat mechanism; through any computing node, it obtains the source data corresponding to the report to be counted, and then synchronizes the target table corresponding to the source data to the analysis database of the current computing node; obtains the report statistical dimension parameters, main data name, query conditions, and statistical items corresponding to the report to be counted; among them, the statistical items include several preset indicator items, and the preset indicator items include indicator ID, indicator attributes, indicator name, and preset operation expressions; generates the current task ID and stores the corresponding relationship between the current task ID and the statistical item in the shared database; the current computing node generates the corresponding relationship between the main data name and the query condition according to the main data name and the query condition Determine the required subject data from the shared database, and generate an actual number of processing tasks based on the actual number of required subject data and the unique identifier of the subject data; wherein each processing task contains a unique identifier of the subject data; the current computing node determines all currently surviving computing nodes through the heartbeat data on redis, and notifies each computing node of the current task ID and the processing task; each computing node determines the required preset indicator items based on the received current task ID, and then uses the preset indicator items to process the subject data corresponding to the processing task to obtain the calculation result; the calculation result is notified to the computing node that generates the processing task; when the computing node obtains the calculation results of all processing tasks, it fills in the target table, and saves the target table out of the warehouse.

[0055] In addition, an embodiment of the present application also provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a method for calculating and generating an intelligent indicator report as described above is implemented.

[0056] So far, the technical solutions of the present disclosure have been described in combination with the above multiple embodiments, but it is easy for those skilled in the art to understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above-mentioned various embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concept and / or technical principle of the present disclosure will fall within the protection scope of the present disclosure.

Claims

1. A method for calculating and generating an intelligent indicator report, characterized in that: The method comprises: When the computing node starts, it registers the node information on the preset redis and sends heartbeat data to redis regularly through the local heartbeat mechanism; Through any computing node, the source data corresponding to the statistical report to be generated is obtained, and then the target table corresponding to the source data is synchronized to the analysis database of the current computing node; Obtain the report statistical dimension parameters, main data name, query conditions, and statistical items corresponding to the report to be statistically analyzed; wherein the statistical items include several preset indicator items, and the preset indicator items include indicator ID, indicator attributes, indicator name, and preset operation expressions; generate the current task ID and store the corresponding relationship between the current task ID and the statistical items in the shared database; The current computing node determines the required subject data from the shared database according to the subject data name and the query condition, and generates an actual number of processing tasks based on the actual number of required subject data and the subject data unique identifier; wherein each processing task includes a subject data unique identifier; The current computing node determines all currently surviving computing nodes through the heartbeat data on redis, and notifies each computing node of the current task ID and processing task; each computing node determines the required preset indicator items based on the received current task ID, and then uses the preset indicator items to process the main data corresponding to the processing task to obtain the calculation result; the calculation result is notified to the computing node that generates the processing task; When the computing node obtains the calculation results of all processing tasks, it fills in the target table and saves the target table out of the database.

2. The method for calculating and generating an intelligent indicator report according to claim 1, characterized in that: When the computing node starts, it registers the node information on the preset redis and sends heartbeat data to redis periodically through the local heartbeat mechanism, including: When the computing node is started, register the node information on the preset redis and set the expiration time; After successful registration, all computing nodes send heartbeat data to Redis through the local heartbeat mechanism at regular intervals within a preset time period to keep the heartbeat alive.

3. The method for calculating and generating an intelligent indicator report according to claim 1, characterized in that: Source data includes database and target table names; Through any computing node, obtain the source data corresponding to the statistical report to be counted, and then synchronize the target table corresponding to the source data to the analysis database of the current computing node, including: Through any computing node, obtain the database and target table name corresponding to the report to be counted; then use the target table name to find the corresponding target table from the database, and then synchronize the target table to the analysis database of the current computing node.

4. The method for calculating and generating an intelligent indicator report according to claim 1, characterized in that: Distribute the current task ID and processing tasks to each computing node, including: The actual number of processing tasks is modulo the number of computing nodes to calculate the allocated number of processing tasks that each computing node needs to calculate.

5. The method for calculating and generating an intelligent indicator report according to claim 1, characterized in that: Determine the required preset indicator items according to the received current task ID, and then use the preset indicator items to process the main data corresponding to the task to obtain the calculation results, specifically including: After receiving the processing task, the computing node creates a task thread for each processing task and puts it into the thread pool; Obtain the correspondence between the current task ID and the preset indicator item from the shared database, and then determine the preset indicator item required for the current task ID, and then create a calculation task for each preset indicator item and add it to the thread pool; Generate the md5 value corresponding to each preset indicator item according to the unique identifier of the subject data, indicator ID, current task ID and report statistical dimension parameters corresponding to the preset indicator item; Query the cached calculation result corresponding to the current md5 value from redis. When the cached calculation result exists, use it directly. When the cached calculation result does not exist, query the cached calculation result corresponding to the current md5 value from the preset indicator result correction and solidification table. When the cached calculation result exists, use it directly. When the cached calculation result does not exist, obtain the indicator ID of the current preset indicator item. Determine the sub-indicator ID corresponding to the current indicator ID, and then load the preset indicator item corresponding to the sub-indicator ID. When the preset indicator item does not have a sub-indicator ID and does not have a cached calculation result, obtain the indicator attribute corresponding to the current preset indicator item. When the indicator attribute is a non-scoring attribute, dynamically execute the preset operation expression in the current preset indicator item through the @SelectProvider annotation of mybatis to obtain the calculation result. When the indicator attribute is a scoring attribute, use the AviatorExpression method to execute the preset operation expression in the current preset indicator item to obtain the calculation result.

6. The method for calculating and generating an intelligent indicator report according to claim 5, characterized in that: After obtaining the calculation result, the method further includes: The calculation result, md5 value, and indicator name are cached in redis as the assembly result.

7. The method for calculating and generating an intelligent indicator report according to claim 5, characterized in that: The method further comprises: When it is determined that the cache calculation result corresponding to any md5 value in the shared database is an erroneous result, the data corresponding to the md5 value is deleted from the shared database; The correct calculation result corresponding to the md5 value is obtained, and then the correct calculation result corresponding to the md5 value is stored in the preset indicator result correction solidification table.

8. The method for calculating and generating an intelligent indicator report according to claim 1, characterized in that: Notify each computing node of the current task ID and processing task, including: Notify each computing node of the current task ID and processing task through the redis message queue; The calculation results are notified to the computing nodes that generate the processing tasks, including: The calculation results are written to the shared database and the computing nodes that generate the processing tasks are notified.

9. An intelligent indicator report calculation and generation device, characterized in that: The device comprises: processor; and a memory having executable code stored thereon, which, when the executable code is executed, enables the processor to execute a smart indicator report calculation and generation method as described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, they implement a method for calculating and generating an intelligent indicator report as described in any one of claims 1-8.

Images