A report calculation method, device and equipment based on a preprocessing component and a medium

Abstract

The application discloses a report calculation method and device based on a preprocessing component, equipment and a medium, wherein the method comprises the following steps: obtaining the computing power value of a current processor, and setting a trigger threshold based on the trigger threshold; obtaining the computing power requirement of a target calculation task, and judging whether the target calculation task triggers the preprocessing component based on the computing power requirement and the trigger threshold; if the preprocessing component is triggered, converting the database language in the target calculation task according to the preprocessing component rule, and executing the target calculation task by using the converted database language; and if the preprocessing component is not triggered, directly executing the target calculation task. The business data can be reasonably preprocessed by analyzing the calculation formula, and the computing performance can be greatly improved under the condition of limited resources. Through the preprocessing technology, the original data of the business system can be pre-arranged into a structure suitable for the report data taking scene, so that flexible and high-performance report calculation is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of report calculation, in particular to a report calculation method and device based on a preprocessing component, equipment and medium. BACKGROUND

[0002] In recent years, with the gradual perfection of information systems, the data in the cells of various reports under the BS architecture are largely from business systems. The data generated by various business systems in enterprise ERPs often need to be presented in the form of tables, which leads to complex data sources of report products. In order to ensure the real-time performance of data, it is necessary to write complex SQL for data extraction from the original business system, and it is difficult to ensure the performance of data extraction and calculation with a large number of complex SQLs.

[0003] In order to improve performance, the existing solutions often rely on data platforms to process, clean and arrange business data through ETL tools and then provide the data to report data extraction and calculation. This method is difficult to ensure data real-time performance and requires additional deployment of data platforms, which increases operation and maintenance costs. SUMMARY

[0004] To solve the above problems, the present application provides a report calculation method and device based on a preprocessing component, equipment and medium, wherein the method comprises:

[0005] An algorithm value of a current processor is obtained, and a trigger threshold value is set based on the algorithm value. An algorithm requirement of a target calculation task is obtained, and it is judged whether the preprocessing component is triggered based on the algorithm requirement and the trigger threshold value. If the preprocessing component is triggered, the database language in the target calculation task is converted according to the preprocessing component rule, and the target calculation task is executed using the converted database language. If the preprocessing component is not triggered, the target calculation task is directly executed.

[0006] In one embodiment, the conversion of the database language in the target calculation task according to the preprocessing component rule specifically comprises: obtaining a report formula in a target report, and converting the report formula into a standard database language; converting the standard database language into a syntax tree, and parsing the standard database language by traversing the syntax tree to obtain a data extraction field, a data extraction source table and a query condition field; giving the report formula with the same data extraction source table with the same first identifier; establishing a target data extraction source table, and importing the data in the data extraction source table into the target data extraction source table; rewriting the data extraction source table in the standard database language into a second identifier of the target data extraction source table, wherein the second identifier is combined from the first identifier and a system variable.

[0007] In an embodiment, the obtaining the computing power requirement of the target computing task specifically comprises: receiving task requirements of the target computing task, and determining computing content of the target computing task through a natural language processing technology; based on the computing content, obtaining a first identifier quantity corresponding to the target computing task and a formula quantity corresponding to each of the first identifiers; and based on the first identifier quantity and the formula quantity, determining the computing power requirement of the target computing task.

[0008] In an embodiment, the converting the database language in the target computing task according to the pre-processing component rule and executing the target computing task using the converted database language specifically comprises: determining all first identifiers corresponding to the target report; based on an execution sequence, generating a pre-processing configuration table according to the first identifiers and the reconstructed standard database language; and the pre-processing configuration table further includes a database language for establishing a target data source table according to the second identifiers and importing data in the data source table to the target data source table.

[0009] In an embodiment, after the parsing the standard database language through the syntax tree to obtain the data source table, the data source field, and the query condition field, the method further comprises: based on the first identifiers, recording the data source corresponding to the first identifiers, the data source field, the filtering condition, and the formula quantity to obtain a parsing result record table.

[0010] In an embodiment, after the executing the target computing task using the converted database language, the method further comprises: obtaining an execution progress of the target computing task; and based on the execution progress, asynchronously deleting the table with the second identifiers as the name.

[0011] In an embodiment, the system variable is a thread identifier for executing the pre-processing component.

[0012] The application further provides a report computing device based on a pre-processing component, comprising:

[0013] A trigger threshold module obtains a computing power value of a current processor and sets a trigger threshold based on the computing power value; a judgment module obtains a computing power requirement of a target computing task and judges whether the target computing task triggers a pre-processing component based on the computing power requirement and the trigger threshold; a pre-processing module converts a database language in the target computing task according to a pre-processing component rule and executes the target computing task using the converted database language if the pre-processing component is triggered; and a direct execution module directly executes the target computing task if the pre-processing component is not triggered.

[0014] The application further provides a report computing device based on a preprocessing component, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform: obtaining a computing power value of a current processor, and setting a triggering threshold value based on the computing power value; obtaining a computing power requirement of a target computing task, and judging whether the target computing task triggers the preprocessing component based on the computing power requirement and the triggering threshold value; if the preprocessing component is triggered, converting a database language in the target computing task according to a preprocessing component rule, and executing the target computing task by using the converted database language; and if the preprocessing component is not triggered, directly executing the target computing task.

[0015] The application further provides a non-volatile computer storage medium storing computer executable instructions, which are configured to: obtain a computing power value of a current processor, and set a triggering threshold value based on the computing power value; obtain a computing power requirement of a target computing task, and judge whether the target computing task triggers a preprocessing component based on the computing power requirement and the triggering threshold value; if the preprocessing component is triggered, convert a database language in the target computing task according to a preprocessing component rule, and execute the target computing task by using the converted database language; and if the preprocessing component is not triggered, directly execute the target computing task.

[0016] The method provided by the application can bring the following beneficial effects: the business data can be reasonably preprocessed by analyzing the calculation formula, and the computing performance can be greatly improved under the condition of limited resources. Through the preprocessing technology, the original data of the business system can be pre-arranged into a structure suitable for the report data taking scene, so that flexible and high-performance report computing is realized. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this application, illustrate certain illustrative embodiments of the application and together with the description serve to explain the application. In the drawings:

[0018] Figure 1 FIG. 1 is a flowchart of a report computing method based on a preprocessing component according to an embodiment of the application;

[0019] Figure 2 FIG. 2 is a logic diagram of the report computing method based on the preprocessing component according to the embodiment of the application;

[0020] Figure 3 FIG. 3 is a flowchart of a preprocessing process according to the embodiment of the application.

[0021] Figure 4 A module schematic diagram of a report calculation device based on a preprocessing component in an embodiment of the present application;

[0022] Figure 5 A structural schematic diagram of a report calculation device based on a preprocessing component in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in detail with the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0024] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0025] Figure 1 A flow schematic diagram of a report calculation method based on a preprocessing component provided by one or more embodiments of the present application. The method can be applied to reports corresponding to different business types, such as internet financial business type, e-commerce business type, instant messaging business type, game business type, public service business type, etc. The flow can be executed by a computing device in the corresponding field (such as a risk control server or a smart mobile terminal corresponding to a payment business, etc.), and some input parameters or intermediate results in the flow are allowed to be manually intervened and adjusted to help improve the accuracy.

[0026] The implementation of the analysis method related to the embodiments of the present application can be a terminal device or a server, and the present application does not make special limitations thereon. For the convenience of understanding and description, the following embodiments are described in detail with the server as an example.

[0027] It should be noted that the server can be a single device, or a system composed of multiple devices, i.e. a distributed server, and the present application does not make specific limitations thereon.

[0028] As shown in Figure 1 , Figure 2 and Figure 3 , the embodiments of the present application provide a report calculation method based on a preprocessing component, comprising:

[0029] S101: Obtain the computing power value of the current processor, and set a trigger threshold value based on the computing power value.

[0030] First, the computing power value of the current processor is determined, and the trigger threshold is set according to the computing power value of the processor. Here, the computing power value refers to a threshold related to the computing power of the processor, such as the minimum remaining computing power of the processor, the current remaining computing power, etc. The trigger threshold is set in advance by the staff and is used for comparison with the required computing power of the target computing task. It can be the required computing power of the target computing task, or the proportion of the required computing power of the target computing task in the current computing power of the processor.

[0031] S102: Obtain the computing power requirement of the target computing task, and determine whether the target computing task triggers the preprocessing component based on the computing power requirement and the trigger threshold.

[0032] Obtain the computing power requirement of the target computing task, and compare the required computing power of the target computing task with the trigger threshold to determine whether the target computing task triggers the preprocessing component, i.e., whether the preprocessing component is needed for processing to reduce the required computing power of the target computing task.

[0033] S103: If the preprocessing component is triggered, convert the database language in the target computing task according to the preprocessing component rule, and execute the target computing task using the converted database language.

[0034] If the trigger condition of the preprocessing component is met, convert the database language in the target computing task according to the preprocessing rule of the preprocessing component, and reasonably preprocess the business data. In the case of limited resources, the computing performance can be greatly improved. The target computing task is executed using the converted database language to reduce the required computing power.

[0035] S104: If the preprocessing component is not triggered, directly execute the target computing task.

[0036] If the trigger condition of the preprocessing component is not met, the target computing task is directly executed according to the original database language, and the result is returned.

[0037] In one embodiment, in the conversion of the database language, the report formula in the target report is first obtained, and the report formula is converted into a standard database language (Structured Query Language, sql), and then the standard database language is converted into a syntax tree, and the standard database language is parsed by traversing the syntax tree to obtain a data retrieval field, a data retrieval source table, and a query condition field. The same first identification (FID) is assigned to the formulas with the same data retrieval source. For example, three data retrieval formulas are all data retrieval from table one and table two, and the same first identification can be assigned to the three data retrieval formulas, and then the target data retrieval source table is established, and the data in the data retrieval source table is imported into the target data retrieval source table. The data retrieval source table in the standard database language is rewritten as the second identification of the target data retrieval source table, and the second identification is combined from the first identification and a system variable. Still taking the above three data retrieval formulas as an example, the three data retrieval formulas are all data retrieval from table one and table two, and the data to be retrieved can be imported into the newly established target data retrieval source table according to the three data retrieval formulas, and the target data retrieval source table is named, such as table a. It should be noted that in the naming, in order to prevent multiple threads from simultaneously preprocessing business data in a processor, the target data retrieval source table is named as FID<!PID! >, and <!PID! > represents a system variable, which will be replaced by the unique identification of the runtime thread in the subsequent. In the external display, only the FID is displayed, that is, table a.

[0038] As shown in Figure 2 In one embodiment, in the conversion of the database language, the total first identification corresponding to the target report can also be determined, and based on the execution order, the pre-processing configuration table is generated according to the first identification and the reconstructed standard database language. The pre-processing configuration table includes the database language for establishing the target data retrieval source table according to the second identification, and importing the data in the data retrieval source table into the target data retrieval source table, that is Figure 2 The part corresponding to the pre-processing step definition in

[0039] Further, in obtaining the computing power requirement of the target computing task, the task requirement of the target computing task is first received, and the computing content of the target computing task is determined through natural language processing technology, and then based on the computing content, the first identification quantity corresponding to the target computing task is obtained, that is, the formula name with the same data retrieval source table, and the formula quantity corresponding to each first identification, that is, the formula quantity corresponding to each first identification, and then the first identification quantity and the formula quantity are multiplied to determine the computing power requirement of the target computing task. As shown in Figure 3In the middle, the calculation task includes calculating 10000 units, all formulas on the report, at this time, because the formulas of different units have different data source tables, so the first identification quantity is 10000, that is, N is 10000. If the number of formulas corresponding to each first identification is 1, then M is 1 at this time.

[0040] As Figure 2 shown, in one embodiment, after parsing the standard database language, the data source corresponding to the first identification, the data field, the filtering condition, and the number of formulas can be recorded based on the obtained first identification to obtain a parsing result record table, as Figure 2 shown in the part of the parsing result record in the middle.

[0041] In one embodiment, after the target calculation task is executed using the converted database language, the table with the second identification as the name, that is, the self-built table, can be deleted asynchronously based on the obtained execution progress of the target calculation task.

[0042] As Figure 4 shown, the present application embodiment also provides a report calculation device based on a preprocessing component, comprising:

[0043] The trigger threshold module 201 obtains the computing power value of the current processor, and sets the trigger threshold based on the computing power value;

[0044] The judgment module 202 obtains the computing power requirement of the target calculation task, and judges whether the target calculation task triggers the preprocessing component based on the computing power requirement and the trigger threshold;

[0045] The preprocessing module 203, if the preprocessing component is triggered, converts the database language in the target calculation task according to the preprocessing component rule, and executes the target calculation task using the converted database language;

[0046] The direct execution module 204, if the preprocessing component is not triggered, directly executes the target calculation task.

[0047] As Figure 5 shown, the present application embodiment also provides a report calculation device based on a preprocessing component, comprising:

[0048] At least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0049] An algorithm value of a current processor is acquired, and a trigger threshold is set based on the algorithm value; an algorithm requirement of a target computing task is acquired, and it is judged whether the target computing task triggers a preprocessing component based on the algorithm requirement and the trigger threshold; if the preprocessing component is triggered, a database language in the target computing task is converted according to a preprocessing component rule, and the target computing task is executed using the converted database language; if the preprocessing component is not triggered, the target computing task is directly executed.

[0050] The embodiment of the present application further provides a nonvolatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are arranged as follows:

[0051] An algorithm value of a current processor is acquired, and a trigger threshold is set based on the algorithm value; an algorithm requirement of a target computing task is acquired, and it is judged whether the target computing task triggers a preprocessing component based on the algorithm requirement and the trigger threshold; if the preprocessing component is triggered, a database language in the target computing task is converted according to a preprocessing component rule, and the target computing task is executed using the converted database language; if the preprocessing component is not triggered, the target computing task is directly executed.

[0052] The embodiments in the present application are described in a progressive manner, and the same and similar parts of each embodiment can be referred to each other, and each embodiment mainly describes the difference from other embodiments. Especially, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple, and the related parts can be referred to the part of the method embodiment.

[0053] The device and medium provided by the embodiment of the present application are one-to-one corresponding with the method, so the device and medium also have the similar beneficial technical effects as the method, and the beneficial technical effects of the device and medium are not repeated here since the beneficial technical effects of the method have been described in detail above.

[0054] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0055] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0056] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0057] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0058] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0059] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer-readable media.

[0060] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0061] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0062] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A report calculation method based on a preprocessing component, characterized in that, include: Obtain the current computing power value of the processor, and set a trigger threshold based on the computing power value; Obtain the computing power requirement of the target computing task, and based on the computing power requirement and the trigger threshold, determine whether the target computing task triggers the preprocessing component; If the preprocessing component is triggered, the database language in the target computing task is converted according to the rules of the preprocessing component, and the target computing task is executed using the converted database language. If the preprocessing component is not triggered, the target computation task is executed directly. The step of converting the database language in the target computing task according to the preprocessing component rules specifically includes: Obtain the report formulas from the target report and convert the report formulas into standard database language; The standard database language is converted into a syntax tree, and the standard database language is parsed by traversing the syntax tree to obtain the data retrieval field, the data retrieval source table, and the query condition field. Assign the same first identifier to report formulas that have the same data source table; Establish a target data source table, and import the data from the data source table into the target data source table; The data source table in the standard database language is rewritten as the second identifier of the target data source table, wherein the second identifier is formed by combining the first identifier with a system variable; The step of converting the database language in the target computation task according to the preprocessing component rules, and then using the converted database language to execute the target computation task, specifically includes: Identify all first identifiers corresponding to the target report; Based on the execution order, a preprocessing configuration table is generated according to the first identifier and the reconstructed standard database language; The preprocessing configuration table also includes a database language for establishing a target data source table based on the second identifier and importing data from the data source table into the target data source table; After parsing the standard database language by traversing the syntax tree to obtain the data retrieval field, the data source table, and the query condition fields, the method further includes: Based on the first identifier, the data source, the data segment, the filtering conditions, and the number of formulas corresponding to the first identifier are recorded to obtain a parsing result record table; The acquisition of the computing power requirements of the target computing task specifically includes: Receive the task requirements of the target computing task and determine the computing content of the target computing task through natural language processing technology; Based on the calculation content, obtain the number of first identifiers corresponding to the target calculation task, and the number of formulas corresponding to each of the first identifiers; Based on the number of the first identifiers and the number of the formulas, the computing power requirement of the target computing task is determined.

2. The method according to claim 1, characterized in that, After executing the target computation task using the converted database language, the method further includes: Obtain the execution progress of the target computation task; Based on the execution progress, the table named after the second identifier is deleted asynchronously.

3. The method according to claim 1, characterized in that, The system variable is the thread identifier when the preprocessing component is executed.

4. A report calculation device based on a preprocessing component, characterized in that, include: The trigger threshold module obtains the current computing power value of the processor and sets a trigger threshold based on the computing power value; The judgment module obtains the computing power requirement of the target computing task, and determines whether the target computing task triggers the preprocessing component based on the computing power requirement and the trigger threshold. The acquisition of the computing power requirements of the target computing task specifically includes: Receive the task requirements of the target computing task and determine the computing content of the target computing task through natural language processing technology; Based on the calculation content, obtain the number of first identifiers corresponding to the target calculation task, and the number of formulas corresponding to each of the first identifiers; Based on the number of the first identifiers and the number of the formulas, the computing power requirement of the target computing task is determined; If the preprocessing module triggers the preprocessing component, it converts the database language in the target computing task according to the rules of the preprocessing component, and then uses the converted database language to execute the target computing task. The step of converting the database language in the target computing task according to the preprocessing component rules specifically includes: Obtain the report formulas from the target report and convert the report formulas into standard database language; The standard database language is converted into a syntax tree, and the standard database language is parsed by traversing the syntax tree to obtain the data retrieval field, the data retrieval source table, and the query condition field. Assign the same first identifier to report formulas that have the same data source table; Establish a target data source table, and import the data from the data source table into the target data source table; The data source table in the standard database language is rewritten as the second identifier of the target data source table, wherein the second identifier is formed by combining the first identifier with a system variable; The step of converting the database language in the target computation task according to the preprocessing component rules, and then using the converted database language to execute the target computation task, specifically includes: Identify all first identifiers corresponding to the target report; Based on the execution order, a preprocessing configuration table is generated according to the first identifier and the reconstructed standard database language; The preprocessing configuration table also includes a database language for establishing a target data source table based on the second identifier and importing data from the data source table into the target data source table; After parsing the standard database language by traversing the syntax tree to obtain the data retrieval field, the data retrieval source table, and the query condition field, the device further includes: Based on the first identifier, the data source, the data segment, the filtering conditions, and the number of formulas corresponding to the first identifier are recorded to obtain a parsing result record table; If the preprocessing component is not triggered, the target computation task is executed directly by the direct execution module.

5. A report calculation device based on a preprocessing component, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform: A report calculation method based on a preprocessing component as described in claim 1.

6. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, a report calculation method based on a preprocessing component as described in claim 1 is implemented.

Images