Statistical result streaming return method and device for data query and terminal equipment

Abstract

The invention provides a statistical result streaming return method and device for data query and terminal equipment, and the method comprises the steps: after determining a target index needing to be queried, preferentially querying a corresponding first statistical result for a first atomic index serving as a minimum statistical unit in the target index, and feeding back the first statistical result. And for a composite index needing to be calculated in the target indexes, splitting the composite index into a second atomic index, and after a second statistical result corresponding to the second atomic index is queried, calculating a composite statistical result of the composite index according to the second statistical result and feeding back the composite statistical result. According to the method and the device, the technical problem that the final statistical result can be fed back in a long time in the index query process in the prior art is solved in a mode of feeding back the statistical result of the target index in a streaming manner, and the use experience of a user is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and in particular to a method, apparatus, and terminal device for streaming statistical results of data queries. Background Art

[0002] Currently, mainstream data statistics programs generally use batch processing mode to calculate indicators. Its core process can be divided into four stages: the user submits a compound query request containing multiple related indicators, and the system decomposes the query request into multiple parallel execution subtasks; then the distributed computing framework shards the massive data and generates intermediate results through multi-node parallel computing; after the calculation tasks of all indicators are completed, the system integrates the scattered intermediate results through a merge sort algorithm, and finally generates a statistical report containing complete related indicators.

[0003] However, existing data statistics methods require waiting until all metrics in the same query are fully counted before obtaining the final statistical results. While this method ensures the integrity and consistency of the final results, complex query tasks can take a long time to provide final statistical results, resulting in long wait times for users. Furthermore, there is currently a lack of real-time statistical progress feedback, preventing users from understanding the progress of the data, resulting in a poor user experience. Summary of the Invention

[0004] The present invention provides a method, apparatus, and terminal device for streaming the statistical results of data queries. After determining the target indicator to be queried, the present invention prioritizes querying the first statistical result corresponding to the first atomic indicator, which serves as the smallest statistical unit within the target indicator, and provides feedback. For composite indicators that need to be calculated within the target indicator, feedback is provided after the corresponding composite statistical results are calculated. By providing streaming feedback of the statistical results of the target indicator, the present invention solves the technical problem in the prior art of requiring a long time to feedback the final statistical results during the indicator query process.

[0005] In a first aspect, an embodiment of the present invention provides a method for streaming statistical results of a data query, the method comprising:

[0006] Obtaining a target indicator for the current batch query, where the target indicator includes a first atomic indicator and a composite indicator, where the composite indicator is a complex statistical unit calculated from the second atomic indicator, and the first atomic indicator and the second atomic indicator are inseparable minimum statistical units;

[0007] Splitting the composite indicator into one or more second atomic indicators according to the calculation process of the composite indicator;

[0008] querying the first atomic indicator to obtain a corresponding first statistical result, and providing feedback;

[0009] When the query of the first atomic indicator is completed, the second atomic indicator is queried to obtain the corresponding second statistical result, and the composite statistical result of the composite indicator is calculated based on the second statistical result and the calculation process of the composite indicator, and feedback is provided.

[0010] The querying of the first atomic indicator to obtain a corresponding first statistical result and providing feedback includes:

[0011] Performing distributed processing on each of the first atomic indicators, and querying in parallel the first statistical result corresponding to each of the atomic indicators;

[0012] The first statistical results are fed back in a query completion order based on the first statistical results.

[0013] The step of calculating the composite statistical result of the composite indicator based on the second statistical result and the calculation process of the composite indicator includes:

[0014] Performing validity verification on the second statistical result corresponding to each of the second atomic indicators;

[0015] If the validity check passes, writing the second statistical result into a temporary buffer area;

[0016] For any one of the composite indicators, when all corresponding second statistical results used to calculate the composite indicator are detected in the temporary buffer area, a composite statistical result of the composite indicator is calculated according to the corresponding second statistical results.

[0017] The performing validity verification on the second statistical result corresponding to each second atomic indicator includes:

[0018] Determining the validity verification order of the second atomic indicators based on a preset priority determination rule;

[0019] According to the validity verification order, the validity of the second statistical result corresponding to each second atomic indicator is verified in turn.

[0020] The determining of the validity verification order of the second atomic indicators based on a preset priority determination rule includes:

[0021] Determining a service priority corresponding to each of the second atomic indicators, and determining a query time consumed and system resources occupied by querying the second statistical result of each of the second atomic indicators;

[0022] Determining a priority score corresponding to each second atomic indicator based on the service priority, query duration, and occupied system resources corresponding to each second atomic indicator;

[0023] The validity verification order of the second atomic indicators is determined according to the priority score.

[0024] Among them, also include:

[0025] The query progress of the target indicator is determined in real time based on the query progress of the first atomic indicator and the completion progress of the composite indicator, and the query progress is visualized.

[0026] The step of determining the query progress of the target indicator based on the query progress of the first atomic indicator and the completion progress of the composite indicator in real time includes:

[0027] Determining in real time the query progress of each of the first atomic indicators, the query progress of each of the second atomic indicators, and the calculation completion progress corresponding to each of the composite indicators;

[0028] Determining the completion progress of each composite indicator according to the query progress of the second atomic indicator corresponding to each composite indicator and the corresponding calculation completion progress;

[0029] The query progress of the target indicator is determined based on the query progress of each of the first atomic indicators and the completion progress of each of the composite indicators.

[0030] In a second aspect, an embodiment of the present invention provides a device for streaming statistical results of a data query, the device comprising:

[0031] a target indicator determination module configured to obtain a target indicator for a current batch query, wherein the target indicator includes a first atomic indicator and a composite indicator, wherein the composite indicator is a complex statistical unit calculated from a second atomic indicator, and wherein the first atomic indicator and the second atomic indicator are inseparable minimum statistical units;

[0032] an indicator splitting module, configured to split the composite indicator into one or more second atomic indicators according to a calculation process of the composite indicator;

[0033] A first result feedback module is configured to query the first atomic indicator to obtain a corresponding first statistical result and provide feedback;

[0034] The second result feedback module is configured to query the second atomic indicator to obtain the corresponding second statistical result when the first atomic indicator query is completed, calculate the composite statistical result of the composite indicator based on the second statistical result and the calculation process of the composite indicator, and provide feedback.

[0035] In a third aspect, an embodiment of the present invention provides a terminal device, the terminal device including a processor and a memory;

[0036] The memory is used to store a computer program and transmit the computer program to the processor;

[0037] The processor is configured to execute the method for streaming return of statistical results of data queries as described in the first aspect according to instructions in the computer program.

[0038] In a fourth aspect, an embodiment of the present invention provides a storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to execute the method for streaming return of statistical results of data queries as described in the first aspect.

[0039] The present invention provides a method, apparatus, and terminal device for streaming the return of statistical results of a data query. After determining the target indicator to be queried, the present invention preferentially queries the corresponding first statistical result for the first atomic indicator as the smallest statistical unit in the target indicator and provides feedback. For the composite indicator to be calculated in the target indicator, the composite indicator is split into second atomic indicators. After querying the second statistical result corresponding to the second atomic indicator, the composite statistical result of the composite indicator is calculated based on the second statistical result and provided as feedback. By providing streaming feedback of the statistical results of the target indicator, the present invention solves the technical problem in the prior art that it takes a long time to feedback the final statistical result during the query of the indicator, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of a method for streaming the return of statistical results of a data query provided by an embodiment of the present invention.

[0041] Figure 2 A schematic diagram of a process for a user to query a target indicator provided by an embodiment of the present invention.

[0042] Figure 3 A flowchart of another method for streaming the return of statistical results of data queries provided by an embodiment of the present invention.

[0043] Figure 4 A schematic structural diagram of a device for streaming return of statistical results of a data query provided by an embodiment of the present invention.

[0044] Figure 5 A schematic structural diagram of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following description and accompanying drawings sufficiently illustrate specific embodiments of the present application to enable those skilled in the art to practice them. The examples represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replace portions and features of other embodiments. The scope of the embodiments of the present application includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, each embodiment may be referred to individually or collectively by the term "invention," which is merely for convenience and is not intended to automatically limit the scope of the application to any single invention or inventive concept if more than one invention is in fact disclosed. Herein, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus comprising a set of elements includes not only those elements, but also other elements not explicitly listed. The various embodiments herein are described in a progressive manner, with each embodiment focusing on the differences from the other embodiments. Reference can be made to the common and similar parts between the various embodiments. For structures, products, etc. disclosed in the embodiments, the description is relatively simple because they correspond to the parts disclosed in the embodiments. For relevant parts, refer to the method description.

[0046] like Figure 1 As shown, Figure 1 A flowchart of a method for streaming the statistical results of a data query provided by an embodiment of the present invention. The method for streaming the statistical results of a data query provided by an embodiment of the present invention can be executed by a terminal device, which can be implemented by software and / or hardware. The terminal device can be composed of two or more physical entities or a single physical entity. For example, the terminal device can be a computer, a host computer, or a tablet. The method for streaming the statistical results of a data query includes the following steps:

[0047] Step 101: Obtain the target indicator of the current batch query. The target indicator includes a first atomic indicator and a composite indicator. The composite indicator is a complex statistical unit calculated from the second atomic indicator. The first atomic indicator and the second atomic indicator are inseparable minimum statistical units.

[0048] In this embodiment, it is first necessary to obtain the target indicator of the current batch query, where the target indicator can be selected by the user through interaction with the terminal device, or the terminal device automatically determines it according to the current business statistical needs. The target indicator includes a first atomic indicator and a composite indicator, where the atomic indicator is an indivisible minimum statistical unit. The indivisible minimum statistical unit refers to a basic statistical standard that cannot be decomposed again in a business scenario and has clear calculation logic and business meaning. As the smallest unit of data statistics, the atomic indicator is directly associated with the original data or basic field and usually has atomic characteristics. For example, the atomic indicator can be the number of payers, the total amount of payments, and the daily sales. The composite indicator is a complex statistical unit calculated from the atomic indicator. For example, the composite indicator can be the per capita payment amount, and the calculation formula is: per capita payment amount = total amount of payments / number of payers. It should be noted that in this application, the first atomic indicator and the second atomic indicator are independent atomic-level statistical units. The ordinal numbers "first" and "second" are only used to identify the technical feature differences of different indicators (such as data sources, statistical dimensions or calculation rules), rather than to represent the derivative relationship or logical hierarchy between indicators. The first and second atomic indicators are both indivisible minimum statistical units, each corresponding to independent data definitions and source standards. The difference in their sources is indicated by ordinal numbers. It is also understandable that in some business scenarios, the first and second atomic indicators may refer to the same atomic indicator.

[0049] Step 102: Split the composite index into one or more second atomic indexes according to the composite index calculation process.

[0050] After determining the composite indicator in the target indicator, it is necessary to split the composite indicator into one or more second atomic indicators according to the calculation process of the composite indicator. For example, for the composite indicator "per capita payment amount", the composite indicator "per capita payment amount" can be split into two second atomic indicators, "total payment amount" and "number of payers" according to its calculation formula: per capita payment amount = total payment amount / number of payers. To determine the calculation process of the composite indicator, the user can write the calculation rules of each composite indicator into the server or local terminal device in advance. After the terminal device determines the composite indicator in the target indicator, it can query the corresponding calculation rules on the server or local terminal device according to the composite indicator, thereby determining the calculation process of the composite indicator.

[0051] Step 103: query the first atomic indicator to obtain the corresponding first statistical result, and provide feedback.

[0052] After splitting the composite indicator, the terminal device needs to first query the local database or server for the numerical value corresponding to each first atomic indicator to obtain the first statistical result corresponding to the first atomic indicator. After querying the first statistical result of the first atomic indicator, the first statistical result of the first atomic indicator is fed back to the user, for example, the first statistical result is visualized so that the user can know the statistical result of the first atomic indicator. In addition, it can be understood that in some business scenarios, the first atomic indicator and the second atomic indicator may refer to the same atomic indicator. In this case, it is necessary to query the first statistical result corresponding to the atomic indicator and provide feedback.

[0053] Step 104: When the query of the first atomic indicator is completed, the second atomic indicator is queried to obtain the corresponding second statistical result, and the composite statistical result of the composite indicator is calculated based on the second statistical result and the calculation process of the composite indicator, and feedback is provided.

[0054] After querying the first statistical results corresponding to all the first atomic indicators, the terminal device needs to further query the value of the second atomic indicator to obtain the second statistical result corresponding to the second atomic indicator (if the second atomic indicator and the first atomic indicator refer to the same atomic indicator, there is no need to query again). After querying the second statistical result corresponding to the second atomic indicator, it is necessary to calculate the composite statistical result corresponding to each composite indicator based on the second statistical result corresponding to the second atomic indicator required in the calculation process of each composite indicator, and feed the composite statistical result back to the user. For example, Figure 2 As shown, Figure 2A schematic diagram of a process for a user to query a target indicator is provided in an embodiment of the present invention. The target indicator to be queried by the user includes atomic indicator A, atomic indicator B, and composite indicator A, where the calculation process of composite indicator A is composite indicator A = atomic indicator B + atomic indicator C. A terminal device first determines, based on the target indicator, that a first atomic indicator includes atomic indicator A and atomic indicator B, and also determines, based on the calculation process of composite indicator A, that a second atomic indicator includes atomic indicator B and atomic indicator C. The terminal device then prioritizes querying the first statistical result corresponding to atomic indicator A and the first statistical result of atomic indicator B, and feeds the first statistical result corresponding to atomic indicator A and the first statistical result of atomic indicator B back to the front end for visualization. The terminal device then needs to further query the second statistical result corresponding to the second atomic indicator. Since atomic indicator B has already been queried during the query of the first atomic indicator, only the second statistical result corresponding to atomic indicator C needs to be queried. After the second statistical result corresponding to atomic indicator C is queried, composite indicator A is calculated based on the first statistical result corresponding to atomic indicator B and the second statistical result corresponding to atomic indicator C. After calculating the composite statistical result corresponding to composite indicator A, the composite statistical result is fed back to the front end for visualization. In addition, when returning the first statistical results and the composite statistical results, a real-time feedback mechanism based on WebSocket or a feedback mechanism based on Server-Sent Events (SSE) can be used. WebSocket has strong real-time performance and can support full-duplex real-time communication, but it involves link management and is more complex to implement. SSE is simple and easy to use and does not require active link management. The specific feedback method can be set according to actual needs and is not specifically limited in this embodiment.

[0055] As described above, the present invention provides a method for streaming the return of statistical results of data queries. After determining the target indicator to be queried, the present invention prioritizes querying the corresponding first statistical result for the first atomic indicator as the smallest statistical unit in the target indicator and provides feedback. For the composite indicator that needs to be calculated in the target indicator, the composite indicator is split into a second atomic indicator, and after querying the second statistical result corresponding to the second atomic indicator, the composite statistical result of the composite indicator is calculated based on the second statistical result and provided as feedback. The present invention solves the technical problem of the prior art that it takes a long time to feedback the final statistical result during the query of the indicator by providing streaming feedback of the statistical results of the target indicator, thereby improving the user experience.

[0056] The embodiment of the present invention also provides another method for returning the statistical results of data query in a streaming manner, such as Figure 3 As shown, Figure 3A flow chart of another method for streaming the return of statistical results of a data query provided by an embodiment of the present invention is provided. The method for streaming the return of statistical results of a data query provided by an embodiment of the present invention is a specialization of the method for streaming the return of statistical results of a data query described above. The method for streaming the return of statistical results of a data query provided by an embodiment of the present invention includes:

[0057] Step 201: Obtain the target indicator of the current batch query. The target indicator includes a first atomic indicator and a composite indicator. The composite indicator is a complex statistical unit calculated from the second atomic indicator. The first atomic indicator and the second atomic indicator are inseparable minimum statistical units.

[0058] Step 202: Split the composite index into one or more second atomic indexes according to the composite index calculation process.

[0059] Step 203: Perform distributed processing on each first atomic indicator, and query the first statistical result corresponding to each atomic indicator in parallel.

[0060] In this embodiment, when querying the first atomic indicator, in order to improve query speed and query efficiency, distributed processing may be performed on each first atomic indicator, and the first statistical result corresponding to each atomic indicator may be queried in parallel.

[0061] Step 204: Feedback the first statistical result based on the query completion order of the first statistical result.

[0062] After the first statistical result corresponding to the first atomic indicator is queried, the terminal device can feedback the first statistical result based on the query completion order of the first statistical result, that is, the first statistical result that is queried first is fed back first. It should be noted that there is no need to wait until all first atomic indicators are queried before feedback is fed back in the query completion order. The first statistical result can be fed back simultaneously during the process of querying the first atomic indicator.

[0063] Step 205: When the query of the first atomic indicator is completed, query the second atomic indicator to obtain a corresponding second statistical result.

[0064] Step 206: Perform validity verification on the second statistical result corresponding to each second atomic indicator.

[0065] After querying the second statistical result corresponding to each second atomic indicator, it is necessary to further perform validity verification on the second statistical result to exclude erroneous data, improve the accuracy and completeness of the data, improve the accuracy of the subsequent calculation of the composite statistical results, and reduce data redundancy and conflict. In one embodiment, the validity verification order of the second atomic indicators can be determined based on the preset priority determination rule, and then the second statistical result corresponding to each second atomic indicator is verified in turn according to the validity verification order. The priority determination rule is used to determine the verification priority of each second atomic indicator, and the second atomic indicator with a higher verification priority is given priority for validity verification. In this embodiment, the priority verification rule needs to be set in advance. For example, the priority determination rule can be set to determine the verification priority according to the degree of influence of the second atomic indicator on the business, or the importance of the indicator can be divided by data domain according to the results of business research and demand analysis. For example, indicators associated with core business domains (such as transactions, users) are checked first. In one embodiment, the validity verification order of the second atomic indicator is determined based on the preset priority determination rule, including the following steps:

[0066] Step 2061: Determine the service priority corresponding to each second atomic indicator, and determine the query time consumed and the system resources occupied by querying the second statistical result of each second atomic indicator.

[0067] In this embodiment, when determining the validity verification order corresponding to the second atomic indicators, it is first necessary to determine the business priority corresponding to each second atomic indicator, and determine the query time consumed by querying the second statistical result of each second atomic indicator and the system resources occupied. The business priority corresponding to each second atomic indicator can be divided in advance according to the degree of alignment between business goals and strategies (such as core transaction indicators and auxiliary operation indicators) or according to the business risks (such as financial settlement error risks) and the scope of impact that may be caused by the failure of evaluation indicators. The query time consumed by the second statistical result corresponding to each second atomic indicator can be determined by performing log analysis or database execution plan analysis, and the system resources occupied by querying the second statistical result of each second atomic indicator can be obtained by monitoring resource consumption such as CPU, memory, and I / O.

[0068] Step 2062: Determine the priority score corresponding to each second atomic indicator based on the service priority, query duration, and occupied system resources corresponding to each second atomic indicator.

[0069] After determining the business priority, query duration, and system resources occupied by each second atomic indicator, the priority score corresponding to each second atomic indicator can be determined. In one embodiment, weights are assigned according to business value (such as α = 60%), and the core indicator score = business priority × α. Then, the weights of resource consumption (β, γ) are set, and the weights of query duration (such as β = 25%) and resource occupancy (such as γ = 15%) are set respectively, and the resource consumption score = (query duration coefficient × β) + (resource occupancy coefficient × γ). The query duration coefficient is set according to the time-consuming interval in which the query duration is located (such as <3 seconds, query duration coefficient = 0.6, 3-10 seconds, query duration coefficient = 0.3, >10 seconds = 0.1), and the resource occupancy coefficient can be defined when determining the key resource type (CPU, memory, I / O), and calculated after assigning weights according to resource scarcity (such as CPU weight 40%, memory 30%, I / O 30%). For example, if a query for a second atomic metric consumes 50% of CPU and 20% of memory, the comprehensive resource coefficient = 50% × 40% + 20% × 30% = 0.26. To calculate the priority score for each second atomic metric, the priority score = core metric score + resource consumption score.

[0070] Step 2063: Determine the validity verification order of the second atomic indicator according to the priority score.

[0071] After determining the priority score, the validity check order of the second atomic indicator can be determined according to the priority score, so that the validity check of the second statistical result corresponding to each second atomic indicator can be performed in sequence according to the validity check order. It can be understood that the higher the priority score of the second atomic indicator, the higher the position of the second atomic indicator in the validity check order.

[0072] The above is the process of determining the validity check order. After the validity check order is determined, the validity check can be performed on the second statistical result corresponding to the second atomic indicator according to the validity check order.

[0073] Step 207: If the validity check passes, write the second statistical result into a temporary buffer area.

[0074] For each second statistical result corresponding to the second atomic indicator, if the validity check passes, the second statistical result corresponding to the second atomic indicator is written into the temporary buffer area. For the second statistical result that fails the validity check, it can be corrected or re-queried.

[0075] Step 208: For any composite indicator, when all corresponding second statistical results used to calculate the composite indicator are detected in the temporary buffer area, a composite statistical result of the composite indicator is calculated according to the corresponding second statistical results and fed back.

[0076] In the process of writing the second statistical results into the temporary cache area, for any composite indicator, if all corresponding second statistical results used to calculate the composite indicator are detected in the temporary cache area, the composite statistical result of the composite indicator can be calculated based on the second statistical results corresponding to the composite indicator, and the calculated composite statistical result can be fed back to the user.

[0077] Step 209: Determine the query progress of the target indicator based on the query progress of the first atomic indicator and the completion progress of the composite indicator in real time, and visualize the query progress.

[0078] In addition, in the process of querying the target indicator, this embodiment also needs to provide real-time feedback on the query progress of the target indicator and perform a visual display so that the user can know the query progress of the target indicator. Specifically, the query progress of the target indicator can be determined based on the query progress of the first atomic indicator and the completion progress of the composite indicator. For example, the terminal device determines the query progress of the target indicator based on the query completion percentage of the first atomic indicator and the calculation completion percentage of the composite indicator.

[0079] On the basis of the above embodiment, the query progress of the target indicator is determined in real time based on the query progress of the first atomic indicator and the completion progress of the composite indicator, including:

[0080] Step 2091: Determine in real time the query progress of each first atomic indicator, the query progress of each second atomic indicator, and the calculation completion progress corresponding to each composite indicator.

[0081] In one embodiment, the terminal device may establish an independent progress tracking unit for the query process of each first atomic indicator, the query process of each second atomic indicator, and the calculation process of each composite indicator, so as to monitor the query progress of each first atomic indicator, the query progress of each second atomic indicator, and the calculation completion progress corresponding to each composite indicator in real time.

[0082] Step 2092: Determine the completion progress of each composite indicator based on the query progress of the second atomic indicator corresponding to each composite indicator and the corresponding calculation completion progress.

[0083] When determining the completion progress of each composite indicator, the completion progress of each composite indicator is divided into two parts: the query progress of the second atomic indicator and the calculation completion progress of the composite indicator. The query progress of the second atomic indicator can be determined by calculating the average query progress of all the second atomic indicators on which the composite indicator depends. For example, if the composite indicator depends on three second atomic indicators, and the progress of each is 50%, 100%, and 0% respectively, the average progress is 50%. After the query of the second atomic indicator is completed, the calculation completion progress of the corresponding composite indicator is tracked. For example, if the calculation process of the composite indicator includes two steps, the calculation completion progress increases by 50% each time a step is completed. After determining the query progress of the second atomic indicator corresponding to each composite indicator and the corresponding calculation completion progress, the completion progress of each composite indicator can be further determined. For example, the calculation time of the composite indicator and the query time of the corresponding second atomic indicator can be predicted in advance, and weights can be assigned according to the ratio of the predicted query time to the calculation time. The completion progress of each composite indicator is then determined by weighted summation. For example, if the query time of the second atomic indicator accounts for 70% of the total time, and the calculation time of the composite indicator accounts for 30% of the total time, then the completion progress of the composite indicator = query progress × 70% + calculation completion progress × 30%.

[0084] Step 2093: Determine the query progress of the target indicator based on the query progress of each first atomic indicator and the completion progress of each composite indicator.

[0085] Finally, the query progress of the target indicator can be determined based on the query progress of each first atomic indicator and the completion progress of each composite indicator. Specifically, when determining the query progress of the target indicator, the total progress of the first atomic indicator can be determined based on the query progress of each first atomic indicator, and the total progress of the composite indicator can be determined based on the completion progress of each composite indicator. Among them, the total progress of the first atomic indicator can be obtained by determining the average value of the progress of all first atomic indicators. For example, if the three first atomic indicators are completed 100%, 80% and 60% respectively, the total progress of the first atomic indicator is (100+80+60) / 3=80%. The total progress of the composite indicator is the average value of the completion progress of all composite indicators. For example, if the completion progress of two composite indicators is 40% and 70% respectively, the total progress of the composite indicator is (40+70) / 2=55%. Finally, the query progress of the target indicator is determined based on the ratio of the number of first atomic indicators to composite indicators in the target indicator. For example, if the target indicator consists of five first atomic indicators and three compound indicators, the query progress of the target indicator = (total progress of the first atomic indicators × 5 + total progress of the compound indicators × 3) / 8. After the query progress of the target indicator is determined in real time, the query progress of the target indicator can be fed back to the user, and the query progress of the target indicator can be visualized.

[0086] As described above, an embodiment of the present invention provides a method for streaming the return of statistical results of data queries. The present invention solves the technical problem of the prior art that it takes a long time to feedback the final statistical results during the query of indicators by streaming the statistical results of the target indicators. In addition, after querying the second statistical results corresponding to each second atomic indicator, the embodiment of the present invention can eliminate erroneous data, improve the accuracy and completeness of the data, improve the accuracy of subsequent calculations of composite statistical results, and reduce data redundancy and conflicts by performing validity verification on the second statistical results. Secondly, in the process of querying the target indicator, the embodiment of the present invention will determine the query progress of the target indicator in real time and provide feedback, so that the user can manage the waiting time expectation according to the query progress of the target indicator, thereby improving the user experience.

[0087] The embodiment of the present invention also provides a data query statistical result streaming return device, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of a device for returning statistical results of data queries in a streaming manner provided by an embodiment of the present invention. The device for returning statistical results of data queries in a streaming manner provided by an embodiment of the present invention includes:

[0088] The target indicator determination module 301 is configured to obtain the target indicator of the current batch query. The target indicator includes a first atomic indicator and a composite indicator. The composite indicator is a complex statistical unit calculated from the second atomic indicator. The first atomic indicator and the second atomic indicator are inseparable minimum statistical units.

[0089] The indicator splitting module 302 is configured to split the composite indicator into one or more second atomic indicators according to the calculation process of the composite indicator.

[0090] The first result feedback module 303 is configured to query the first atomic indicator to obtain the corresponding first statistical result and provide feedback.

[0091] The second result feedback module 304 is configured to query the second atomic indicator to obtain the corresponding second statistical result when the first atomic indicator query is completed, calculate the composite statistical result of the composite indicator based on the second statistical result and the calculation process of the composite indicator, and provide feedback.

[0092] The first result feedback module 303 includes:

[0093] a parallel processing submodule configured to perform distributed processing on each first atomic indicator and query the first statistical result corresponding to each atomic indicator in parallel;

[0094] The first result feedback submodule is configured to feed back the first statistical result based on the query completion order of the first statistical result.

[0095] The second result feedback module 304 includes:

[0096] A validity check submodule, configured to perform validity check on the second statistical result corresponding to each second atomic indicator;

[0097] a data cache submodule, configured to write the second statistical result into a temporary cache area if the validity check passes;

[0098] The composite statistical result determination submodule is configured to, for any composite indicator, calculate the composite statistical result of the composite indicator according to the corresponding second statistical results when all corresponding second statistical results used to calculate the composite indicator are detected in the temporary buffer area.

[0099] Among them, the validity check submodule includes:

[0100] a verification order determination unit, configured to determine the validity verification order of the second atomic indicators based on a preset priority determination rule;

[0101] The validity verification unit is configured to perform validity verification on the second statistical result corresponding to each second atomic indicator in turn according to the validity verification order.

[0102] Among them, the validity verification unit is specifically configured to determine the business priority corresponding to each second atomic indicator, and determine the query time consumed and the system resources occupied for querying the second statistical results of each second atomic indicator; based on the business priority, query time and system resources occupied corresponding to each second atomic indicator, determine the priority score corresponding to each second atomic indicator; determine the validity verification order of the second atomic indicators according to the priority score.

[0103] Among them, also include:

[0104] The query progress feedback module is configured to determine the query progress of the target indicator in real time based on the query progress of the first atomic indicator and the completion progress of the composite indicator, and to visualize the query progress.

[0105] The query progress feedback module includes:

[0106] A progress monitoring submodule is configured to determine in real time the query progress of each first atomic indicator, the query progress of each second atomic indicator, and the calculation completion progress corresponding to each composite indicator;

[0107] A completion progress determination submodule configured to determine the completion progress of each composite indicator based on the query progress of the second atomic indicator corresponding to each composite indicator and the corresponding calculation completion progress;

[0108] The query progress determination submodule is configured to determine the query progress of the target indicator based on the query progress of each first atomic indicator and the completion progress of each composite indicator.

[0109] The apparatus for streaming return of statistical results of data queries provided in an embodiment of the present invention is included in a terminal device and can be used to execute the method for streaming return of statistical results of data queries provided in the above embodiments, and has corresponding functions and beneficial effects.

[0110] It is worth noting that in the embodiment of the above-mentioned device for streaming the return of statistical results of data queries, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the various functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0111] This embodiment also provides a terminal device, such as Figure 5 As shown, Figure 5 A schematic diagram of the structure of a terminal device provided in an embodiment of the present invention, wherein the terminal device 40 includes a processor 400 and a memory 401;

[0112] The memory 401 is used to store the computer program 402 and transmit the computer program 402 to the processor 400;

[0113] The processor 400 is configured to execute the steps in the embodiment of the method for returning statistical results of a data query in a streaming manner according to the instructions in the computer program 402 .

[0114] Exemplarily, computer program 402 may be divided into one or more modules / units, one or more of which are stored in memory 401 and executed by processor 400 to implement the present application. One or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of computer program 402 in terminal device 40.

[0115] The terminal device 40 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device 40 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will appreciate that Figure 5 It is merely an example of the terminal device 40 and does not constitute a limitation on the terminal device 40. The terminal device 40 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 40 may also include input and output devices, network access devices, buses, etc.

[0116] The processor 400 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0117] The memory 401 may be an internal storage unit of the terminal device 40, such as a hard disk or memory of the terminal device 40. The memory 401 may also be an external storage device of the terminal device 40, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the terminal device 40. Furthermore, the memory 401 may include both an internal storage unit of the terminal device 40 and an external storage device. The memory 401 is used to store computer programs and other programs and data required by the terminal device 40. The memory 401 may also be used to temporarily store data that has been output or is about to be output.

[0118] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0120] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0121] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes various media that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0123] An embodiment of the present invention further provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a method for streaming statistical results of a data query. The method includes the following steps:

[0124] Get the target indicator of the current batch query. The target indicator includes a first atomic indicator and a composite indicator. The composite indicator is a complex statistical unit calculated from the second atomic indicator. The first atomic indicator and the second atomic indicator are the smallest inseparable statistical units.

[0125] According to the calculation process of the composite indicator, the composite indicator is split into one or more second atomic indicators;

[0126] Querying the first atomic indicator to obtain a corresponding first statistical result and providing feedback;

[0127] When the query of the first atomic indicator is completed, the second atomic indicator is queried to obtain the corresponding second statistical result, and the composite statistical result of the composite indicator is calculated based on the second statistical result and the calculation process of the composite indicator, and feedback is provided.

[0128] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the embodiments of the present invention have been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the embodiments of the present invention, the embodiments of the present invention may also include more other equivalent embodiments, and the scope of the embodiments of the present invention is determined by the scope of the appended claims.

Claims

1. A method for returning statistical results of data queries in a streaming manner, characterized in that: The method comprises: Obtaining a target indicator for the current batch query, where the target indicator includes a first atomic indicator and a composite indicator, where the composite indicator is a complex statistical unit calculated from the second atomic indicator, and the first atomic indicator and the second atomic indicator are inseparable minimum statistical units; Splitting the composite indicator into one or more second atomic indicators according to the calculation process of the composite indicator; querying the first atomic indicator to obtain a corresponding first statistical result, and providing feedback; When the query of the first atomic indicator is completed, the second atomic indicator is queried to obtain the corresponding second statistical result, and the composite statistical result of the composite indicator is calculated based on the second statistical result and the calculation process of the composite indicator, and feedback is provided.

2. The method for returning statistical results of data query in a streaming manner according to claim 1, characterized in that: The querying of the first atomic indicator to obtain a corresponding first statistical result and providing feedback includes: Performing distributed processing on each of the first atomic indicators, and querying in parallel the first statistical result corresponding to each of the atomic indicators; The first statistical results are fed back in a query completion order based on the first statistical results.

3. The method for returning statistical results of data query in a streaming manner according to claim 1, characterized in that: The calculating of the composite statistical result of the composite indicator based on the second statistical result and the calculation process of the composite indicator includes: Performing validity verification on the second statistical result corresponding to each of the second atomic indicators; If the validity check passes, writing the second statistical result into a temporary buffer area; For any one of the composite indicators, when all corresponding second statistical results used to calculate the composite indicator are detected in the temporary buffer area, a composite statistical result of the composite indicator is calculated according to the corresponding second statistical results.

4. The method for returning statistical results of data query in a streaming manner according to claim 3, characterized in that: The performing validity verification on the second statistical result corresponding to each second atomic indicator includes: Determining the validity verification order of the second atomic indicators based on a preset priority determination rule; According to the validity verification order, the validity of the second statistical result corresponding to each second atomic indicator is verified in turn.

5. The method for returning statistical results of data query in a streaming manner according to claim 4, characterized in that: The determining of the validity verification order of the second atomic indicators based on a preset priority determination rule includes: Determining a service priority corresponding to each of the second atomic indicators, and determining a query time consumed and system resources occupied by querying the second statistical result of each of the second atomic indicators; Determining a priority score corresponding to each second atomic indicator based on the service priority, query duration, and occupied system resources corresponding to each second atomic indicator; The validity verification order of the second atomic indicators is determined according to the priority score.

6. The method for streaming statistical results of data query according to claim 1, characterized in that: Also includes: The query progress of the target indicator is determined in real time based on the query progress of the first atomic indicator and the completion progress of the composite indicator, and the query progress is visualized.

7. The method for streaming back statistical results of data query according to claim 6, characterized in that: The determining, in real time, the query progress of the target indicator based on the query progress of the first atomic indicator and the completion progress of the composite indicator includes: Determining in real time the query progress of each of the first atomic indicators, the query progress of each of the second atomic indicators, and the calculation completion progress corresponding to each of the composite indicators; Determining the completion progress of each composite indicator according to the query progress of the second atomic indicator corresponding to each composite indicator and the corresponding calculation completion progress; The query progress of the target indicator is determined based on the query progress of each of the first atomic indicators and the completion progress of each of the composite indicators.

8. A streaming return device for statistical results of data query, characterized in that: The device comprises: a target indicator determination module configured to obtain a target indicator for a current batch query, wherein the target indicator includes a first atomic indicator and a composite indicator, wherein the composite indicator is a complex statistical unit calculated from a second atomic indicator, and wherein the first atomic indicator and the second atomic indicator are inseparable minimum statistical units; an indicator splitting module, configured to split the composite indicator into one or more second atomic indicators according to a calculation process of the composite indicator; A first result feedback module is configured to query the first atomic indicator to obtain a corresponding first statistical result and provide feedback; The second result feedback module is configured to query the second atomic indicator to obtain the corresponding second statistical result when the first atomic indicator query is completed, calculate the composite statistical result of the composite indicator based on the second statistical result and the calculation process of the composite indicator, and provide feedback.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method for streaming return of statistical results of a data query according to any one of claims 1 to 7 according to instructions in the computer program.

10. A storage medium storing computer executable instructions, characterized in that: When executed by a computer processor, the computer executable instructions are used to execute the method for returning statistical results of a data query in a streaming manner according to any one of claims 1 to 7.

Images