Data statistics method and device, storage medium and computer equipment

Abstract

The invention relates to the technical field of financial science and technology and the technical field of computers, and discloses a data statistical method and device, a storage medium and computer equipment, and the method comprises the steps: obtaining a target statistical data table name and a demand data statistical mode; obtaining a target statistical data table in a preset data warehouse based on the target statistical data table name, and obtaining a target statistical field name and a target statistical field value corresponding to the target statistical field name based on the target statistical data table; and for each target statistical field value corresponding to the same target statistical field name, performing data statistics on the target statistical field value based on a preset data statistical statement corresponding to the demand data statistical mode to obtain a data statistical result of the target statistical field name, and returning the respective data statistical result of each target statistical field name. By pre-packaging various data statistical methods, a user does not need to independently perform logic verification on each field, so that repeated work is reduced, and the development efficiency is improved.

Description

Technical Field

[0001] The present application relates to the fields of financial technology and computer technology, and in particular to a data statistics method and apparatus, storage medium, and computer equipment. Background Art

[0002] In the field of financial technology, as the business of financial institutions continues to expand and deepen, the data tables accumulated in data warehouses are becoming increasingly rich, and the financial indicators and business dimensions contained in these tables are becoming more and more complicated, which undoubtedly brings unprecedented challenges to the accuracy and integrity verification of data. Especially in key financial application scenarios such as risk assessment, credit rating, and investment analysis, the need for verification of statistical indicators such as maximum, minimum, and mean is particularly prominent. However, although the logical framework of these common verification logics is relatively fixed, and the application between different data tables has a high similarity, if each data table requires financial technicians to manually write such verification logic, it will not only cause a lot of repetitive work and reduce work efficiency, but also easily increase the risk of errors when faced with massive fields, thereby affecting the accuracy and timeliness of financial decisions. Summary of the invention

[0003] In view of this, the present application provides a data statistics method and device, storage medium, and computer equipment. Based on the target statistical data table name, a target statistical data table is obtained in a preset data warehouse, and the target statistical field name and the target statistical field value corresponding to the target statistical field name are obtained based on the target statistical data table; for each target statistical field value corresponding to the same target statistical field name, data statistics are performed on the target statistical field value based on the preset data statistical statement corresponding to the required data statistical method, and the data statistical result of the target statistical field name is obtained, and the data statistical result of each target statistical field name is returned. By pre-packaging various data statistical methods, there is no need for users to perform logical verification for each field separately, which reduces duplication of work and improves development efficiency.

[0004] According to one aspect of the present application, a data statistics method is provided, which is applied to a data statistics system. The data statistics system supports multiple data statistics methods, each of which corresponds to a preset data statistics statement. The data statistics method includes finding the maximum value of data, finding the minimum value of data, summing data, and deduplicating data. The method includes:

[0005] Get the target statistical data table name and required data statistical method;

[0006] Based on the target statistical data table name, the target statistical data table is obtained in a preset data warehouse, and based on the target statistical data table, the target statistical field name and the target statistical field value corresponding to the target statistical field name are obtained, wherein the target statistical data table includes a plurality of target statistical field names, each target statistical field name corresponds to a plurality of target statistical field values, each target statistical field name corresponds to a field type, and the field type includes a numeric type and a non-numeric type;

[0007] For each target statistical field value corresponding to the same target statistical field name, data statistics are performed on the target statistical field value based on a preset data statistical statement corresponding to the required data statistical method to obtain a data statistical result of the target statistical field name;

[0008] Based on the data statistical results of each target statistical field name, a data statistical result table obtained by statistics of the target statistical data table in the required data statistical mode is obtained, and the data statistical result table is returned.

[0009] Optionally, the preset data statistics statement includes a preset data maximum value solving statement, a preset data minimum value solving statement, a preset data summation statement and a preset data deduplication counting statement, the data statistical result includes a data maximum value statistical result, a data minimum value statistical result, a data summation statistical result and a data deduplication counting statistical result, and the preset data statistics statement corresponding to the required data statistics method performs data statistics on the target statistical field value to obtain the data statistical result of the target statistical field name, including:

[0010] When the required data statistics method is to find the maximum value of data, the values ​​of each target statistical field are traversed based on the preset data maximum value solving statement to obtain the maximum value statistical result of the target statistical field name;

[0011] When the required data statistics method is to find the minimum value of data, traverse the values ​​of each target statistical field based on the preset data minimum value solution statement to obtain the data minimum value statistical result of the target statistical field name;

[0012] When the required data statistics method is data summation, the values ​​of each target statistical field are traversed based on the preset data summation statement to obtain the data summation statistical result of the target statistical field name;

[0013] When the required data statistics method is data deduplication counting, each target statistical field value is traversed based on the preset data deduplication counting statement, and after removing the duplicate target statistical field values, the remaining non-duplicate target statistical field values ​​are counted to obtain the data deduplication counting statistical result of the target statistical field name.

[0014] Optionally, based on the target statistical data table name, obtaining a target statistical data table in a preset data warehouse, and obtaining a target statistical field name and a target statistical field value corresponding to the target statistical field name based on the target statistical data table includes:

[0015] In the preset data warehouse, query the target statistical data table structure information corresponding to the target statistical data table name;

[0016] Based on the target statistical data table structure information, each target statistical field name corresponding to the target statistical data table and the target statistical field value corresponding to each target statistical field name are extracted.

[0017] Optionally, the demand data statistics method includes multiple methods, and the data statistics results based on the respective target statistical field names are used to obtain the data statistics result table of the target statistical data table under the demand data statistics method, including:

[0018] For any target statistical field name, based on the statistical results of the target statistical field name under each required data statistical mode, obtain the data statistical result table row information of the target statistical field name;

[0019] Based on the data statistics result table row information of each target statistical field name, a data statistics result table is obtained.

[0020] Optionally, the obtaining of the target statistical data table name and the required data statistics method corresponding to the data statistics instruction includes:

[0021] The target statistical data table name and the required data statistical method are obtained based on a preset data statistical time interval, wherein the preset data statistical time interval, the target statistical data table name and the required data statistical method are obtained after the user configures the data statistical system.

[0022] Optionally, before obtaining the target statistical data table name and the required data statistical method, the method further includes:

[0023] Obtain multiple data statistics methods and the corresponding data statistics statements for each data statistics method;

[0024] The data statistics statement is encapsulated in a data statistics system so that users can perform data statistics based on the data statistics system.

[0025] Optionally, the data statistics system is developed based on Scala.

[0026] According to another aspect of the present application, a data statistics device is provided, the device comprising:

[0027] The statistical demand determination module is used to obtain the target statistical data table name and the required data statistical method;

[0028] A statistical data acquisition module is used to acquire a target statistical data table in a preset data warehouse based on the target statistical data table name, and acquire a target statistical field name and a target statistical field value corresponding to the target statistical field name based on the target statistical data table, wherein the target statistical data table includes a plurality of target statistical field names, each target statistical field name corresponds to a plurality of target statistical field values, each target statistical field name corresponds to a field type, and the field type includes a numerical type and a non-numerical type;

[0029] A data statistics execution module, for performing data statistics on each target statistical field value corresponding to the same target statistical field name based on a preset data statistics statement corresponding to the required data statistics method, to obtain a data statistics result of the target statistical field name;

[0030] The statistical result returning module is used to obtain the data statistical result table of the target statistical data table under the required data statistical method based on the data statistical results of each target statistical field name, and return the data statistical result table.

[0031] Optionally, the preset data statistics statement includes a preset data maximum value solution statement, a preset data minimum value solution statement, a preset data summation statement and a preset data deduplication counting statement, the data statistics result includes a data maximum value statistical result, a data minimum value statistical result, a data summation statistical result and a data deduplication counting statistical result, and the data statistics execution module is further used to:

[0032] When the required data statistics method is to find the maximum value of data, the values ​​of each target statistical field are traversed based on the preset data maximum value solving statement to obtain the maximum value statistical result of the target statistical field name;

[0033] When the required data statistics method is to find the minimum value of data, traverse the values ​​of each target statistical field based on the preset data minimum value solution statement to obtain the data minimum value statistical result of the target statistical field name;

[0034] When the required data statistics method is data summation, the values ​​of each target statistical field are traversed based on the preset data summation statement to obtain the data summation statistical result of the target statistical field name;

[0035] When the required data statistics method is data deduplication counting, each target statistical field value is traversed based on the preset data deduplication counting statement, and after removing the duplicate target statistical field values, the remaining non-duplicate target statistical field values ​​are counted to obtain the data deduplication counting statistical result of the target statistical field name.

[0036] Optionally, the statistical data acquisition module is further used to:

[0037] In the preset data warehouse, query the target statistical data table structure information corresponding to the target statistical data table name;

[0038] Based on the target statistical data table structure information, each target statistical field name corresponding to the target statistical data table and the target statistical field value corresponding to each target statistical field name are extracted.

[0039] Optionally, the demand data statistics method includes multiple methods, and the statistical result returning module is further used to:

[0040] For any target statistical field name, based on the statistical results of the target statistical field name under each required data statistical mode, obtain the data statistical result table row information of the target statistical field name;

[0041] Based on the data statistics result table row information of each target statistical field name, a data statistics result table is obtained.

[0042] Optionally, the statistical demand determination module is further used to:

[0043] The target statistical data table name and the required data statistical method are obtained based on a preset data statistical time interval, wherein the preset data statistical time interval, the target statistical data table name and the required data statistical method are obtained after the user configures the data statistical system.

[0044] Optionally, the device further comprises a statistical packaging module, configured to:

[0045] Obtain multiple data statistics methods and the corresponding data statistics statements for each data statistics method;

[0046] The data statistics statement is encapsulated in a data statistics system so that users can perform data statistics based on the data statistics system.

[0047] Optionally, the data statistics system is developed based on Scala.

[0048] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned data statistics method is implemented.

[0049] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned data statistics method when executing the program.

[0050] By means of the above technical scheme, the present application provides a data statistics method and device, storage medium, and computer equipment, which obtains a target statistical data table in a preset data warehouse based on the target statistical data table name, and obtains the target statistical field name and the target statistical field value corresponding to the target statistical field name based on the target statistical data table; for each target statistical field value corresponding to the same target statistical field name, the target statistical field value is statistically analyzed based on the preset data statistical statement corresponding to the required data statistical method, the data statistical result of the target statistical field name is obtained, and the data statistical result of each target statistical field name is returned. By pre-packaging various data statistical methods, the user does not need to perform logical verification for each field separately, which reduces duplication of work and improves development efficiency.

[0051] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 A schematic diagram of a data statistics method provided in an embodiment of the present application is shown;

[0054] Figure 2 A schematic diagram showing a flow chart of another data statistics method provided in an embodiment of the present application is shown;

[0055] Figure 3 A schematic diagram of a data statistical result table provided in an embodiment of the present application is shown;

[0056] Figure 4 A schematic diagram showing a flow chart of another data statistics method provided in an embodiment of the present application is shown;

[0057] Figure 5 A schematic diagram of the structure of a data statistics device provided in an embodiment of the present application is shown;

[0058] Figure 6 A schematic diagram of the structure of another data statistics device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0060] In this embodiment, a data statistics method is provided, such as Figure 1 As shown, it is applied to a data statistics system, the data statistics system supports multiple data statistics methods, each data statistics method corresponds to a preset data statistics statement, the data statistics method includes finding the maximum value of data, finding the minimum value of data, summing data and deduplicating data, and the method includes:

[0061] Step 101, obtain the target statistical data table name and the required data statistics method.

[0062] In the above embodiments of the present application, for example, it can be applied to the field of financial technology. In the daily operation of financial institutions, it is often necessary to collect and analyze a large amount of financial data to assist decision makers in making more informed decisions. For example, banks need to collect statistics on key indicators such as bad debt rates and yields of various loan businesses in order to assess business risks and optimize business strategies.

[0063] When the bank's data analysts issue data statistics instructions based on the data statistics system on the operating terminal, that is, after configuring the target statistical data table name and the required data statistics method through the data statistics system, the data statistics system first parses the aforementioned data statistics instructions to determine the target statistical data table name that needs to be counted and the specific required data statistics method (such as sum, average, maximum value, etc.).

[0064] Step 102, based on the target statistical data table name, obtain the target statistical data table in the preset data warehouse, and obtain the target statistical field name and the target statistical field value corresponding to the target statistical field name based on the target statistical data table, wherein the target statistical data table includes multiple target statistical field names, each target statistical field name corresponds to multiple target statistical field values, each target statistical field name corresponds to a field type, and the field type includes numeric type and non-numeric type.

[0065] Next, the data statistics system searches for and obtains the target statistical data table in the preset data warehouse. The aforementioned data warehouse can be a database within the bank, storing a large amount of business data. After obtaining the target statistical data table, the data statistics system further analyzes the structure of the target statistical data table, identifies the target statistical field name and the target statistical field value of the aforementioned target statistical field name, and in particular, each target statistical field name corresponds to a field type (such as numeric type, non-numeric type, etc.). The target statistical field names are, for example, age and nickname, etc. Correspondingly, the field values ​​under age are, for example, 22, 30, 34, 26, etc., and the field values ​​under nickname are, for example, aaa, zzz, etc.

[0066] Step 103 , for each target statistical field value corresponding to the same target statistical field name, data statistics are performed on the target statistical field value based on a preset data statistical statement corresponding to the required data statistical method to obtain a data statistical result of the target statistical field name.

[0067] Step 104, based on the data statistical results of each target statistical field name, obtain a data statistical result table of the target statistical data table in the required data statistical mode, and return the data statistical result table.

[0068] Next, the data statistics system selects a preset data statistics statement corresponding to the required data statistics method according to the required data statistics method. The aforementioned preset data statistics statement can be an SQL query statement or other statement suitable for data statistics, which can perform accurate data statistics on the target statistical field name.

[0069] Finally, the data statistics system executes the preset data statistics statement, performs data statistics on each target statistical field value corresponding to the same target statistical field name, and finally forms a data statistical result table based on the data statistical results of each target statistical field name, and returns the aforementioned data statistical result table.

[0070] Currently, spark's DataFrame class, such as descibe and summary, supports few functions, such as summation, deduplication counting and other statistical methods are not supported. This application pre-packages various statistical methods into the data statistics system, allowing users to perform data statistics in a variety of statistical methods with one click, while reducing repeated logic verification work, and can quickly preview the data of the entire table according to various statistical indicators, realizing dynamic generation of verification results.

[0071] By applying the technical solution of this embodiment, based on the target statistical data table name, the target statistical data table is obtained in the preset data warehouse, and the target statistical field name and the target statistical field value corresponding to the target statistical field name are obtained based on the target statistical data table; for each target statistical field value corresponding to the same target statistical field name, the target statistical field value is statistically analyzed based on the preset data statistical statement corresponding to the required data statistical method, the data statistical result of the target statistical field name is obtained, and the data statistical result of each target statistical field name is returned. By pre-packaging various data statistical methods, the user does not need to perform logical verification for each field separately, which reduces duplication of work and improves development efficiency.

[0072] Further, as a refinement and expansion of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another data statistics method is provided, which is applied to a data statistics system. The data statistics system supports multiple data statistics methods, each of which corresponds to a preset data statistics statement. The data statistics method includes finding the maximum value of data, finding the minimum value of data, summing data, and de-duplicating data. Figure 2 As shown, the method includes:

[0073] Step 201, obtain the target statistical data table name and the required data statistics method.

[0074] Step 202: In a preset data warehouse, query the target statistical data table structure information corresponding to the target statistical data table name.

[0075] Step 203, based on the target statistical data table structure information, extract the target statistical field names corresponding to the target statistical data table, and the target statistical field values ​​corresponding to the target statistical field names, wherein the target statistical data table contains multiple target statistical field names, each target statistical field name corresponds to multiple target statistical field values, each target statistical field name corresponds to a field type, and the field types include numeric and non-numeric types.

[0076] In the above embodiment of the present application, the data statistics system can develop a spark program through scala, and the user can configure the data statistics system in advance, and pass in the table to be read in the form of configuration. Then, the data statistics system obtains the target statistical data table name and the required data statistical method, and in the preset data warehouse, queries the target statistical data table structure information corresponding to the target statistical data table name. Then, based on the target statistical data table structure information, extract the target statistical field names corresponding to the target statistical data table, and the target statistical field values ​​corresponding to the target statistical field names, to prepare for subsequent data statistics. That is, the data statistics system dynamically reads the field name and field type of the table, and at the same time, according to the configured statistical method, in particular, if only the maximum value and the minimum value are required, the data statistics system only transmits these two items to save computing resources. For statistical methods that are only applicable to numerical types, such as percentiles, means, etc., the data statistics system can divide them into numerical fields and non-numerical fields according to the field type read, and perform applicable statistical methods on the field values ​​of the corresponding types separately to ensure the robustness of the program.

[0077] Step 204, for each target statistical field value corresponding to the same target statistical field name, when the required data statistics method is to find the data maximum value, traverse each target statistical field value based on a preset data maximum value solution statement to obtain the data maximum value statistical result of the target statistical field name.

[0078] Step 205, when the required data statistics method is to find the minimum value of data, traverse each target statistical field value based on a preset data minimum value solution statement to obtain the data minimum value statistical result of the target statistical field name.

[0079] Step 206, when the required data statistics method is data summation, traverse each target statistical field value based on a preset data summation statement to obtain the data summation statistical result of the target statistical field name.

[0080] Step 207, when the required data statistics method is data deduplication counting, traverse each target statistical field value based on the preset data deduplication counting statement, remove duplicate target statistical field values, and count the remaining non-duplicate target statistical field values ​​to obtain the data deduplication counting statistical result of the target statistical field name, wherein the preset data statistical statement includes a preset data maximum value solution statement, a preset data minimum value solution statement, a preset data summation statement and a preset data deduplication counting statement, and the data statistical results include data maximum value statistical results, data minimum value statistical results, data summation statistical results and data deduplication counting statistical results.

[0081] Next, for each target statistical field value corresponding to the same target statistical field name, when the required data statistical method is to find the maximum value of the data, the preset data maximum value solving statement is used to traverse each target statistical field value to obtain the maximum value statistical result of the target statistical field name. When the required data statistical method is to find the minimum value of the data, the preset data minimum value solving statement is used to traverse each target statistical field value to obtain the minimum value statistical result of the target statistical field name. When the required data statistical method is to sum the data, the preset data sum statement is used to traverse each target statistical field value to obtain the data sum statistical result of the target statistical field name. When the required data statistical method is to remove duplicate data and count, the preset data removal and counting statement is used to traverse each target statistical field value, and after removing duplicate target statistical field values, the remaining non-duplicate target statistical field values ​​are counted to obtain the deduplication counting statistical result of the target statistical field name.

[0082] In particular, various statistical methods can be uniformly abstracted into a base class Operation, which defines the basic interface and framework of statistical operations. For specific statistical requirements (user-configured statistical methods), such as sum operations and duplicate counting operations, they are implemented as sum Operation subclasses and duplicate counting Operation subclasses respectively. These subclasses inherit the base class Operation and implement specific statistical logic. At the same time, when the data statistical system is running, for unknown or newly added statistical methods, they can be converted into actual SQL operation statements through dynamic parsing and operation abstraction mechanisms. For example, the sum Operation subclass will be converted into a sum(field) statement when dynamically generating SQL, and the duplicate counting Operation subclass will be converted into a count(distinct field) statement. Finally, through this abstraction and dynamic generation mechanism of various Operations, not only the flexibility and scalability of the data statistical system are improved, but also the implementation and integration of subsequent newly added statistical methods are greatly facilitated.

[0083] Step 208, for any target statistical field name, based on the data statistical results obtained for the target statistical field name under each required data statistical method, obtain the data statistical result table row information of the target statistical field name, wherein the required data statistical methods include multiple methods.

[0084] Step 209, based on the data statistics result table row information of each target statistical field name, a data statistics result table is obtained, and the data statistics result table is returned.

[0085] Finally, for any target statistical field name, based on the statistical results of the target statistical field name under each required data statistical method, the data statistical result table row information of the target statistical field name is obtained, and based on the statistical result table row information of each target statistical field name, the data statistical result table is obtained, and the data statistical result table is returned. The returned data statistical result table is, for example, Figure 3 As shown, Figure 3 Field 1, field 2 to field n represent n different target statistical field names. Therefore, in the above data statistical result table, each row is the statistical information of each field name, and each column is the result of the corresponding required statistical method. Because the statistical method is relatively fixed, and the table structure may change frequently, when the table field name changes, the result will automatically change according to the field change, and no additional processing is required.

[0086] By applying the technical solution of this embodiment, in the preset data warehouse, the target statistical data table structure information corresponding to the target statistical data table name is queried, and the target statistical field names corresponding to the target statistical data table and the target statistical field values ​​corresponding to the target statistical field names are extracted. For each target statistical field value corresponding to the same target statistical field name, when the required data statistical method is to find the maximum value of the data, the preset data maximum value solution statement is used to traverse each target statistical field value to obtain the data maximum value statistical result of the target statistical field name. When the required data statistical method is to find the minimum value of the data, the preset data minimum value solution statement is used to traverse each target statistical field value to obtain the data minimum value statistical result of the target statistical field name. When the required data statistical method is to sum the data, the preset data summation statement is used to traverse each target statistical field value to obtain the data summation statistical result of the target statistical field name. When the required data statistics method is data deduplication counting, each target statistical field value is traversed based on the preset data deduplication counting statement, and after removing duplicate target statistical field values, the remaining non-duplicate target statistical field values ​​are counted to obtain the data deduplication counting statistics result of the target statistical field name. For any target statistical field name, based on the data statistical results of the target statistical field name under each required data statistical method, the data statistical result table row information of the target statistical field name is obtained, based on the data statistical result table row information of each target statistical field name, a data statistical result table is obtained, and the data statistical result table is returned. 通过 The required statistical method is configured to save computing resources. The data statistics system is developed by Scala. If there are new statistical methods in the future, they only need to be adapted according to the configuration. The system has high scalability. At the same time, if the default configuration is used to solidify the statistical method, users can directly use the system to obtain statistical information of each field, which is simple and easy to use.

[0087] Further, as a refinement and expansion of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another data statistics method is provided, which is applied to a data statistics system. The data statistics system supports multiple data statistics methods, each of which corresponds to a preset data statistics statement. The data statistics method includes finding the maximum value of data, finding the minimum value of data, summing data, and de-duplicating data. Figure 4 As shown, the method includes:

[0088] Step 301, obtaining multiple data statistics methods and data statistics statements corresponding to each data statistics method, and encapsulating the data statistics statements in a data statistics system so that users can perform data statistics based on the data statistics system, wherein the data statistics system is developed based on Scala.

[0089] Step 302, obtaining a target statistical data table name and a required data statistical method based on a preset data statistical time interval, wherein the preset data statistical time interval, the target statistical data table name and the required data statistical method are obtained after the user configures the data statistical system.

[0090] Step 303, based on the target statistical data table name, obtain the target statistical data table in the preset data warehouse, and obtain the target statistical field name and the target statistical field value corresponding to the target statistical field name based on the target statistical data table, wherein the target statistical data table includes multiple target statistical field names, each target statistical field name corresponds to multiple target statistical field values, each target statistical field name corresponds to a field type, and the field type includes numeric type and non-numeric type.

[0091] Step 304 , for each target statistical field value corresponding to the same target statistical field name, data statistics are performed on the target statistical field value based on a preset data statistical statement corresponding to the required data statistical method to obtain a data statistical result of the target statistical field name.

[0092] Step 305, based on the data statistical results of each target statistical field name, obtain the data statistical result table of the target statistical data table in the required data statistical mode, and return the data statistical result table.

[0093] In the above embodiments of the present application, a data statistics system can be pre-packaged. Specifically, multiple data statistics methods and data statistics statements corresponding to each data statistics method are obtained, and the data statistics statements are packaged in the data statistics system so that users can perform data statistics based on the data statistics system.

[0094] Next, the target statistical data table name and the required data statistical method are obtained based on the preset data statistical time interval. The preset data statistical time interval, the target statistical data table name and the required data statistical method can be obtained after the user configures the data statistical system. The preset data statistical time interval can be one day or other time periods. For this purpose, a time interval can be set in advance. Then, the data statistical system can perform data statistics automatically after reaching the predetermined time point without the need for the user to "start" it separately. Based on the target statistical data table name, the data statistical system obtains the target statistical data table in the preset data warehouse, and obtains the target statistical field name and the target statistical field value corresponding to the target statistical field name based on the target statistical data table. For each target statistical field value corresponding to the same target statistical field name, the target statistical field value is statistically analyzed based on the preset data statistical statement corresponding to the required data statistical method to obtain the data statistical result of the target statistical field name; based on the data statistical results of each target statistical field name, the data statistical result table of the target statistical data table under the required data statistical method is obtained, and the data statistical result table is returned.

[0095] By applying the technical solution of this embodiment, a variety of data statistics methods and the data statistics statements corresponding to each data statistics method are obtained, and the data statistics statements are encapsulated in the data statistics system so that users can perform data statistics based on the data statistics system. The target statistical data table name and the required data statistics method are obtained based on the preset data statistics time interval, the target statistical data table is obtained in the preset data warehouse based on the target statistical data table name, and the target statistical field name and the target statistical field value corresponding to the target statistical field name are obtained based on the target statistical data table. For each target statistical field value corresponding to the same target statistical field name, the target statistical field value is statistically analyzed based on the preset data statistics statement corresponding to the required data statistics method to obtain the data statistical result of the target statistical field name; based on the data statistical results of each target statistical field name, the data statistical result table of the target statistical data table under the required data statistics method is obtained, and the data statistical result table is returned. By pre-encapsulating various data statistical methods, there is no need for users to perform logical verification for each field separately, which reduces duplication of work and improves development efficiency.

[0096] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a data statistics device, such as Figure 5 As shown, the device comprises:

[0097] The statistical requirement determination module 401 is used to obtain the name of the target statistical data table and the required data statistical method;

[0098] The statistical data acquisition module 402 is used to acquire a target statistical data table in a preset data warehouse based on the target statistical data table name, and acquire a target statistical field name and a target statistical field value corresponding to the target statistical field name based on the target statistical data table, wherein the target statistical data table includes a plurality of target statistical field names, each target statistical field name corresponds to a plurality of target statistical field values, each target statistical field name corresponds to a field type, and the field type includes a numeric type and a non-numeric type;

[0099] The data statistics execution module 403 is used to perform data statistics on each target statistical field value corresponding to the same target statistical field name based on a preset data statistics statement corresponding to the required data statistics method, and obtain a data statistics result of the target statistical field name;

[0100] The statistical result returning module 404 is used to obtain a data statistical result table of the target statistical data table in the required data statistical mode based on the data statistical results of each target statistical field name, and return the data statistical result table.

[0101] Optionally, the preset data statistics statement includes a preset data maximum value solution statement, a preset data minimum value solution statement, a preset data summation statement and a preset data deduplication counting statement, the data statistics result includes a data maximum value statistical result, a data minimum value statistical result, a data summation statistical result and a data deduplication counting statistical result, and the data statistics execution module 403 is further used to:

[0102] When the required data statistics method is to find the maximum value of data, the values ​​of each target statistical field are traversed based on the preset data maximum value solving statement to obtain the maximum value statistical result of the target statistical field name;

[0103] When the required data statistics method is to find the minimum value of data, traverse the values ​​of each target statistical field based on the preset data minimum value solution statement to obtain the data minimum value statistical result of the target statistical field name;

[0104] When the required data statistics method is data summation, the values ​​of each target statistical field are traversed based on the preset data summation statement to obtain the data summation statistical result of the target statistical field name;

[0105] When the required data statistics method is data deduplication counting, each target statistical field value is traversed based on the preset data deduplication counting statement, and after removing the duplicate target statistical field values, the remaining non-duplicate target statistical field values ​​are counted to obtain the data deduplication counting statistical result of the target statistical field name.

[0106] Optionally, the statistical data acquisition module 402 is further configured to:

[0107] In the preset data warehouse, query the target statistical data table structure information corresponding to the target statistical data table name;

[0108] Based on the target statistical data table structure information, each target statistical field name corresponding to the target statistical data table and the target statistical field value corresponding to each target statistical field name are extracted.

[0109] Optionally, the demand data statistics method includes multiple methods, and the statistical result returning module 404 is further used to:

[0110] For any target statistical field name, based on the statistical results of the target statistical field name under each required data statistical mode, obtain the data statistical result table row information of the target statistical field name;

[0111] Based on the data statistics result table row information of each target statistical field name, a data statistics result table is obtained.

[0112] Optionally, the statistical demand determination module 401 is further used to:

[0113] The target statistical data table name and the required data statistical method are obtained based on a preset data statistical time interval, wherein the preset data statistical time interval, the target statistical data table name and the required data statistical method are obtained after the user configures the data statistical system.

[0114] Optionally, the data statistics system is developed based on Scala.

[0115] Furthermore, the present application provides another data statistics device, such as Figure 6 As shown, the device comprises:

[0116] The statistical requirement determination module 501 is used to obtain the name of the target statistical data table and the required data statistical method;

[0117] The statistical data acquisition module 502 is used to acquire a target statistical data table in a preset data warehouse based on the target statistical data table name, and acquire a target statistical field name and a target statistical field value corresponding to the target statistical field name based on the target statistical data table, wherein the target statistical data table includes a plurality of target statistical field names, each target statistical field name corresponds to a plurality of target statistical field values, each target statistical field name corresponds to a field type, and the field type includes a numeric type and a non-numeric type;

[0118] The data statistics execution module 503 is used to perform data statistics on each target statistical field value corresponding to the same target statistical field name based on a preset data statistics statement corresponding to the required data statistics method, and obtain a data statistics result of the target statistical field name;

[0119] The statistical result returning module 504 is used to obtain a data statistical result table of the target statistical data table in the required data statistical mode based on the data statistical results of each target statistical field name, and return the data statistical result table;

[0120] The statistical method encapsulation module 505 is used to obtain multiple data statistical methods and data statistical statements corresponding to each data statistical method; and encapsulate the data statistical statements in a data statistical system so that users can perform data statistics based on the data statistical system.

[0121] Optionally, the preset data statistics statement includes a preset data maximum value solution statement, a preset data minimum value solution statement, a preset data summation statement and a preset data deduplication counting statement, the data statistics result includes a data maximum value statistical result, a data minimum value statistical result, a data summation statistical result and a data deduplication counting statistical result, and the data statistics execution module 503 is further used to:

[0122] When the required data statistics method is to find the maximum value of data, the values ​​of each target statistical field are traversed based on the preset data maximum value solving statement to obtain the maximum value statistical result of the target statistical field name;

[0123] When the required data statistics method is to find the minimum value of data, traverse the values ​​of each target statistical field based on the preset data minimum value solution statement to obtain the data minimum value statistical result of the target statistical field name;

[0124] When the required data statistics method is data summation, the values ​​of each target statistical field are traversed based on the preset data summation statement to obtain the data summation statistical result of the target statistical field name;

[0125] When the required data statistics method is data deduplication counting, each target statistical field value is traversed based on the preset data deduplication counting statement, and after removing the duplicate target statistical field values, the remaining non-duplicate target statistical field values ​​are counted to obtain the data deduplication counting statistical result of the target statistical field name.

[0126] Optionally, the statistical data acquisition module 502 is further configured to:

[0127] In the preset data warehouse, query the target statistical data table structure information corresponding to the target statistical data table name;

[0128] Based on the target statistical data table structure information, each target statistical field name corresponding to the target statistical data table and the target statistical field value corresponding to each target statistical field name are extracted.

[0129] Optionally, the demand data statistics method includes multiple methods, and the statistical result returning module 504 is further used to:

[0130] For any target statistical field name, based on the statistical results of the target statistical field name under each required data statistical mode, obtain the data statistical result table row information of the target statistical field name;

[0131] Based on the data statistics result table row information of each target statistical field name, a data statistics result table is obtained.

[0132] Optionally, the statistical demand determination module 501 is further used to:

[0133] The target statistical data table name and the required data statistical method are obtained based on a preset data statistical time interval, wherein the preset data statistical time interval, the target statistical data table name and the required data statistical method are obtained after the user configures the data statistical system.

[0134] Optionally, the data statistics system is developed based on Scala.

[0135] It should be noted that for other corresponding descriptions of the functional units involved in the data statistics device provided in the embodiment of the present application, reference can be made to Figure 1 , Figure 2 and Figure 4 The corresponding description in the method will not be repeated here.

[0136] Based on the above Figure 1 , Figure 2 and Figure 4 The method shown in the embodiment of the present application accordingly provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 , Figure 2 and Figure 4 The statistical method of the data shown.

[0137] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0138] Based on the above Figure 1 , Figure 2and Figure 4 The method shown, and Figure 5 , Figure 6 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 , Figure 2 and Figure 4 The statistical method of the data shown.

[0139] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0140] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0141] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.

[0142] Through the description of the above implementation methods, the technical personnel in this field can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or can be implemented by hardware based on the target statistical data table name, obtain the target statistical data table in the preset data warehouse, and obtain the target statistical field name and the target statistical field value corresponding to the target statistical field name based on the target statistical data table; for each target statistical field value corresponding to the same target statistical field name, perform data statistics on the target statistical field value based on the preset data statistical statement corresponding to the required data statistical method, obtain the data statistical result of the target statistical field name, and return the data statistical result of each target statistical field name. By pre-packaging various data statistical methods, there is no need for users to perform logical verification for each field separately, which reduces duplication of work and improves development efficiency.

[0143] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0144] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A data statistics method, characterized in that: Applied to a data statistics system, the data statistics system supports multiple data statistics methods, each data statistics method corresponds to a preset data statistics statement, the data statistics method includes finding the maximum value of data, finding the minimum value of data, summing data and deduplicating data, and the method includes: Get the target statistical data table name and required data statistical method; Based on the target statistical data table name, the target statistical data table is obtained in a preset data warehouse, and based on the target statistical data table, the target statistical field name and the target statistical field value corresponding to the target statistical field name are obtained, wherein the target statistical data table includes a plurality of target statistical field names, each target statistical field name corresponds to a plurality of target statistical field values, each target statistical field name corresponds to a field type, and the field type includes a numeric type and a non-numeric type; For each target statistical field value corresponding to the same target statistical field name, data statistics are performed on the target statistical field value based on a preset data statistical statement corresponding to the required data statistical method to obtain a data statistical result of the target statistical field name; Based on the data statistical results of each target statistical field name, a data statistical result table obtained by statistics of the target statistical data table in the required data statistical mode is obtained, and the data statistical result table is returned.

2. The method according to claim 1, characterized in that The preset data statistics statement includes a preset data maximum value solution statement, a preset data minimum value solution statement, a preset data summation statement, and a preset data deduplication counting statement. The data statistical results include data maximum value statistics results, data minimum value statistics results, data summation statistics results, and data deduplication counting statistics results. The preset data statistics statement corresponding to the required data statistics method performs data statistics on the target statistical field value to obtain the data statistical results of the target statistical field name, including: When the required data statistics method is to find the maximum value of data, the values ​​of each target statistical field are traversed based on the preset data maximum value solving statement to obtain the maximum value statistical result of the target statistical field name; When the required data statistics method is to find the minimum value of data, traverse the values ​​of each target statistical field based on the preset data minimum value solution statement to obtain the data minimum value statistical result of the target statistical field name; When the required data statistics method is data summation, the values ​​of each target statistical field are traversed based on the preset data summation statement to obtain the data summation statistical result of the target statistical field name; When the required data statistics method is data deduplication counting, each target statistical field value is traversed based on the preset data deduplication counting statement, and after removing the duplicate target statistical field values, the remaining non-duplicate target statistical field values ​​are counted to obtain the data deduplication counting statistical result of the target statistical field name.

3. The method according to claim 1, characterized in that: The step of obtaining a target statistical data table in a preset data warehouse based on the target statistical data table name, and obtaining a target statistical field name and a target statistical field value corresponding to the target statistical field name based on the target statistical data table includes: In the preset data warehouse, query the target statistical data table structure information corresponding to the target statistical data table name; Based on the target statistical data table structure information, each target statistical field name corresponding to the target statistical data table and the target statistical field value corresponding to each target statistical field name are extracted.

4. The method according to claim 1, characterized in that The demand data statistics method includes multiple methods. The data statistics results based on the respective target statistical field names are used to obtain the data statistics result table of the target statistical data table under the demand data statistics method, including: For any target statistical field name, based on the statistical results of the target statistical field name under each required data statistical mode, obtain the data statistical result table row information of the target statistical field name; Based on the data statistics result table row information of each target statistical field name, a data statistics result table is obtained.

5. The method according to claim 1, characterized in that The obtaining of the target statistical data table name and the required data statistics method corresponding to the data statistics instruction includes: The target statistical data table name and the required data statistical method are obtained based on a preset data statistical time interval, wherein the preset data statistical time interval, the target statistical data table name and the required data statistical method are obtained after the user configures the data statistical system.

6. The method according to claim 1, characterized in that Before obtaining the target statistical data table name and the required data statistical method, the method further includes: Obtain multiple data statistics methods and the corresponding data statistics statements for each data statistics method; The data statistics statement is encapsulated in a data statistics system so that users can perform data statistics based on the data statistics system.

7. The method according to any one of claims 1 to 6, characterized in that The data statistics system is developed based on Scala.

8. A data statistics device, characterized in that: The device comprises: The statistical demand determination module is used to obtain the target statistical data table name and the required data statistical method; A statistical data acquisition module is used to acquire a target statistical data table in a preset data warehouse based on the target statistical data table name, and acquire a target statistical field name and a target statistical field value corresponding to the target statistical field name based on the target statistical data table, wherein the target statistical data table includes a plurality of target statistical field names, each target statistical field name corresponds to a plurality of target statistical field values, each target statistical field name corresponds to a field type, and the field type includes a numerical type and a non-numerical type; A data statistics execution module, for performing data statistics on each target statistical field value corresponding to the same target statistical field name based on a preset data statistics statement corresponding to the required data statistics method, to obtain a data statistics result of the target statistical field name; The statistical result returning module is used to obtain the data statistical result table of the target statistical data table under the required data statistical method based on the data statistical results of each target statistical field name, and return the data statistical result table.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for data warehouse data statistics described in any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method for data warehouse data statistics described in any one of claims 1 to 7 is implemented.

Images