A real-time ranking statistics method, system and storage medium based on multi-dimensional big data

Abstract

This application provides a method, system, and storage medium for real-time ranking statistics based on multi-dimensional big data. This application utilizes a multi-dimensional statistical form to store user data in a first database and synchronizes the user data to a second database. By configuring the aggregation statistical strategy of the second database, the user data is aggregated and statistically analyzed to generate ranking data, which is periodically sent and updated to obtain cached data. Based on user access requirements, the latest cached data is retrieved and sent to the terminal to be displayed in the form of a ranking list. This application, by establishing a multi-dimensional statistical form to process user data, can solve the problem of real-time aggregation statistics of large volumes of multi-dimensional data, with fast data response speed. The acquisition and calculation of ranking data are completed by two separate databases, eliminating the problem of high system memory usage caused by program calculation after data is read into memory, and also avoiding the phenomenon of MySQL aggregation calculation affecting the overall system performance.

Description

Technical Field

[0001] The present application belongs to the field of computer data processing technology, and more specifically, relates to a real-time ranking statistics method, system and storage medium based on multi-dimensional big data. Background Art

[0002] Rankings exist in many application systems, activities, and games, and data is mostly stored in relational databases such as ORACLE and MYSQL. When the system calculates real-time ranking data, there are several common solutions. Solution 1 is to read the data into memory, and the program script calculates the statistics and returns the results; Solution 2 is to use the query statement of the relational database itself to perform aggregate statistics and return the statistical results; Solution 3 is to calculate the statistical results of the existing data before storing the data each time, obtain the ranking data, and then store the new ranking data in the database for direct query.

[0003] The above three solutions are more common in ranking applications with smaller data volumes. However, when the data volume is large, Solution 1 will face the problem of occupying a large amount of memory space and slow calculation. Solution 2 has the disadvantage of slow database CURD, occupying a large amount of database computing resources, and causing a significant decrease in program performance. Solution 3 has certain advantages over the first two solutions, but under the premise of high concurrency and rapid incremental data, it still has the problem of slow calculation and high data statistical error rate. Summary of the invention

[0004] In view of this, the present application provides a real-time ranking statistics method, system and storage medium based on multi-dimensional big data, which can solve the real-time aggregation statistics of large amounts of data and multi-dimensional data, and has the advantages of fast response speed and strong stability.

[0005] The specific technical solutions of this application are as follows:

[0006] The first aspect of the present application provides a real-time ranking statistics method based on multi-dimensional big data, comprising the following steps:

[0007] Configure statistical dimension information and create multi-dimensional statistical forms based on the statistical dimension information;

[0008] Collect user data in the platform, insert the user data into a multi-dimensional statistical form according to the data type, and save it in the first database;

[0009] Synchronize user data to the second database, configure the aggregation statistics strategy of the second database, aggregate and count the user data to generate ranking data, and regularly send and update the ranking data to obtain cached data;

[0010] Receive user access demand information, retrieve the latest cached data based on the user access demand information, and send it to the terminal to display it in the form of a ranking list.

[0011] Preferably, the multi-dimensional statistical form is established according to the statistical dimension information as follows:

[0012] Acquire the data types generated by the platform, including time information, region information, frequency information, review information, rating information, and amount information;

[0013] Aggregate and analyze one or more information indicators in the data type to generate single-dimensional aggregate information, where the dimensional aggregation includes time aggregation, user aggregation, and organization aggregation;

[0014] Use the data type of dimension aggregation information under different dimensions as a template to create dimension statistics forms under each dimension.

[0015] Preferably, the user data in the collection platform is specifically:

[0016] Receive user behavior signals and collect single user data under all user behavior signals in the platform;

[0017] Monitor whether the deviation of data values ​​in the data types associated with the same user behavior signal exceeds a preset value;

[0018] If the value exceeds the preset value, the data value with a deviation exceeding the preset value is weightedly corrected according to the data sample size.

[0019] Preferably, synchronizing user data to the second database specifically includes:

[0020] Process the user data in the multi-dimensional statistical form according to the statistical dimension information;

[0021] Monitor the data sample size in a single data type. When the data sample size exceeds the preset column capacity, aggregate statistics are performed on the data values ​​in the data type to generate a first aggregate statistical result and use it as the data value under the data type for data overwriting;

[0022] The user data after column processing is saved in the second database in the form of column data.

[0023] Preferably, the aggregation statistics strategy for configuring the second database is specifically as follows:

[0024] Aggregate and count the data values ​​under all data categories respectively based on the columns of the second database;

[0025] For each type of statistical dimension information, one or several key data types are selected to introduce weight coefficients, and weighted calculation is performed on the data values ​​in the data type to generate a second aggregate statistical result.

[0026] Preferably, the user data is aggregated and counted to generate ranking data as follows:

[0027] Set the aggregated statistical results of each user to a value based on the application scenario. The value range is the maximum value, minimum value, sum value, average value or sample value.

[0028] According to the application scenario settings, the aggregated statistical values ​​of each user are sorted in a preset manner to generate ranking data.

[0029] Preferably, regularly sending and updating ranking data to obtain cached data is specifically as follows:

[0030] Send the generated ranking data to the server to generate historical cache data;

[0031] The update frequency of the cache data is configured in the manner of interval time setting, and the latest ranking data is regularly obtained from the second database according to the update frequency to generate the latest cache data;

[0032] Historical cache data is periodically marked as invalid based on the update frequency.

[0033] Preferably, it also includes:

[0034] The cached data is labeled according to the generation time, statistical dimension information and number of users;

[0035] The marked cache data is sent back to the first database regularly, and a link mode between the tag and the data is established.

[0036] A second aspect of the present application provides a real-time ranking statistics system based on multi-dimensional big data, including a memory and a processor, wherein the memory includes a real-time ranking statistics program based on multi-dimensional big data, and when the real-time ranking statistics program based on multi-dimensional big data is executed by the processor, the following steps are implemented:

[0037] Configure statistical dimension information and create multi-dimensional statistical forms based on the statistical dimension information;

[0038] Collect user data in the platform, insert the user data into a multi-dimensional statistical form according to the data type, and save it in the first database;

[0039] Synchronize user data to the second database, configure the aggregation statistics strategy of the second database, aggregate and count the user data to generate ranking data, and regularly send and update the ranking data to obtain cached data;

[0040] Receive user access demand information, retrieve the latest cached data based on the user access demand information, and send it to the terminal to display it in the form of a ranking list.

[0041] Preferably, the multi-dimensional statistical form is established according to the statistical dimension information as follows:

[0042] Acquire the data types generated by the platform, including time information, region information, frequency information, review information, rating information, and amount information;

[0043] Aggregate and analyze one or more information indicators in the data type to generate single-dimensional aggregate information, where the dimensional aggregation includes time aggregation, user aggregation, and organization aggregation;

[0044] Use the data type of dimension aggregation information under different dimensions as a template to create dimension statistics forms under each dimension.

[0045] Preferably, the user data in the collection platform is specifically:

[0046] Receive user behavior signals and collect single user data under all user behavior signals in the platform;

[0047] Monitor whether the deviation of data values ​​in the data types associated with the same user behavior signal exceeds a preset value;

[0048] If the value exceeds the preset value, the data value with a deviation exceeding the preset value is weightedly corrected according to the data sample size.

[0049] Preferably, synchronizing user data to the second database specifically includes:

[0050] Process the user data in the multi-dimensional statistical form according to the statistical dimension information;

[0051] Monitor the data sample size in a single data type. When the data sample size exceeds the preset column capacity, aggregate statistics are performed on the data values ​​in the data type to generate a first aggregate statistical result and use it as the data value under the data type for data overwriting;

[0052] The user data after column processing is saved in the second database in the form of column data.

[0053] Preferably, the aggregation statistics strategy for configuring the second database is specifically as follows:

[0054] Aggregate and count the data values ​​under all data categories respectively based on the columns of the second database;

[0055] For each type of statistical dimension information, one or several key data types are selected to introduce weight coefficients, and weighted calculation is performed on the data values ​​in the data type to generate a second aggregate statistical result.

[0056] Preferably, the user data is aggregated and counted to generate ranking data as follows:

[0057] Set the aggregated statistical results of each user to a value based on the application scenario. The value range is the maximum value, minimum value, sum value, average value or sample value.

[0058] According to the application scenario settings, the aggregated statistical values ​​of each user are sorted in a preset manner to generate ranking data.

[0059] Preferably, regularly sending and updating ranking data to obtain cached data is specifically as follows:

[0060] Send the generated ranking data to the server to generate historical cache data;

[0061] The update frequency of the cache data is configured in the manner of interval time setting, and the latest ranking data is regularly obtained from the second database according to the update frequency to generate the latest cache data;

[0062] Historical cache data is periodically marked as invalid based on the update frequency.

[0063] Preferably, it also includes:

[0064] The cached data is labeled according to the generation time, statistical dimension information and number of users;

[0065] The marked cache data is sent back to the first database regularly, and a link mode between the tag and the data is established.

[0066] A third aspect of the present application provides a computer-readable storage medium, which includes a real-time ranking statistics program based on multi-dimensional big data. When the real-time ranking statistics program based on multi-dimensional big data is executed by a processor, the steps of the real-time ranking statistics method based on multi-dimensional big data are implemented.

[0067] In summary, the present application provides a real-time ranking statistics method, system and storage medium based on multi-dimensional big data. The present application uses a multi-dimensional statistical form to save user data in a first database and synchronize user data to a second database. By configuring the aggregation statistical strategy of the second database, the user data is aggregated and statistically generated to generate ranking data and the ranking data is regularly sent and updated to obtain cached data; according to the user access demand information, the latest cached data is retrieved and sent to the terminal to be displayed in the form of a ranking list. The present application processes user data by establishing a multi-dimensional statistical form, which can solve the real-time aggregation statistics of large amounts of data and multi-dimensional data, and the data response speed is fast; the acquisition and calculation of the ranking data are completed by two databases respectively, which gets rid of the traditional problem of high system memory usage caused by the program calculating a large amount of data after reading it into the memory, and also gets rid of the phenomenon that the overall system performance is affected by MYSQL aggregation calculation; the aggregation dimension of the ranking data can be freely changed, and the user experience is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0069] Figure 1 A flowchart of a real-time ranking statistics method based on multi-dimensional big data is provided for this application;

[0070] Figure 2 This application provides a block diagram of a real-time ranking statistics system based on multi-dimensional big data. DETAILED DESCRIPTION

[0071] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application are clearly and completely described. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] Please refer to Figure 1 , Figure 1 The present invention provides a flowchart of a real-time ranking statistics method based on multi-dimensional big data.

[0073] The first aspect of the embodiment of the present application provides a real-time ranking statistics method based on multi-dimensional big data, comprising the following steps:

[0074] S102: Configure statistical dimension information, and create a multi-dimensional statistical form according to the statistical dimension information;

[0075] S104: Collect user data in the platform, insert the user data into a multi-dimensional statistical form according to the data type, and save it in the first database;

[0076] S106: Synchronize user data to the second database, configure the aggregation statistics strategy of the second database, aggregate and count the user data to generate ranking data, and regularly send and update the ranking data to obtain cached data;

[0077] S108: receiving user access demand information, retrieving the latest cached data according to the user access demand information, and sending it to the terminal for display in the form of a ranking list.

[0078] It should be noted that the first database is a relational database, such as MYSQL, which is the main database of the business application and is responsible for storing the data generated by the user using the system in the application, such as the user's answer score, the user's participation in the lottery record, etc. The second database is an OLAP relational database, such as CLICKHOUSE, which is a column database specifically used for OLAP analysis. It is also a relational database in essence. It adopts the idea of ​​space for time and undertakes aggregation statistics more than 100 times faster than MYSQL. It can be used as a slave database of MYSQL, such as the user score ranking obtained by the user's answer score dimension, the organization activity ranking obtained by the number of participating users, etc. The second database only performs incremental operations, and the data cannot be modified or deleted. The aggregation query task of the second database is performed asynchronously and is calculated by an independent process. After execution, the hourly ranking, daily ranking, and weekly ranking are generated. The cached data can use a data cache server, such as REDIS, which can be used as a ranking data cache for each aggregation query.

[0079] According to the embodiment of the present application, the multi-dimensional statistical form is established according to the statistical dimension information as follows:

[0080] Acquire the data types generated by the platform, including time information, region information, frequency information, review information, rating information, and amount information;

[0081] Aggregate and analyze one or more information indicators in the data type to generate single-dimensional aggregate information, where the dimensional aggregation includes time aggregation, user aggregation, and organization aggregation;

[0082] Use the data type of dimension aggregation information under different dimensions as a template to create dimension statistics forms under each dimension.

[0083] It should be noted that the data type is obtained from historical user data or system-generated data within the platform. When generating single-dimensional aggregate information, it can be determined based on the correlation between the dimension and the data type. For example, time dimension aggregation is used to obtain source data based on time information, user aggregation is used to obtain source data based on region information, frequency information, etc., and organization aggregation is used to obtain source data based on review information, rating information, amount information, etc.

[0084] According to the embodiment of the present application, the user data in the collection platform is specifically:

[0085] Receive user behavior signals and collect single user data under all user behavior signals in the platform;

[0086] Monitor whether the deviation of data values ​​in the data types associated with the same user behavior signal exceeds a preset value;

[0087] If the value exceeds the preset value, the data value with a deviation exceeding the preset value is weightedly corrected according to the data sample size.

[0088] It should be noted that user data can be behavioral data generated under user operation nodes within the platform. The data types associated under the same user behavior signal refer to data types that logically influence each other and have a joint effect, such as user reputation evaluation scores and official review information. The preset value of the degree of deviation can be set freely. Weighted correction based on the data sample size means that after weight analysis of the sample size of the data value, the weight coefficient is introduced to correct the data value with a larger degree of deviation, so as to eliminate the risk of large errors when the sample size is small.

[0089] According to an embodiment of the present application, synchronizing user data to the second database is specifically:

[0090] Process the user data in the multi-dimensional statistical form according to the statistical dimension information;

[0091] Monitor the data sample size in a single data type. When the data sample size exceeds the preset column capacity, aggregate statistics are performed on the data values ​​in the data type to generate a first aggregate statistical result and use it as the data value under the data type for data overwriting;

[0092] The user data after column processing is saved in the second database in the form of column data.

[0093] It should be noted that the second database performs operations in the form of column data. The data columns can be determined according to different statistical dimensions. Each statistical dimension can contain source data of one or more data types. For data that exceeds the column capacity, a fixed range can be defined for pre-aggregation statistics, and then sent to the second database for statistics. This can handle a large amount of data statistical operations and improve the system operation speed. The first aggregate statistical result not only includes the calculation results of the data value, but also includes the corresponding sample size, weight ratio and other information.

[0094] According to an embodiment of the present application, the aggregation statistics strategy for configuring the second database is specifically as follows:

[0095] Aggregate and count the data values ​​under all data categories respectively based on the columns of the second database;

[0096] For each type of statistical dimension information, one or several key data types are selected to introduce weight coefficients, and weighted calculation is performed on the data values ​​in the data type to generate a second aggregate statistical result.

[0097] It should be noted that the data types in each type of statistical dimension information can be partially repeated or the same, but the weight coefficients of key data types are different. The weight coefficient can be determined based on the degree of influence of the data type on the dimension statistics, and is generally between 0.5 and 2.

[0098] According to the embodiment of the present application, the user data is aggregated and counted to generate the ranking data as follows:

[0099] Set the aggregated statistical results of each user to a value based on the application scenario. The value range is the maximum value, minimum value, sum value, average value or sample value.

[0100] According to the application scenario settings, the aggregated statistical values ​​of each user are sorted in a preset manner to generate ranking data.

[0101] It should be noted that the application scenario can be a user-defined setting or a system default setting. The preset sorting method refers to the generated ranking data being arranged in ascending or descending order, which can be a user-defined setting or a system default setting.

[0102] According to the embodiment of the present application, regularly sending and updating the ranking data to obtain cached data is specifically as follows:

[0103] Send the generated ranking data to the server to generate historical cache data;

[0104] The update frequency of the cache data is configured in the manner of interval time setting, and the latest ranking data is regularly obtained from the second database according to the update frequency to generate the latest cache data;

[0105] Historical cache data is periodically marked as invalid based on the update frequency.

[0106] It should be noted that cached data can be automatically counted and updated according to the set interval, which can be daily, weekly, monthly, or annual. If new ranking data is generated, the historical cache data will be sealed, and the latest cache data will be retrieved each time as the ranking data requested by the user.

[0107] According to an embodiment of the present application, it also includes:

[0108] The cached data is labeled according to the generation time, statistical dimension information and number of users;

[0109] The marked cache data is sent back to the first database regularly, and a link mode between the tag and the data is established.

[0110] It should be noted that the cached data is stored in the first database for data traceability, and the data management is more stable. Setting tags for the cached data facilitates subsequent search queries and retrieval. The interval time for sending back can be consistent with the update frequency of the cached data.

[0111] In another embodiment of the present application, inserting user data into a multi-dimensional statistical table according to the data type is specifically as follows:

[0112] Write user data into the data category corresponding to the single-dimensional statistical form according to the data type;

[0113] Monitor whether there are data values ​​under each data category in the single-dimensional statistical form. If no data value exists, retrieve the initial value of the data type according to the statistical dimension information and write it into the corresponding data category.

[0114] It should be noted that if the data acquisition is incomplete or the data type does not exist, the initial value of the data type can be retrieved to complete the data. The initial value can be determined based on the average of the system's historical big data or customized based on industry experience.

[0115] In another embodiment of the present application, the latest cache data is retrieved according to the user access demand information as follows:

[0116] Collect statistical dimension information of user needs according to user access demand information, and use the statistical dimension information of user needs as a search formula to query corresponding ranking data in the server;

[0117] According to the user access demand information, the application scenario information of the user's demand is collected, and the ranking data is corrected and updated according to the application scenario information of the user's demand.

[0118] It should be noted that the user access demand information comes from the user's self-selected options. The user can select multiple requirements at the same time to retrieve multi-dimensional ranking data. The value rules of the current ranking data can be modified according to the application scenario information of the user's requirements.

[0119] In another embodiment of the present application, it also includes:

[0120] Monitor the data type and data value of the received user access demand information to determine whether there is matching statistical dimension information;

[0121] If there is no matching statistical dimension information, the statistical dimension information is reconfigured by performing aggregation analysis based on the data type and data value required by the user.

[0122] It should be noted that if the statistical dimension information in the cached data does not include the statistical dimension required by the user, the statistical dimension information can be reconfigured directly according to the current user's demand information, which can meet the statistical needs of different users and improve the user experience.

[0123] Please refer to Figure 2 , Figure 2 This application provides a block diagram of a real-time ranking statistics system based on multi-dimensional big data.

[0124] A second aspect of an embodiment of the present application provides a real-time ranking statistics system based on multi-dimensional big data, including a memory 21 and a processor 22, wherein the memory 21 includes a real-time ranking statistics program based on multi-dimensional big data, and when the real-time ranking statistics program based on multi-dimensional big data is executed by the processor 22, the following steps are implemented:

[0125] Configure statistical dimension information and create multi-dimensional statistical forms based on the statistical dimension information;

[0126] Collect user data in the platform, insert the user data into a multi-dimensional statistical form according to the data type, and save it in the first database;

[0127] Synchronize user data to the second database, configure the aggregation statistics strategy of the second database, aggregate and count the user data to generate ranking data, and regularly send and update the ranking data to obtain cached data;

[0128] Receive user access demand information, retrieve the latest cached data based on the user access demand information, and send it to the terminal to display it in the form of a ranking list.

[0129] According to the embodiment of the present application, the multi-dimensional statistical form is established according to the statistical dimension information as follows:

[0130] Obtain the data types generated by the platform, including time information, region information, frequency information, review information, rating information, and amount information;

[0131] Aggregate and analyze one or more information indicators in the data type to generate single-dimensional aggregate information, where the dimensional aggregation includes time aggregation, user aggregation, and organization aggregation;

[0132] Use the data type of dimension aggregation information under different dimensions as a template to create dimension statistics forms under each dimension.

[0133] According to the embodiment of the present application, the user data in the collection platform is specifically:

[0134] Receive user behavior signals and collect single user data under all user behavior signals in the platform;

[0135] Monitor whether the deviation of data values ​​in the data types associated with the same user behavior signal exceeds a preset value;

[0136] If the value exceeds the preset value, the data value with a deviation exceeding the preset value is weightedly corrected according to the data sample size.

[0137] According to an embodiment of the present application, synchronizing user data to the second database is specifically:

[0138] Process the user data in the multi-dimensional statistical form according to the statistical dimension information;

[0139] Monitor the data sample size in a single data type. When the data sample size exceeds the preset column capacity, aggregate statistics are performed on the data values ​​in the data type to generate a first aggregate statistical result and use it as the data value under the data type for data overwriting;

[0140] The user data after column processing is saved in the second database in the form of column data.

[0141] According to an embodiment of the present application, the aggregation statistics strategy for configuring the second database is specifically as follows:

[0142] Aggregate and count the data values ​​under all data categories based on the columns of the second database;

[0143] For each type of statistical dimension information, one or several key data types are selected to introduce weight coefficients, and weighted calculation is performed on the data values ​​in the data type to generate a second aggregate statistical result.

[0144] According to the embodiment of the present application, the user data is aggregated and counted to generate the ranking data as follows:

[0145] Set the aggregated statistical results of each user to a value based on the application scenario. The value range is the maximum value, minimum value, sum value, average value or sample value.

[0146] According to the application scenario settings, the aggregated statistical values ​​of each user are sorted in a preset manner to generate ranking data.

[0147] According to the embodiment of the present application, regularly sending and updating the ranking data to obtain cached data is specifically as follows:

[0148] Send the generated ranking data to the server to generate historical cache data;

[0149] The update frequency of the cache data is configured in the manner of interval time setting, and the latest ranking data is regularly obtained from the second database according to the update frequency to generate the latest cache data;

[0150] Historical cache data is periodically marked as invalid based on the update frequency.

[0151] According to an embodiment of the present application, it also includes:

[0152] The cached data is labeled according to the generation time, statistical dimension information and number of users;

[0153] The marked cache data is sent back to the first database regularly, and a link mode between the tag and the data is established.

[0154] In another embodiment of the present application, inserting user data into a multi-dimensional statistical table according to the data type is specifically as follows:

[0155] Write user data into the data category corresponding to the single-dimensional statistical form according to the data type;

[0156] Monitor whether there are data values ​​under each data category in the single-dimensional statistical form. If no data value exists, retrieve the initial value of the data type according to the statistical dimension information and write it into the corresponding data category.

[0157] In another embodiment of the present application, the latest cache data is retrieved according to the user access demand information as follows:

[0158] Collect statistical dimension information of user needs according to user access demand information, and use the statistical dimension information of user needs as a search formula to query corresponding ranking data in the server;

[0159] According to the user access demand information, the application scenario information of the user's demand is collected, and the ranking data is corrected and updated according to the application scenario information of the user's demand.

[0160] In another embodiment of the present application, it also includes:

[0161] Monitor the data type and data value of the received user access demand information to determine whether there is matching statistical dimension information;

[0162] If there is no matching statistical dimension information, the statistical dimension information is reconfigured by performing aggregation analysis based on the data type and data value required by the user.

[0163] A third aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a real-time ranking statistics program based on multi-dimensional big data. When the real-time ranking statistics program based on multi-dimensional big data is executed by a processor, the steps of the real-time ranking statistics method based on multi-dimensional big data are implemented. For details, see Figure 1 The description of the method steps will not be repeated here.

[0164] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0165] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0166] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0167] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0168] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0169] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A real-time ranking statistics method based on multi-dimensional big data, characterized in that: The steps include: Configure statistical dimension information and create multi-dimensional statistical forms based on the statistical dimension information; Collect user data in the platform, insert the user data into a multi-dimensional statistical form according to the data type, and save it in the first database; Synchronize user data to the second database, configure the aggregation statistics strategy of the second database, aggregate and count the user data to generate ranking data, and regularly send and update the ranking data to obtain cached data; Receive user access demand information, retrieve the latest cache data according to the user access demand information, and send it to the terminal to display it in the form of a ranking list; Synchronizing user data to the second database is specifically as follows: Process the user data in the multi-dimensional statistical form according to the statistical dimension information; Monitor the data sample size in a single data type. When the data sample size exceeds the preset column capacity, aggregate statistics are performed on the data values ​​in the data type to generate a first aggregate statistical result and use it as the data value under the data type for data overwriting; The user data after column processing is saved in the second database in the form of column data; The specific configuration of the aggregation statistics strategy of the second database is: Aggregate and count the data values ​​under all data categories respectively based on the columns of the second database; For each type of statistical dimension information, one or several key data types are selected to introduce weight coefficients, and weighted calculation is performed on the data values ​​in the data type to generate a second aggregate statistical result.

2. The real-time ranking statistics method based on multi-dimensional big data according to claim 1 is characterized in that: The specific steps for establishing a multi-dimensional statistical form based on statistical dimension information are as follows: Acquire the data types generated by the platform, including time information, region information, frequency information, review information, rating information, and amount information; Aggregate and analyze one or more information indicators in the data type to generate single-dimensional aggregate information, where the dimensional aggregation includes time aggregation, user aggregation, and organization aggregation; Use the data type of dimension aggregation information under different dimensions as a template to create dimension statistics forms under each dimension.

3. The real-time ranking statistics method based on multi-dimensional big data according to claim 1 is characterized in that: The user data collected on the platform are as follows: Receive user behavior signals and collect single user data under all user behavior signals in the platform; Monitor whether the deviation of data values ​​in the data types associated with the same user behavior signal exceeds a preset value; If the value exceeds the preset value, the data value with a deviation exceeding the preset value is weightedly corrected according to the data sample size.

4. The real-time ranking statistics method based on multi-dimensional big data according to claim 1 is characterized in that: The specific data for generating the ranking list by aggregating and counting user data is as follows: Set the aggregated statistical results of each user to a value based on the application scenario. The value range is the maximum value, minimum value, sum value, average value or sample value. According to the application scenario settings, the aggregated statistical values ​​of each user are sorted in a preset manner to generate ranking data.

5. The real-time ranking statistics method based on multi-dimensional big data according to claim 1 is characterized in that: Regularly send and update the ranking data to obtain cached data: Send the generated ranking data to the server to generate historical cache data; The update frequency of the cache data is configured in the manner of interval time setting, and the latest ranking data is regularly obtained from the second database according to the update frequency to generate the latest cache data; Historical cache data is periodically marked as invalid based on the update frequency.

6. The real-time ranking statistics method based on multi-dimensional big data according to claim 1 is characterized in that: Also includes: The cached data is labeled according to the generation time, statistical dimension information and number of users; The marked cache data is sent back to the first database regularly, and a link mode between the tag and the data is established.

7. A real-time ranking statistics system based on multi-dimensional big data, characterized in that: It includes a memory and a processor, wherein the memory includes a real-time ranking statistics program based on multi-dimensional big data, and when the real-time ranking statistics program based on multi-dimensional big data is executed by the processor, the steps of the real-time ranking statistics method based on multi-dimensional big data as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a real-time ranking statistics program based on multi-dimensional big data. When the real-time ranking statistics program based on multi-dimensional big data is executed by a processor, the steps of the real-time ranking statistics method based on multi-dimensional big data as described in any one of claims 1 to 6 are implemented.

Images