Method and device for processing multi-dimensional statistical data, and computer device

Abstract

The present disclosure relates to the technical field of data processing, and specifically discloses a multi-dimensional statistical data processing method and device and computer equipment. The method comprises the following steps: acquiring transaction data of a business system in real time, and storing the transaction data in a message queue; performing initial processing on the transaction data in the message queue to obtain transaction detail data and initial processing data; acquiring metadata configuration information of a multi-dimensional model, and acquiring a current result of multi-dimensional statistical data to be processed according to the metadata configuration information; and calculating the multi-dimensional statistical data according to the transaction detail data, the initial processing data and the current result to obtain an update result of the multi-dimensional statistical data. The present disclosure adopts a set of multi-dimensional models to uniformly plan data models, realizes real-time collection and processing of transaction data, and improves data processing timeliness.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus, and computer equipment for processing multidimensional statistical data. Background Technology

[0002] With the rapid development of digitalization, higher demands are being placed on data statistics and query applications that handle high-concurrency and high-time-sensitivity data. For banks and financial institutions at this stage, outdated reports cannot support efficient and transparent management, and simple real-time transaction data can no longer meet business analysis needs. What is needed now is to use data analysis and statistical methods to combine historical and real-time data for integrated analysis to gain a more comprehensive and rapid understanding of the business.

[0003] In related technologies, the processing of real-time transaction data relies on the timeliness of data warehouses and data exchange platforms. Since the timeliness of data warehouses is T+1, data extraction and distribution are completed through data exchange platforms, and data processing is completed through local batch scheduling. The processing time of multidimensional statistical data is mostly noon on T+1, and in extreme cases it may be T+2. This cannot meet the timeliness requirements of current business departments for real-time and near-real-time data analysis and statistics of such multidimensional data. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, storage medium, and computer program product for processing multidimensional statistical data to address the aforementioned technical problems.

[0005] Firstly, this disclosure provides a method for processing multidimensional statistical data. The method includes:

[0006] Real-time acquisition of transaction data from the business system, and storage of the transaction data in a message queue;

[0007] The transaction data in the message queue is initially processed to obtain transaction detail data and initial processed data.

[0008] Obtain the metadata configuration information of the multidimensional model, and obtain the current result of the multidimensional statistical data to be processed based on the metadata configuration information;

[0009] The multidimensional statistical data is calculated based on the transaction details, the initial processing data, and the current result to obtain the updated result of the multidimensional statistical data.

[0010] In one embodiment, the multidimensional statistical data is stored in a relational database, and the method further includes:

[0011] The transaction details and the initial processing data are stored in a cache database, which is also used to store the metadata configuration information of the multidimensional model.

[0012] In one embodiment, obtaining the metadata configuration information of the multidimensional model and obtaining the current result of the multidimensional statistical data to be processed based on the metadata configuration information includes:

[0013] Read the metadata configuration information of the multidimensional model from the cache database;

[0014] Based on the metadata configuration information, the current result corresponding to the multidimensional statistical data in the multidimensional data table is obtained from the relational database, and the relational database is also used to store the multidimensional data table.

[0015] In one embodiment, the method further includes:

[0016] The multidimensional data table is updated based on the update results of the multidimensional statistical data.

[0017] In one embodiment, the construction steps of the multidimensional model include:

[0018] Configure the constituent elements of the multidimensional model and generate the metadata configuration information; the constituent elements include fact tables, dimension tables, indexes, dimension fields, and measure fields;

[0019] The multidimensional model is created based on the constituent elements, and a multidimensional data table of the multidimensional model is generated.

[0020] The metadata configuration information is loaded into the cache database in real time, and the multidimensional model and the multidimensional data table are loaded into the relational database.

[0021] In one embodiment, the construction step of the multidimensional model further includes:

[0022] Calculate all dimension combinations and save the slice data corresponding to the dimension combinations to the relational database.

[0023] In one embodiment, the method includes:

[0024] Receive a query request from a customer system, and determine query parameters based on the query request. The query parameters include one or more of the following: the table name, field information, and condition information of the multidimensional data table.

[0025] The query parameters and metadata configuration information are combined to form a structured query language;

[0026] The target multidimensional data table is obtained by calling the query service based on the metadata configuration information and structured query language.

[0027] Return the target multidimensional data table to the client system.

[0028] Secondly, this disclosure also provides an apparatus for processing multidimensional statistical data. The apparatus includes:

[0029] The message queue module is used to acquire transaction data from the business system in real time and store the transaction data into the message queue.

[0030] The data processing module is used to perform initial processing on the transaction data in the message queue to obtain transaction detail data and initial processed data;

[0031] The multidimensional model module is used to obtain the metadata configuration information of the multidimensional model and obtain the current result of the multidimensional statistical data to be processed based on the metadata configuration information.

[0032] The data calculation module is used to calculate the multidimensional statistical data based on the transaction details data, the initial processing data, and the current result, and to obtain the updated result of the multidimensional statistical data.

[0033] In one embodiment, the multidimensional statistical data is stored in a relational database, and the apparatus includes:

[0034] The cache database module is used to store the transaction details data and the initial processing data into the cache database, and the cache database is also used to store the metadata configuration information of the multidimensional model.

[0035] In one embodiment, the multidimensional model module includes:

[0036] Metadata configuration information unit, used to read the metadata configuration information of the multidimensional model from the cache database;

[0037] A multidimensional data table unit is used to obtain the current result corresponding to the multidimensional statistical data in the multidimensional data table in the relational database according to the metadata configuration information. The relational database is also used to store the multidimensional data table.

[0038] In one embodiment, the device includes:

[0039] The data update module is used to update the multidimensional data table based on the update results of the multidimensional statistical data.

[0040] In one embodiment, the apparatus includes a multidimensional model building module, the multidimensional model building module comprising:

[0041] A configuration unit is used to configure the constituent elements of the multidimensional model and generate the metadata configuration information; the constituent elements include a fact table, a dimension table, an index, dimension fields, and a measure field;

[0042] Metadata unit, used to create the multidimensional model based on the constituent elements, and generate a multidimensional data table of the multidimensional model;

[0043] The loading unit is used to load the metadata configuration information into the cache database in real time, and to load the multidimensional model and the multidimensional data table into the relational database.

[0044] In one embodiment, the multidimensional model building module further includes:

[0045] A slice data unit is used to calculate all dimension combinations and save the slice data corresponding to the dimension combinations to the relational database.

[0046] In one embodiment, the device further includes:

[0047] The query request module is used to receive query requests from the client system and determine query parameters based on the query requests. The query parameters include one or more of the following: the table name, field information, and condition information of the multidimensional data table.

[0048] The query statement module is used to construct a structured query language based on the query parameters and the metadata configuration information;

[0049] The query invocation module is used to obtain the target multidimensional data table by invoking the query service based on the metadata configuration information and the structured query language;

[0050] The result return module is used to return the target multidimensional data table to the client system.

[0051] Thirdly, this disclosure also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described method for processing multidimensional statistical data.

[0052] Fourthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method for processing multidimensional statistical data.

[0053] Fifthly, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method for processing multidimensional statistical data.

[0054] The methods, apparatus, computer equipment, storage media, and computer program products for processing the aforementioned multidimensional statistical data have at least the following beneficial effects:

[0055] This disclosure avoids the drawbacks of existing multidimensional models that rely on building two separate models to calculate real-time and historical data, leading to redundant development and operational difficulties. Instead, it adopts a single multidimensional model for unified data model planning, enabling real-time collection and processing of transaction data. This improves data processing efficiency, reduces development costs, and enhances the stability of the multidimensional model. Furthermore, by using a stream-batch integrated approach to update multidimensional data and statistics in real time, it solves the problems of long multidimensional data model construction times and the difficulty of integrating real-time and historical data. This allows for rapid deployment, activation, and application of the multidimensional data model, resulting in high business responsiveness. Data read / write separation reduces the impact of batch processing on normal business functions, enabling rational allocation and use of system resources and supporting various analytical scenarios. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments or conventional technologies of this disclosure, the accompanying drawings used in the description of the embodiments or conventional technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is an application environment diagram of a method for processing multidimensional statistical data in one embodiment;

[0058] Figure 2 This is a flowchart illustrating a method for processing multidimensional statistical data in one embodiment;

[0059] Figure 3 This is a flowchart illustrating a method for processing multidimensional statistical data in another embodiment;

[0060] Figure 4 This is a flowchart illustrating a method for processing multidimensional statistical data in yet another embodiment;

[0061] Figure 5 This is a flowchart illustrating a method for processing multidimensional statistical data in yet another embodiment;

[0062] Figure 6 This is a flowchart illustrating a method for processing multidimensional statistical data in yet another embodiment;

[0063] Figure 7 This is a structural block diagram of a multidimensional statistical data processing device in one embodiment;

[0064] Figure 8 This is a structural block diagram of a multidimensional statistical data processing device in another embodiment;

[0065] Figure 9 This is a structural block diagram of a multidimensional statistical data processing device in another embodiment;

[0066] Figure 10 This is a structural block diagram of a multidimensional statistical data processing device in another embodiment;

[0067] Figure 11 This is a structural block diagram of a multidimensional statistical data processing device in another embodiment;

[0068] Figure 12 This is a block diagram of the internal structure of a computer device in one embodiment. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure.

[0071] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any specific order.

[0072] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having,” etc., specify the presence of the stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. Meanwhile, in this specification, the term “and / or” includes any and all combinations of the associated listed items.

[0073] The method for processing multidimensional statistical data provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the multidimensional statistical data processing system 110 communicates with the business system 120 and the customer query system 130 via a network. The multidimensional statistical data processing system 110 and the business system 120 can be deployed on a server, while the customer query system 130 can be deployed on a terminal. The multidimensional statistical data processing system 110 includes a data storage system 1102, which stores the data that the multidimensional statistical data processing system 110 needs to process. The data storage system 1102 includes a cache database and a relational database. The data storage system 1102 can be integrated on a server or placed in the cloud or on another network server. The multidimensional statistical data processing system 110 can acquire transaction data from the business system 120 in real time, process and manage the transaction data in multiple dimensions, and update the multidimensional statistical data in a timely manner. Simultaneously, the multidimensional statistical data processing system 110 can also receive query requests sent by the customer query system 130 and return the corresponding target multidimensional data table to the customer query system 130 based on the query request. The aforementioned terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The aforementioned servers can be implemented using independent servers or server clusters composed of multiple servers.

[0074] Furthermore, the multidimensional statistical data processing system 110 includes: a message queue 1104 for receiving transaction data from the storage business system and performing enqueue and dequeue operations; a data processing module 1106 for initially processing the transaction data in the message queue to obtain transaction detail data and initially processed data, and storing the transaction detail data and the initially processed data in the cache database; the data processing module 1106 includes an access interface for connecting to the message queue, a computing engine, and a storage interface for connecting to the cache database; a multidimensional model management module 1108 for constructing and managing the multidimensional model; and a general query service module 1110 connected to the multidimensional model management module for calling the query interface to perform queries according to query requests from the client system 130 and returning the query results to the client system 130. The cache database is also used to store metadata configuration information of the multidimensional model, and the relational database is used to store the multidimensional model and multidimensional statistical data.

[0075] The multidimensional model management module 1108 includes: a metadata management unit for constructing multidimensional models and defining metadata; a data table management unit for managing multidimensional data tables for multidimensional statistical data; an index management unit for managing indexes of the multidimensional data tables; an interface management unit for managing query interfaces for query statistics; a maintenance strategy management unit for managing maintenance strategies for multidimensional statistical data; and a permission management unit for managing permission opening for query statistics.

[0076] In some embodiments of this disclosure, such as Figure 2 As shown, a method for processing multidimensional statistical data is provided, which can be applied to... Figure 1 Taking a multidimensional statistical data processing system as an example, the following steps are included:

[0077] Step S10: Obtain transaction data from the business system in real time and store the transaction data in a message queue.

[0078] Specifically, transaction data can refer to transaction data from banking-related businesses within the financial sector, and this data is often generated in large volumes. Real-time acquired transaction data is stored in a message queue using streaming data processing methods. For example, the message queue could be a Kafka queue; Kafka is a high-throughput distributed publish-subscribe messaging system. Streaming data can refer to a sequence of sequential, large-volume, fast, and continuously arriving data sequences. Generally, streaming data can be viewed as a dynamic data collection that grows infinitely over time.

[0079] Step S20: Perform initial processing on the transaction data in the message queue to obtain transaction detail data and initial processing data.

[0080] Specifically, transaction data can be initially processed using a real-time computing engine to obtain transaction details and initial processed data. Transaction details can include information such as the transaction name, transaction price, transaction time, transaction object, and transaction subject. Initial processed data refers to coarsely processed transaction data, such as format standardization, simple summation, average, standard deviation, variance, and year-on-year / month-on-month change rate calculations. This method can be implemented based on the Flink framework, that is, driving the consumption of transaction data in the message queue through the Flink framework. Flink is a framework and distributed processing engine used for stateful computation on unbounded and bounded data streams. Flink can run in all common cluster environments and can perform computations at memory speed and arbitrary scale. The computing engine typically calculates metrics or dimension members based on data, arithmetic operators, constants, and functions.

[0081] Step S30: Obtain the metadata configuration information of the multidimensional model, and obtain the current result of the multidimensional statistical data to be processed based on the metadata configuration information.

[0082] Specifically, the metadata configuration information of the multidimensional model is obtained. A multidimensional model typically refers to a data organization model in a data warehouse, defined by dimensions and facts. Metadata usually describes the data itself, providing descriptive information about data and information resources, primarily describing data attributes. This information can support functions such as indicating storage location, historical data, resource lookup, and file records. Metadata configuration information can include metadata definitions, names, attributes, structure, and storage information. By obtaining the metadata configuration information, and combining it with the name, attributes, and storage information of the multidimensional statistical data to be processed, the current results of the multidimensional statistical data can be obtained.

[0083] Step S40: Calculate the multidimensional statistical data based on the transaction details data, the initial processing data, and the current result to obtain the updated result of the multidimensional statistical data.

[0084] Specifically, the current results of the acquired multidimensional statistical data are combined with the initial processed data and transaction details data to recalculate the multidimensional statistical data, and the calculated results are used as the updated results of the multidimensional statistical data.

[0085] The aforementioned method for processing multidimensional statistical data avoids the drawbacks of existing multidimensional models that require building two separate models to calculate real-time and historical data, leading to redundant development and operational difficulties. Instead, it adopts a single multidimensional model for unified data model planning, enabling real-time collection and processing of transaction data. This improves data processing efficiency, reduces development costs, and enhances the stability of the multidimensional model. Furthermore, by using a stream-batch integrated approach to update multidimensional data, real-time updates of multidimensional statistical data are achieved, resolving the issues of long multidimensional data model construction times and the difficulty of integrating real-time and historical data. This enables rapid deployment, activation, and application of the multidimensional data model, resulting in high business responsiveness. Data read / write separation reduces the impact of batch processing on normal business functions, allowing for rational allocation and use of system resources and supporting various analytical scenarios.

[0086] In some embodiments of this disclosure, the multidimensional statistical data is stored in a relational database, and the method further includes:

[0087] The transaction details and the initial processing data are stored in a cache database, which is also used to store the metadata configuration information of the multidimensional model.

[0088] Specifically, the multidimensional statistical data processing system uses two databases for data storage. The multidimensional statistical data is stored in a relational database, such as Oracle, which can be used to store both the multidimensional model and the multidimensional statistical data. After obtaining the transaction details and initial processing data, these are stored in a cache database. This cache database can be Redis (Remote Dictionary Server), used for lightweight processing data storage at the customer level, specifically for storing transaction details and initial processing data. The cache database also stores metadata configuration information for the multidimensional model.

[0089] This embodiment implements data storage using a relational database and a cache database. The metadata configuration information in the cache database can be used to quickly locate the multidimensional statistical data to be processed in the relational database, which improves the orderliness of data storage management and helps to quickly locate and update data.

[0090] In some embodiments of this disclosure, such as Figure 3 As shown, step S30 includes:

[0091] Step S32: Read the metadata configuration information of the multidimensional model from the cache database.

[0092] Step S34: Based on the metadata configuration information, obtain the current result corresponding to the multidimensional statistical data in the multidimensional data table in the relational database. The relational database is also used to store the multidimensional data table.

[0093] Specifically, the cache database stores metadata configuration information. The multidimensional statistical data processing system can quickly read the metadata configuration information of the multidimensional model from the cache database, locate the multidimensional data table in the relational database based on the metadata configuration information, and further obtain the current result corresponding to the multidimensional statistical data in the multidimensional data table. That is, the multidimensional statistical data is stored in the relational database in the form of a multidimensional data table. The multidimensional data table can be a data collection that includes various combinations of dimensions.

[0094] This embodiment stores multidimensional statistical data in a relational database in the form of multidimensional data tables, which strengthens the management and planning of multidimensional statistical data and improves storage and retrieval efficiency.

[0095] In some embodiments of this disclosure, the method further includes:

[0096] The multidimensional data table is updated based on the update results of the multidimensional statistical data.

[0097] Specifically, multidimensional data tables in relational databases can be updated in real time based on the update results of multidimensional statistical data.

[0098] This embodiment ensures the real-time performance of the multidimensional data table by updating the multidimensional statistical data in real time based on the update results of the multidimensional statistical data.

[0099] In some embodiments of this disclosure, such as Figure 4 As shown, the construction steps of the multidimensional model include:

[0100] Step A10: Configure the constituent elements of the multidimensional model and generate the metadata configuration information; the constituent elements include fact tables, multidimensional data tables, indexes, dimension fields, and measure fields.

[0101] Specifically, when building a multidimensional model, the first step is to configure its constituent elements. These elements can include fact tables, dimension tables, indexes, dimension fields, and measure fields. A fact table typically stores fact records, such as system logs or transaction records, and its records grow dynamically over time. A dimension table is a table that corresponds to the fact table; it stores the attribute values ​​of dimensions and can be associated with the fact table. Essentially, it extracts and standardizes frequently occurring attributes from the fact table for management in a single table. An index provides pointers to data values ​​stored in specified columns of a table, and these pointers are then sorted according to a specified sort order. The database uses the index to find a specific value and then follows the pointers to find the row containing that value. Dimension fields can refer to data categories and attribute fields, while measure fields can refer to numerical data fields.

[0102] Step A20: Create the multidimensional model based on the constituent elements, and generate a multidimensional data table for the multidimensional model.

[0103] Step A30: Load the metadata configuration information into the cache database in real time, and load the multidimensional model and the multidimensional data table into the relational database.

[0104] Specifically, a multidimensional model is created based on the configured components, and metadata and multidimensional data tables are generated. The metadata configuration information is loaded into the cache database in real time, and the multidimensional model, metadata, and multidimensional data tables are loaded into the relational database.

[0105] This embodiment can configure the constituent elements of multidimensional data as needed according to the configuration information, improving the flexibility of data management planning. At the same time, it can load the metadata configuration information into the cache database in real time and load the multidimensional model and multidimensional data table into the relational database, ensuring the timeliness of the database.

[0106] In some embodiments of this disclosure, the construction steps of the multidimensional model further include:

[0107] Calculate all dimension combinations and save the slice data corresponding to the dimension combinations to the relational database.

[0108] Specifically, based on the dimension table and fact table of the multidimensional model, all dimension combinations are calculated, and slice data is saved for each dimension combination. The mapping relationship between dimension combinations can be quickly established based on the slice data. Sliced ​​data refers to the data located and integrated after combining and segmenting the query dimensions. Sliced ​​data is used to distribute and store dimension combinations, reducing the burden on relational databases. Combined with sliced ​​data, when processing transaction data, such as... Figure 5As shown, the multidimensional statistical data processing system acquires transaction data from the business system in real time. The transaction data undergoes initial processing to obtain transaction detail data and initial processed data. The multidimensional data is then processed and updated in the multidimensional model management module. Multidimensional data is sliced ​​according to the sliced ​​data. For example, the number of multidimensional data slices can be N, where N is a positive integer.

[0109] This embodiment pre-calculates all combinations of dimensions when constructing a multidimensional model and saves slice data for each dimension combination, which facilitates the establishment of mapping relationships between dimension combinations. Even in multidimensional scenarios, dimension combinations can be stored in a distributed manner, reducing the burden on relational databases and improving the operational performance of relational databases.

[0110] In some embodiments of this disclosure, such as Figure 6 As shown, the method includes:

[0111] Step B10: Receive a query request from the customer system, and determine the query parameters based on the query request. The query parameters include one or more of the following: the table name, field information, and condition information of the multidimensional data table.

[0112] Specifically, the system receives query requests from client systems, which request queries for target multidimensional data tables. By parsing the query requests, the corresponding query parameters are obtained, and the target multidimensional data table can be uniquely identified based on these parameters.

[0113] Step B20: Concatenate the query parameters and the metadata configuration information into a structured query language.

[0114] Specifically, it combines metadata configuration information, such as concatenating query parameters and metadata definitions into a structured query language.

[0115] Step B30: Obtain the target multidimensional data table by calling the query service based on the metadata configuration information and structured query language.

[0116] Specifically, the query service is invoked. Based on metadata configuration information and a structured query language, the query service searches and matches multidimensional data tables to obtain the target multidimensional data table with the highest hit rate. Here, the query service can refer to a general functional module that directly interacts with the client's query needs.

[0117] Step B40: Return the target multidimensional data table to the client system.

[0118] Specifically, the query service can return the target multidimensional data table to the client system through the query interface.

[0119] This embodiment does not require concern about various dimension combinations. It can query and match multidimensional data tables by calling a general query service based on metadata configuration information and structured query language. This avoids the shortcomings of multidimensional data query and statistics functions being scattered across various business systems, which cannot provide a common query service for statistics. In addition, based on sliced ​​data, queries can be performed quickly based on dimensions and fact tables, while simplifying the interface design of the query service and reducing the consumption of system hardware interface resources, thus achieving efficient configuration and utilization of system hardware resources. The highly encapsulated real-time query interface based on sliced ​​data provides better system support for query and statistics scenarios involving multiple systems, multiple scenarios, and multiple tasks.

[0120] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0121] Based on the same inventive concept, this disclosure also provides a multidimensional statistical data processing apparatus for implementing the multidimensional statistical data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the multidimensional statistical data processing apparatus provided below can be found in the limitations of the multidimensional statistical data processing method described above, and will not be repeated here.

[0122] The apparatus may include a system (including a distributed system), software (application), module, component, server, client, etc., that uses the methods described in the embodiments of this specification, combined with necessary hardware implementation. Based on the same innovative concept, the apparatuses in one or more embodiments provided in this disclosure are as described in the following embodiments. Since the implementation schemes and methods for solving the problem by the apparatus are similar, the implementation of the specific apparatus in the embodiments of this specification can refer to the implementation of the foregoing methods, and repeated details will not be repeated. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatuses described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0123] In some embodiments of this disclosure, such as Figure 7 As shown, a multidimensional statistical data processing device is provided. This device can be the aforementioned terminal, a server, or a module, component, device, or unit integrated into the terminal. The device Z00 may include:

[0124] The message queue module Z10 is used to acquire transaction data from the business system in real time and store the transaction data into the message queue.

[0125] The data processing module Z20 is used to perform initial processing on the transaction data in the message queue to obtain transaction detail data and initial processed data;

[0126] The multidimensional model module Z30 is used to obtain the metadata configuration information of the multidimensional model and obtain the current result of the multidimensional statistical data to be processed based on the metadata configuration information.

[0127] The data calculation module Z40 is used to calculate the multidimensional statistical data based on the transaction details data, the initial processing data, and the current result, and to obtain the updated result of the multidimensional statistical data.

[0128] In some embodiments of this disclosure, such as Figure 8 As shown, the multidimensional statistical data is stored in a relational database, and the device Z00 includes:

[0129] The cache database module Z50 is used to store the transaction details data and the initial processing data into the cache database. The cache database is also used to store the metadata configuration information of the multidimensional model.

[0130] In some embodiments of this disclosure, such as Figure 9 As shown, the multidimensional model module Z30 includes:

[0131] Metadata configuration information unit Z32 is used to read the metadata configuration information of the multidimensional model from the cache database;

[0132] The multidimensional data table unit Z34 is used to obtain the current result corresponding to the multidimensional statistical data in the multidimensional data table in the relational database according to the metadata configuration information. The relational database is also used to store the multidimensional data table.

[0133] In some embodiments of this disclosure, the apparatus includes:

[0134] The data update module is used to update the multidimensional data table based on the update results of the multidimensional statistical data.

[0135] In some embodiments of this disclosure, such as Figure 10As shown, the device Z00 includes a multi-dimensional model building module Z60, which includes:

[0136] Configuration unit Z62 is used to configure the constituent elements of the multidimensional model and generate the metadata configuration information; the constituent elements include fact table, dimension table, index, dimension field, and measure field;

[0137] Metadata unit Z64 is used to create the multidimensional model based on the constituent elements and generate a multidimensional data table of the multidimensional model;

[0138] The loading unit Z66 is used to load the metadata configuration information into the cache database in real time, and to load the multidimensional model and the multidimensional data table into the relational database.

[0139] In some embodiments of this disclosure, the multidimensional model building module further includes:

[0140] A slice data unit is used to calculate all dimension combinations and save the slice data corresponding to the dimension combinations to the relational database.

[0141] In some embodiments of this disclosure, such as Figure 11 As shown, the device Z00 further includes:

[0142] The query request module Z70 is used to receive query requests from the customer system and determine query parameters based on the query requests. The query parameters include one or more of the following: the table name, field information, and condition information of the multidimensional data table.

[0143] The query statement module Z80 is used to construct a structured query language based on the query parameters and the metadata configuration information.

[0144] The query call module Z90 is used to obtain the target multidimensional data table by calling the query service based on the metadata configuration information and structured query language;

[0145] The result return module Z100 is used to return the target multidimensional data table to the client system.

[0146] Each module in the aforementioned multidimensional statistical data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; in actual implementation, other division methods may be used.

[0147] Based on the aforementioned embodiments of the multidimensional statistical data processing method, in another embodiment provided in this disclosure, a computer device is provided. This computer device may be a server, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for processing multidimensional statistical data.

[0148] Those skilled in the art will understand that the structure shown in the figure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0149] Based on the embodiments described above for processing multidimensional statistical data, in another embodiment provided in this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the above method embodiments.

[0150] Based on the embodiments described above for processing multidimensional statistical data, in another embodiment provided in this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0153] In the description of this specification, references to terms such as "some embodiments," "other embodiments," and "ideal embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative descriptions of the above terms do not necessarily refer to the same embodiments or examples.

[0154] It is understood that the various embodiments of the methods described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. Relevant details can be found in the descriptions of other method embodiments.

[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features of the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the scope of protection of this disclosure. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method of processing multidimensional statistical data, characterized by, include: Real-time acquisition of transaction data from the business system, and storage of the transaction data in a message queue; The transaction data in the message queue is initially processed to obtain transaction detail data and initial processed data. Read the metadata configuration information of the multidimensional model from the cache database; Based on the metadata configuration information, the current result corresponding to the multidimensional statistical data in the multidimensional data table is obtained in the relational database. The relational database is also used to store the multidimensional data table, and the multidimensional statistical data is stored in the relational database. The transaction details and the initial processing data are stored in a cache database, which is also used to store the metadata configuration information of the multidimensional model. Based on the dimension table and fact table of the multidimensional model, all dimension combinations are calculated, and slice data is saved for each dimension combination; the slice data is used to distribute the dimension combinations for storage, reducing the burden on the relational database. The multidimensional statistical data is calculated based on the transaction details, the initial processing data, and the current result to obtain the updated result of the multidimensional statistical data.

2. The method of claim 1, wherein, The method further includes: The multidimensional data table is updated based on the update results of the multidimensional statistical data.

3. The method of claim 1, wherein, The construction steps of the multidimensional model include: Configure the constituent elements of the multidimensional model and generate the metadata configuration information; the constituent elements include fact tables, dimension tables, indexes, dimension fields, and measure fields; The multidimensional model is created based on the constituent elements, and a multidimensional data table of the multidimensional model is generated. The metadata configuration information is loaded into the cache database in real time, and the multidimensional model and the multidimensional data table are loaded into the relational database.

4. The method of claim 3, wherein, The construction steps of the multidimensional model also include: Calculate all dimension combinations and save the slice data corresponding to the dimension combinations to the relational database.

5. The method according to claim 4, characterized in that, The method includes: Receive a query request from a customer system, and determine query parameters based on the query request. The query parameters include one or more of the following: the table name, field information, and condition information of the multidimensional data table. The query parameters and metadata configuration information are combined to form a structured query language; The target multidimensional data table is obtained by calling the query service based on the metadata configuration information and structured query language. Return the target multidimensional data table to the client system.

6. A device for processing multidimensional statistical data, characterized in that, The device includes: The message queue module is used to acquire transaction data from the business system in real time and store the transaction data into the message queue. The data processing module is used to perform initial processing on the transaction data in the message queue to obtain transaction detail data and initial processed data; The multidimensional model module is used to read the metadata configuration information of the multidimensional model from the cache database; Based on the metadata configuration information, the current result corresponding to the multidimensional statistical data in the multidimensional data table is obtained in the relational database. The relational database is also used to store the multidimensional data table, and the multidimensional statistical data is stored in the relational database. The transaction details and the initial processing data are stored in a cache database, which is also used to store the metadata configuration information of the multidimensional model. Based on the dimension table and fact table of the multidimensional model, all dimension combinations are calculated, and slice data is saved for each dimension combination; the slice data is used to distribute the dimension combinations for storage, reducing the burden on the relational database. The data calculation module is used to calculate the multidimensional statistical data based on the transaction details data, the initial processing data, and the current result, and to obtain the updated result of the multidimensional statistical data.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images