Real-time aggregation method and device for detailed data, electronic device and storage medium

Abstract

The application discloses a kind of real-time summary method, device, electronic equipment and storage medium of detailed data, it is related to data processing technical field.The method comprises: obtaining different categories of detailed data generated by real-time business system;Wherein, detailed data includes at least one of the following: business data, entity data, rule data and configuration data;In response to the request instruction of target summary index, determine the candidate detailed data matched with request instruction from detailed data;Candidate detailed data is processed based on target data processing model, and target summary result matched with target summary index is obtained;Candidate detailed data and target summary result are rendered and displayed on target display page.The scheme of the application can quickly and accurately determine the summary result of detailed data, and display in the same page, to avoid the inconsistency between detailed data and summary result displayed in the same page provides basis.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, and storage medium for real-time aggregation of detailed data. Background Technology

[0002] In the Enterprise Resource Planning (ERP) industry, a large amount of data is usually involved, such as sales data of various merchants or distributors, order data of different products, logistics data or log data, etc.

[0003] Currently, detailed data and summary results are calculated separately. In practice, detailed data is written directly to the database, while summary results are written to the database after consuming detailed data. This results in the detailed data and summary results being written to the database at different times, which can lead to inconsistencies between the detailed data and summary results displayed on the same page.

[0004] How to quickly and accurately determine the summary results of detailed data and display them on the same page, so as to provide a basis for avoiding inconsistencies between the detailed data and the summary results displayed on the same page, is a key research issue in the industry. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for real-time aggregation of detailed data, so as to quickly and accurately determine the aggregation results of detailed data and display them on the same page, providing a basis for avoiding inconsistencies between the detailed data displayed on the same page and the aggregation results.

[0006] According to one aspect of the present invention, a method for real-time aggregation of detailed data is provided, the method comprising:

[0007] Obtain detailed data of different categories generated in real time by the business system; wherein the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data;

[0008] In response to a request instruction for a target summary indicator, candidate detailed data matching the request instruction are determined from the detailed data;

[0009] The candidate detailed data is processed based on the target data processing model to obtain the target summary result that matches the target summary index;

[0010] The target display page renders and displays the candidate detailed data and the target summary results.

[0011] According to another aspect of the present invention, a real-time data aggregation apparatus is provided, the apparatus comprising:

[0012] The detailed data acquisition module is used to acquire different categories of detailed data generated in real time by the business system; wherein, the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data;

[0013] The candidate detailed data determination module is used to determine candidate detailed data that matches the request instruction from the detailed data in response to the request instruction of the target summary indicator;

[0014] The target summary result determination module processes the candidate detailed data based on the target data processing model to obtain the target summary result that matches the target summary indicator;

[0015] The rendering and display module is used to render and display the candidate detailed data and the target summary results on the target display page.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the real-time data aggregation method for detailed data as described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the real-time data aggregation method for detailed data as described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the real-time data aggregation method for detailed data as described in any embodiment of the present invention.

[0022] The technical solution of this invention involves acquiring detailed data of different categories generated in real time by a business system; wherein the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data; responding to a request instruction for a target summary indicator, determining candidate detailed data matching the request instruction from the detailed data; processing the candidate detailed data based on a target data processing model to obtain a target summary result matching the target summary indicator; and rendering and displaying the candidate detailed data and the target summary result on a target display page. This allows for quick and accurate determination of the summary result of the detailed data, which is displayed on the same page, providing a basis for avoiding inconsistencies between the detailed data and the summary result displayed on the same page.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a method for real-time aggregation of detailed data according to Embodiment 1 of the present invention;

[0026] Figure 2 This is a flowchart of a method for real-time aggregation of detailed data according to Embodiment 2 of the present invention;

[0027] Figure 3 This is a flowchart of a method for real-time aggregation of detailed data according to Embodiment 3 of the present invention;

[0028] Figure 4 This is a schematic diagram of real-time aggregation of detailed data provided in Embodiment 3 of the present invention;

[0029] Figure 5 This is a schematic diagram of the structure of a real-time data aggregation device according to Embodiment 4 of the present invention;

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the real-time data aggregation method of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1 This is a flowchart of a real-time data aggregation method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where detailed data generated by a business system is aggregated in real time. This method can be executed by a real-time data aggregation device, which can be implemented in hardware and / or software. This real-time data aggregation device can be configured in electronic devices such as computers, servers, cloud platforms, or tablet computers. Figure 1 As shown, the method includes:

[0035] Step 110: Obtain detailed data of different categories generated in real time by the business system.

[0036] The detailed data includes at least one of the following: business data, entity data, rule data, and configuration data.

[0037] The business system can be an ERP system, an e-commerce platform, a logistics management system, or a production management system, etc., and this embodiment does not limit it.

[0038] Optionally, in one implementation of this embodiment, business data (e.g., order data, sales data, or after-sales data), entity data (e.g., product name, product identifier, or product attributes), rule data (e.g., classification criteria, data source, or permission management), and configuration data (e.g., payment method configuration, logistics and delivery settings, promotional activity configuration, or inventory management) generated by the business system can be obtained in real time through a message queue (e.g., Flink).

[0039] Optionally, in this embodiment, obtaining detailed data of different categories generated in real time by the business system may include: responding in real time to the processing instructions of different businesses through a message queue, obtaining the business data generated during the processing of each business, and querying the entity data associated with each business data; or, querying the rule data and configuration data stored in the reference database through a message queue; and storing each business data, entity data, rule data, and configuration data in different detailed data tables of the target database based on the attribute information of each business data, entity data, rule data, and configuration data.

[0040] The different services that the message queue can respond to in real time can be order placement for the target product, profit analysis for the target product, order tracking for the target product, or after-sales service for the target product, etc., and this embodiment does not limit them.

[0041] In one optional implementation of this embodiment, the business data generated during the processing of each business can be obtained in real time through a message queue. For example, the transaction amount, order time, user identifier, or product link information of the order can be obtained from the order business. Furthermore, the entity data associated with each of the business data can be queried. For example, the product name, product identifier, or product attribute corresponding to each business data can be queried.

[0042] In another optional implementation of this embodiment, rule data and configuration data associated with each service can also be queried in real time through a message queue. The reference database can be a relational database or a distributed database, and the number can be one or multiple, which is not limited in this embodiment. In this embodiment, rule data and configuration data can be stored in different databases or in different areas of the same database. For example, rule data can be stored in the first database, configuration data of the first service can be stored in the first area of ​​the second database, configuration data of the second service can be stored in the second area of ​​the second database, etc., which is not limited in this embodiment.

[0043] In an optional implementation of this embodiment, after obtaining the detailed data involved in the above steps, the business data, entity data, rule data, and configuration data can be further stored in different detailed data tables of the target database based on the attribute information of the obtained business data, entity data, rule data, and configuration data.

[0044] The attribute information of each detailed data can be the category of each detailed data, the time of generation, the business identifier that generated the detailed data, etc.; the target database can also be a relational database or a distributed database, etc., and this embodiment does not limit it.

[0045] Optionally, in this embodiment, after obtaining each business data, each entity data, each rule data, and each configuration data, the attribute information of each data can be further determined. Furthermore, based on the attribute information of each detailed data, each detailed data can be stored in a detailed data table in the target database. For example, in this embodiment, business data generated at the same time can be stored in the same detailed data table; detailed data of the same merchant or the same product can also be stored in the same detailed data table; and detailed data with the same business identifier can also be stored in the same detailed data table. This embodiment does not limit these possibilities.

[0046] Step 120: In response to the request instruction of the target summary indicator, determine the candidate detailed data that matches the request instruction from the detailed data.

[0047] The target summary indicators can be: high-profit items (products), low-profit items, total profit, today's best-selling items, negative-profit items, remaining inventory, or negative profit, etc., but this embodiment does not limit them.

[0048] Optionally, in this embodiment, after receiving the request instruction for the target summary indicator, candidate detailed data matching the target summary indicator can be further determined from the detailed data obtained above.

[0049] In an optional implementation of this embodiment, in response to a request instruction for a target summary indicator, determining candidate detailed data matching the request instruction from the detailed data may include: determining a target data processing model matching the target summary indicator; obtaining independent variables in the target data processing model and determining each independent variable as reference detailed data associated with the target summary indicator; determining a reference detailed data table in the target database that stores the reference detailed data; and filtering each reference detailed data table based on preset time information to obtain each candidate detailed data.

[0050] The target data processing model can be a net profit analysis model, a negative profit analysis model, a best-selling product determination model, or a purchase volume determination model, etc., and this embodiment does not limit it. In this embodiment, the target data processing model can include multiple independent variables. For example, if the target data processing model is a net profit analysis model, then the related independent variables can be: product purchase price, product selling price, logistics cost, labor cost, warehouse cost, advertising fee, platform commission, or tax, etc. It can be understood that the individual variables involved in the above-mentioned net profit analysis model are the detailed data content that matches the summary indicator "net profit", such as: product purchase price, product selling price, logistics cost, labor cost, warehouse cost, advertising fee, platform commission, or tax, etc., and their related independent variables.

[0051] Optionally, in this embodiment, after receiving the request instruction for the target summary indicator, a target data processing model matching the target summary indicator can be further determined. For example, in this embodiment, the target data processing model can be determined based on the identification information of the target summary indicator, and the identification information of the target summary indicator matches the identification information of the target data processing model. Further, the independent variables of the target data processing model can be obtained, and each independent variable can be determined as reference detailed data associated with the target summary indicator. For example, the reference detailed data in the above example can be the purchase price, selling price, logistics cost, labor cost, warehouse cost, advertising fee, platform commission, and tax of the target product. Further, the target database can be defined as storing detailed data tables for the purchase price, selling price, logistics cost, labor cost, warehouse cost, advertising fee, platform commission, and tax of the target product; in this embodiment, these are referred to as reference detailed data tables.

[0052] In this embodiment, the preset time information can be a period of time such as the past day, the past three days, the past week, or the past month, and this embodiment does not limit it.

[0053] Optionally, in this embodiment, after obtaining the reference detail data table storing each reference detail data, the reference detail data table can be filtered based on preset time information. For example, the first reference detail data that is not within the preset time information can be filtered out to obtain candidate detail data.

[0054] Step 130: Process the candidate detailed data based on the target data processing model to obtain the target summary result that matches the target summary index.

[0055] Optionally, after determining the candidate detailed data that matches the request instruction for the target summary indicator, the candidate detailed data can be further processed based on the target data processing model determined above, so as to obtain the target summary result that matches the target summary indicator.

[0056] In an optional implementation of this embodiment, after determining the candidate detailed data that matches the request instruction of the target summary indicator, the candidate detailed data can be directly input into the target data processing model, so that the target data processing model can perform iterative calculation on each candidate detailed data to obtain the target summary result that matches the target summary indicator.

[0057] Optionally, in this embodiment, processing the candidate detailed data based on the target data processing model to obtain a target summary result matching the target summary indicator may include: inputting each candidate detailed data into the target data processing model, performing iterative calculations on the candidate detailed data based on the target data processing model, and obtaining the target summary result if the iterative calculation is determined to be successful.

[0058] In an optional implementation of this embodiment, after determining the candidate detailed data that matches the request instruction for the target summary indicator, the candidate detailed data can be directly input into the target data processing model, thereby performing iterative calculations on each candidate detailed data through the target data processing model. Furthermore, it can be determined whether any abnormal or interrupted events occur during the iterative calculation process. If no abnormal or interrupted events occur, it can be determined that the summary process of the candidate detailed data has run successfully, and the target summary result can be obtained at this time. If it is determined that an abnormal or interrupted event occurs during the iterative calculation process, it can be determined that the summary process of the candidate detailed data has failed, and the target summary result cannot be obtained at this time.

[0059] Step 140: Render and display the candidate detailed data and the target summary results on the target display page.

[0060] The target display page can be any page in the business system, and this embodiment does not limit it.

[0061] Optionally, in this embodiment, after obtaining the target summary result that matches the target summary indicator, the candidate detailed data and the target summary result can be displayed simultaneously on the target display page; for example, the candidate detailed data and the target summary result can be displayed in different data tables.

[0062] In an optional implementation of this embodiment, rendering and displaying the candidate detailed data and the target summary result on the target display page may include: determining a target display format that matches the target summary result, and displaying the candidate detailed data and the target summary result on the target display page based on the target display format; wherein, the target display format may include: a data table, a graph, a dashboard, or a report.

[0063] Optionally, in this embodiment, after obtaining the target summary result matching the target summary indicator, the display format matching the target summary result can be further determined. For example, the target display format can be determined based on the data volume of the target summary result. For example, if the data volume of the target summary result is large, the target display format can be determined to be a data table; if the data volume of the target summary result has many types, the target display format can be determined to be a dashboard. Furthermore, based on the target display format, the candidate detailed data and the target summary result can be displayed separately in the target display interface.

[0064] In this embodiment, after obtaining the detailed data, the system does not directly perform summary calculations on the obtained detailed data. Instead, after receiving a summary instruction, it filters the obtained detailed data to obtain candidate detailed data that matches the summary instruction. The candidate detailed data can then be summarized to obtain a summary result that matches the target summary indicator. This ensures that the candidate detailed data and the summary result correspond, further guaranteeing that the candidate detailed data displayed on the display page is consistent with the target summary result. In other words, there will be no situation where the displayed detailed data is inconsistent with the summary result, meaning that the displayed candidate detailed data cannot be used to calculate the target summary result.

[0065] The technical solution of this embodiment acquires detailed data of different categories generated in real time by the business system; wherein the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data; in response to a request instruction for a target summary indicator, candidate detailed data matching the request instruction is determined from the detailed data; the candidate detailed data is processed based on a target data processing model to obtain a target summary result matching the target summary indicator; the candidate detailed data and the target summary result are rendered and displayed on a target display page, which can quickly and accurately determine the summary result of the detailed data and display it on the same page, providing a basis for avoiding inconsistencies between the detailed data and the summary result displayed on the same page.

[0066] Example 2

[0067] Figure 2This is a flowchart of a real-time data aggregation method according to Embodiment 2 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 2 As shown, the method includes:

[0068] Step 210: Obtain detailed data of different categories generated in real time by the business system.

[0069] Step 220: In response to the request instruction of the target summary indicator, determine the candidate detailed data that matches the request instruction from the detailed data.

[0070] Step 230: Process the candidate detailed data based on the target data processing model to obtain the target summary result that matches the target summary index.

[0071] Step 240: Render and display the candidate detailed data and the target summary results on the target display page.

[0072] Step 250: In response to the update instruction for detailed data, determine the update candidate detailed data that matches the request instruction; input the update candidate detailed data into the target data processing model for update iteration calculation; if the update iteration calculation is successful, obtain the target summary update result; update the display content of the target display page based on the target summary update result and the update candidate detailed data.

[0073] Understandably, since the business system processes related business in real time, the detailed data produced by the business system will also be frequently updated. Accordingly, after the target summary results and candidate detailed data are displayed on the target display page, if it is determined that the detailed data has been updated, then the updated candidate detailed data that matches the request instruction of the target summary indicator can be further determined. For example, each updated detailed data can be filtered to determine the updated candidate detailed data.

[0074] Furthermore, the determined candidate update details can be input into the target data processing model that matches the target summary index for iterative calculation. Further, if the update iteration calculation is successful, the target summary update result matching the target summary index can be obtained. For example, it can be determined whether any abnormal or interrupted events occur during the update iteration calculation. If no abnormal or interrupted events occur, the summary process of updating the candidate details can be determined to have succeeded, and the target summary update result can be obtained. If an abnormal or interrupted event occurs during the update iteration calculation, the summary process of updating the candidate details can be determined to have failed, and the target summary update result cannot be obtained.

[0075] Furthermore, the content displayed on the target page can be updated (replaced) based on the target summary update results and the updated candidate detailed data.

[0076] In this embodiment, upon receiving an update instruction for detailed data, candidate detailed data matching the request instruction can be identified; the candidate detailed data is input into the target data processing model for update iteration calculation; if the update iteration calculation is successful, the target summary update result is obtained; and the content displayed on the target display page is updated based on the target summary update result and the candidate detailed data. This allows for real-time updates of the summary results of the target summary indicators and ensures that the displayed candidate detailed data matches the target summary result.

[0077] Example 3

[0078] Figure 3 This is a flowchart of a real-time data aggregation method according to Embodiment 3 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 3 As shown, the method includes:

[0079] Step 310: Obtain detailed data of different categories generated in real time by the business system.

[0080] Step 320: In response to the request instruction of the target summary indicator, determine the candidate detailed data that matches the request instruction from the detailed data.

[0081] Step 330: Process the candidate detailed data based on the target data processing model.

[0082] Step 340: If the iterative calculation fails, analyze the candidate detailed data to identify abnormal candidate detailed data, analyze the abnormal candidate detailed data, and repair the abnormal candidate detailed data based on the analysis results to obtain the first candidate detailed data; re-perform iterative calculation on the first candidate detailed data based on the target data processing model until the iterative calculation is determined to be successful; or, if the update iterative calculation fails, analyze the update candidate detailed data to identify abnormal update candidate detailed data, analyze the abnormal update candidate detailed data, and repair the abnormal update candidate detailed data based on the analysis results to obtain the second candidate detailed data; re-perform update iterative calculation on the second candidate detailed data based on the target data processing model until the update iterative calculation is determined to be successful.

[0083] Optionally, in this embodiment, during the process of inputting each candidate detailed data into the target data processing model and iteratively calculating the candidate detailed data based on the target data processing model, if it is determined that the iterative calculation has failed, the candidate detailed data can be further analyzed to identify abnormal candidate detailed data. Further analysis of the abnormal candidate detailed data is possible, and the abnormal candidate detailed data can be repaired based on the analysis results to obtain the first candidate detailed data. Furthermore, iterative calculation can be re-performed based on the target data processing model on the first candidate detailed data until iterative calculation is determined to be successful.

[0084] In another optional embodiment of this example, during the process of inputting each of the updated candidate detailed data into the target data processing model and iteratively calculating the updated candidate detailed data based on the target data processing model, if it is determined that the update iteration calculation has failed, the updated candidate detailed data can be further analyzed to identify abnormal updated candidate detailed data. Furthermore, the abnormal updated candidate detailed data can be analyzed and repaired based on the analysis results to obtain second candidate detailed data. Furthermore, the update iteration calculation can be re-performed based on the target data processing model for the second candidate detailed data until it is determined that the update iteration calculation has succeeded.

[0085] It should be noted that in this embodiment, when abnormal data occurs (which can be abnormal detailed data or abnormal summary result data, and this embodiment does not limit it), the data will first be repaired by extraction, transformation or loading. If the above methods cannot be successfully repaired, the abnormal data can be temporarily filtered, and then the abnormal data can be analyzed and repaired. After the abnormal data is repaired, it will be recalculated. This can ensure the integrity of the statistical data, that is, ensure the consistency between the page detailed data and the summary data.

[0086] In this embodiment, if the iterative calculation fails, the candidate detailed data is analyzed to identify abnormal candidate detailed data. This abnormal candidate detailed data is then analyzed and repaired based on the analysis results to obtain first candidate detailed data. The first candidate detailed data is then iteratively calculated again based on the target data processing model until the iterative calculation is successful. Alternatively, if the update iterative calculation fails, the update candidate detailed data is analyzed to identify abnormal update candidate detailed data. This abnormal update candidate detailed data is then analyzed and repaired based on the analysis results to obtain second candidate detailed data. The second candidate detailed data is then iteratively updated again based on the target data processing model until the update iterative calculation is successful. This approach allows for real-time analysis and processing of abnormal data, providing a basis for ensuring correct summary results.

[0087] To better understand the detailed data aggregation method involved in the embodiments of the present invention, Figure 4 This is a schematic diagram illustrating real-time aggregation of detailed data according to Embodiment 3 of the present invention; as shown below. Figure 4 As shown, it mainly includes three steps: data preparation, data computation, and service capabilities.

[0088] In the specific implementation, during the data preparation step, Flink can provide real-time detailed data, which may include business tables such as orders and after-sales service, product entity tables, rule tables that affect the definition of scope, and merchant configuration data.

[0089] Furthermore, during the data calculation process, real-time detailed data and summary data can be processed in a unified manner, which is the prerequisite for achieving consistency between detailed and summary data; the service capability can provide services for detailed data and summary results simultaneously.

[0090] Example 4

[0091] Figure 5 This is a schematic diagram of a real-time data aggregation device according to Embodiment 3 of the present invention. Figure 5 As shown, the device includes: a detailed data acquisition module 510, a candidate detailed data determination module 520, a target summary result determination module 530, and a rendering and display module 540.

[0092] The detailed data acquisition module 510 is used to acquire different categories of detailed data generated in real time by the business system; wherein the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data;

[0093] The candidate detailed data determination module 520 is used to determine candidate detailed data that matches the request instruction from the detailed data in response to a request instruction from the target summary indicator.

[0094] The target summary result determination module 530 processes the candidate detailed data based on the target data processing model to obtain the target summary result that matches the target summary indicator;

[0095] The rendering and display module 540 is used to render and display the candidate detailed data and the target summary results on the target display page.

[0096] In this embodiment, a detailed data acquisition module acquires different categories of detailed data generated in real time by the business system. The detailed data includes at least one of the following: business data, entity data, rule data, and configuration data. A candidate detailed data determination module responds to a request instruction from a target summary indicator and determines candidate detailed data matching the request instruction from the detailed data. A target summary result determination module processes the candidate detailed data based on a target data processing model to obtain a target summary result matching the target summary indicator. A rendering and display module renders and displays the candidate detailed data and the target summary result on a target display page. This allows for quick and accurate determination of the summary result of the detailed data, which is then displayed on the same page, providing a basis for avoiding inconsistencies between the detailed data and the summary result displayed on the same page.

[0097] In an optional implementation of this embodiment, the detailed data acquisition module 510 is specifically used to respond in real time to the processing instructions of different services through a message queue, acquire the business data generated during the processing of each business, and query the entity data associated with each business data.

[0098] Alternatively, you can query the rule data and configuration data stored in the reference database through the message queue;

[0099] Based on the attribute information of each of the aforementioned business data, entity data, rule data, and configuration data, the aforementioned business data, entity data, rule data, and configuration data are stored in different detailed data tables in the target database.

[0100] In an optional implementation of this embodiment, the candidate detailed data determination module 520 is specifically used to determine the target data processing model that matches the target summary index;

[0101] Obtain the independent variables in the target data processing model, and determine each independent variable as reference detailed data associated with the target summary indicator;

[0102] Determine the reference detail data table in the target database that stores the reference detail data;

[0103] Based on preset time information, each of the reference detail data tables is filtered to obtain each of the candidate detail data.

[0104] In an optional implementation of this embodiment, the target summary result determination module 530 is specifically used to input each of the candidate detailed data into the target data processing model, and perform iterative calculations on the candidate detailed data based on the target data processing model;

[0105] If the iterative calculation is confirmed to have run successfully, the target summary result is obtained.

[0106] In an optional implementation of this embodiment, the rendering and display module 540 is specifically used to determine a target display format that matches the target summary result, and to display the candidate detailed data and the target summary result respectively on the target display page based on the target display format;

[0107] The target display formats include: data tables, graphs, dashboards, or reports.

[0108] In an optional implementation of this embodiment, the real-time aggregation device for detailed data further includes: an update module, configured to determine candidate detailed data for update that matches the request instruction in response to an update instruction for detailed data;

[0109] The updated candidate detailed data is input into the target data processing model for update and iterative calculation.

[0110] If the update iteration calculation is confirmed to be successful, the target summary update result is obtained;

[0111] The content displayed on the target display page is updated based on the target summary update results and the update candidate detail data.

[0112] In an optional implementation of this embodiment, the real-time aggregation device for detailed data further includes: an anomaly analysis module, used to analyze the candidate detailed data, determine abnormal candidate detailed data in the candidate detailed data, analyze the abnormal candidate detailed data, and repair the abnormal candidate detailed data according to the analysis results to obtain the first candidate detailed data when it is determined that the iterative calculation has failed;

[0113] Re-iterate the calculation of the first candidate detailed data based on the target data processing model until the iterative calculation is confirmed to be successful.

[0114] or,

[0115] If the update iteration calculation fails, the update candidate detailed data is analyzed to identify abnormal update candidate detailed data. The abnormal update candidate detailed data is then analyzed and repaired based on the analysis results to obtain the second candidate detailed data.

[0116] The second candidate detailed data is recalculated based on the target data processing model until the update and iteration calculation is confirmed to be successful.

[0117] The real-time data aggregation device provided in the embodiments of the present invention can execute the real-time data aggregation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0118] In the technical solutions of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of detailed data generated by the business systems involved all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] Example 5

[0120] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0121] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0122] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0123] Processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a real-time aggregation method for detailed data, which includes: acquiring detailed data of different categories generated in real time by a business system; wherein the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data; in response to a request instruction for a target aggregation metric, determining candidate detailed data from the detailed data that matches the request instruction; processing the candidate detailed data based on a target data processing model to obtain a target aggregation result that matches the target aggregation metric; and rendering and displaying the candidate detailed data and the target aggregation result on a target display page.

[0124] In some embodiments, the real-time data aggregation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the real-time data aggregation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the real-time data aggregation method by any other suitable means (e.g., by means of firmware).

[0125] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0127] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0128] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0129] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0130] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0131] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0132] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0133] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements a database detection method as provided in any embodiment of this application.

[0134] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0136] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for real-time aggregation of detailed data, characterized in that, The method includes: Obtain detailed data of different categories generated in real time by the business system; wherein the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data; In response to a request instruction for a target summary indicator, a target data processing model matching the target summary indicator is determined; Obtain the independent variables in the target data processing model, and determine each independent variable as reference detailed data associated with the target summary indicator; Determine the reference detail data table in the target database that stores the reference detail data; Based on preset time information, each of the reference detailed data tables is filtered to obtain each candidate detailed data. Each candidate detailed data is then input into the target data processing model, and the candidate detailed data is iteratively calculated based on the target data processing model. If the iterative calculation is successful, a target summary result matching the target summary index is obtained; If the iterative calculation fails, the abnormal data in the candidate detailed data is repaired by extraction, transformation or loading. If the repair fails, the abnormal data is temporarily filtered and the repair task is recorded. The iterative calculation is then performed again based on the repaired candidate detailed data until the target summary result is obtained. The candidate detailed data and the target summary results are rendered and displayed on the target display page; The method further includes: in response to an update instruction for detail data, determining candidate update detail data that matches the request instruction; The updated candidate detailed data is input into the target data processing model for update and iterative calculation. If the update iteration calculation is confirmed to be successful, the target summary update result is obtained; The content displayed on the target display page is updated based on the target summary update results and the update candidate detail data.

2. The method for real-time aggregation of detailed data according to claim 1, characterized in that, The acquisition of detailed data of different categories generated in real time by the business system includes: The system responds in real time to processing instructions from different services via message queues, retrieves the business data generated during the processing of each business, and queries the entity data associated with each business data. Alternatively, you can query the rule data and configuration data stored in the reference database through the message queue; Based on the attribute information of each of the aforementioned business data, entity data, rule data, and configuration data, the aforementioned business data, entity data, rule data, and configuration data are stored in different detailed data tables in the target database.

3. The method for real-time aggregation of detailed data according to claim 1, characterized in that, The rendering and displaying of the candidate detailed data and the target summary results on the target display page includes: Determine the target display format that matches the target summary result, and display the candidate detailed data and the target summary result respectively on the target display page based on the target display format; The target display formats include: data tables, graphs, dashboards, or reports.

4. The method for real-time aggregation of detailed data according to claim 1, characterized in that, The method further includes: If it is determined that the iterative calculation has failed, the candidate detailed data is analyzed to identify abnormal candidate detailed data. The abnormal candidate detailed data is then analyzed and repaired based on the analysis results to obtain the first candidate detailed data. Re-iterate the calculation of the first candidate detailed data based on the target data processing model until the iterative calculation is confirmed to be successful. or, If the update iteration calculation fails, the update candidate detailed data is analyzed to identify abnormal update candidate detailed data. The abnormal update candidate detailed data is then analyzed and repaired based on the analysis results to obtain the second candidate detailed data. The second candidate detailed data is recalculated based on the target data processing model until the update and iteration calculation is confirmed to be successful.

5. A device for real-time aggregation of detailed data, characterized in that, include: The detailed data acquisition module is used to acquire different categories of detailed data generated in real time by the business system; wherein, the detailed data includes at least one of the following: business data, entity data, rule data, and configuration data; The candidate detailed data determination module is used to determine the target data processing model that matches the target summary indicator in response to the request instruction of the target summary indicator. Obtain the independent variables in the target data processing model, and determine each independent variable as reference detailed data associated with the target summary indicator; Determine the reference detail data table in the target database that stores the reference detail data; Based on preset time information, each of the aforementioned reference detail data tables is filtered to obtain each candidate detail data; The target summary result determination module is used to input each of the candidate detailed data into the target data processing model, and perform iterative calculations on the candidate detailed data based on the target data processing model; If the iterative calculation is successful, a target summary result matching the target summary index is obtained; If the iterative calculation fails, the abnormal data in the candidate detailed data is repaired by extraction, transformation or loading. If the repair fails, the abnormal data is temporarily filtered and the repair task is recorded. The iterative calculation is then performed again based on the repaired candidate detailed data until the target summary result is obtained. The rendering and display module is used to render and display the candidate detailed data and the target summary results on the target display page; The real-time aggregation device for detailed data also includes: an update module, used to determine candidate detailed data for updating that matches the request instruction in response to an update instruction for detailed data; The updated candidate detailed data is input into the target data processing model for update and iterative calculation. If the update iteration calculation is confirmed to be successful, the target summary update result is obtained; The content displayed on the target display page is updated based on the target summary update results and the update candidate detail data.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the real-time aggregation method for detailed data according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the real-time data aggregation method according to any one of claims 1-4.

Images