Data summarization processing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing and financial science and technology, and particularly discloses a data summarization processing method and device, computer equipment and a storage medium. According to the method and the device, the first association relationship between the existing service modules can be determined according to the processing logic of each service, data is extracted from the data table corresponding to the service module based on the first association relationship, the service detail table corresponding to each service is generated, and the main key field of each service detail table is obtained; the summarized data table is obtained based on the main key fields and the service detail tables, data of different service modules are integrated, the situation that data are not circulated among the service systems is avoided, and the consistency of the data is improved. Moreover, the to-be-changed data is obtained in time through the data description information of the dimension table, and the summarized data table is updated, so that the data consistency is ensured at any time.

Description

Technical Field

[0001] The present application relates to the field of data processing and financial technology, and in particular to a data aggregation processing method, device, computer equipment and storage medium. Background Art

[0002] With the widespread development of computer technology in the field of financial business, the amount of data has gradually increased. However, due to the relative independence of the data in the business system, when the company's account managers, business operations, financial personnel and other parties obtain the detailed data of physical examinations in their respective systems, there will always be deviations, resulting in different communication problems and costs. For example, the customer's physical examination data is stored in the customer health information management system, and health insurance marketing belongs to the product marketing system. At this time, if the health insurance business personnel want to obtain the customer's physical examination data to analyze their health risks, so as to make targeted product recommendations and premium calculations to customers, they need to obtain data across systems and negotiate with the managers of the customer health information management system. And when the data is updated, the business party that obtains the data across systems cannot update the data it obtains in time, resulting in inconsistent data. Therefore, how to realize the aggregation of various business data and improve the consistency of data has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides a data aggregation processing method, device, computer equipment and storage medium to achieve the aggregation of various business data and improve the consistency of data.

[0004] In a first aspect, the present application provides a data aggregation processing method, the method comprising:

[0005] Acquire the business processing logic of at least one business, and determine, based on the business processing logic, a first association relationship between existing business modules related to each of the businesses;

[0006] Extracting associated fields and data corresponding to the associated fields from the data tables of the existing business modules based on the first associated relationships, and generating a business detail table for each of the businesses based on the associated fields and data corresponding to the associated fields;

[0007] Obtaining the primary key fields of each of the businesses, and generating a summary data table based on the primary key fields of each of the businesses and the business detail table;

[0008] When the data description information of the dimension table is detected, the data to be changed is determined based on the data description information, and the summary data table is updated based on the data to be changed to summarize the changed data.

[0009] In a second aspect, the present application further provides a data aggregation processing device, the device comprising:

[0010] A first association relationship determination module, used to obtain a business processing logic of at least one business, and determine a first association relationship between existing business modules related to each of the businesses based on the business processing logic;

[0011] A business detail table generating module, configured to extract associated fields and data corresponding to the associated fields from the data tables of the existing business modules based on the first associated relationships, and generate a business detail table for each of the businesses based on the associated fields and data corresponding to the associated fields;

[0012] A summary data table generation module, used to obtain the primary key fields of each of the businesses, and generate a summary data table based on the primary key fields of each of the businesses and the business detail table;

[0013] The module for updating data to be changed is used to determine the data to be changed based on the data description information when the data description information of the dimension table is detected, and to update the summary data table based on the data to be changed to summarize the changed data.

[0014] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the data aggregation processing method as described above when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the data aggregation processing method as described above.

[0016] The present application discloses a data aggregation processing method, device, computer equipment and storage medium, which obtains the business processing logic of at least one business, and determines the first association relationship between the existing business modules related to each of the businesses based on the business processing logic; extracts the association field and the data corresponding to the association field from the data table of each of the existing business modules based on the first association relationship, and generates the business detail table of each of the businesses based on the association field and the data corresponding to the association field; obtains the main key field of each of the businesses, and generates a summary data table based on the main key field of each of the businesses and the business detail table; when the data description information of the dimension table is detected, determines the data to be changed based on the data description information, and updates the summary data table based on the data to be changed to summarize the changed data. The present application can determine the first association relationship between the existing business modules according to the processing logic of each business, and extracts data from the data table corresponding to the business module based on the first association relationship, generates the business detail table corresponding to each business, obtains the main key field of each business detail table, obtains the summary data table based on each main key field and the business detail table, integrates the data of different business modules, avoids the situation where data is not circulated between business systems, and improves the consistency of data. In addition, the data to be changed is obtained in a timely manner through the data description information of the dimension table, and the summary data table is updated to ensure data consistency at all times. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic flow chart of a first embodiment of a data aggregation processing method provided by an embodiment of the present application;

[0019] Figure 2 is a schematic flow chart of a second embodiment of a data aggregation processing method provided by an embodiment of the present application;

[0020] Figure 3 is a schematic flow chart of a third embodiment of a data aggregation processing method provided by an embodiment of the present application;

[0021] Figure 4 A schematic block diagram of a data aggregation processing device provided in an embodiment of the present application;

[0022] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] The embodiments of the present application provide a data aggregation processing method, device, computer equipment and storage medium. Among them, the data aggregation processing method can be applied to a server, integrating the data of different business modules, avoiding the situation where data is not circulated between business systems, and improving the consistency of data. In addition, the data description information of the dimension table is used to obtain the data to be changed in a timely manner, and the summary data table is updated to ensure data consistency at all times. Among them, the server can be an independent server or a server cluster.

[0028] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] See also Figure 1 , Figure 1 This is a schematic flow chart of a data aggregation processing method provided by an embodiment of the present application. The data aggregation processing method can be applied to a server to integrate data from different business modules, thereby avoiding the situation where data is not circulated between business systems and improving data consistency. In addition, the data to be changed is obtained in a timely manner through the data description information of the dimension table, and the aggregated data table is updated to ensure data consistency at all times.

[0030] like Figure 1 As shown, the data aggregation processing method specifically includes steps S101 to S104.

[0031] S101, obtaining a business processing logic of at least one business, and determining a first association relationship between existing business modules related to each of the businesses based on the business processing logic;

[0032] In one embodiment, since different businesses correspond to different processing logics, the associated business modules may be different, and the data that needs to be obtained may also be different. For example, the business modules related to customer physical examinations may include a customer information recording module, a customer physical examination information recording module, a physical examination process recording module, and a physical examination result acquisition module. The data generated by the physical examination include customer information, whether the customer has participated in the physical examination, customer physical examination results, customer physical examination time consumption, etc. After obtaining the relevant data of the customer's physical examination, the insurance company or medical institution may have a customer physical examination situation analysis business, a customer health risk analysis business, a physical examination process optimization business, etc. The business modules associated with the customer physical examination situation analysis business include a customer information recording module and a customer physical examination information recording module, and it is necessary to obtain customer information and whether the customer has a physical examination. The business modules associated with the customer health risk analysis business include a customer information recording module and a physical examination result acquisition module, and it is necessary to obtain customer information and customer physical examination results. Therefore, it is necessary to determine the business modules associated with each business and the association relationship between business modules

[0033] In one embodiment, the business processing logic can be input by business personnel, or obtained by monitoring the processing flow of historical business processing through a preset real-time platform and analyzing it. Analyze the business processing logic, determine different business modules related to the business and the direct association order between modules, and obtain the first association relationship. For example, health management information is issued, associated with health management enterprise organization information, then associated with card rights and interests related information, and then associated with physical examination details.

[0034] S102, extracting associated fields and data corresponding to the associated fields from the data tables of the existing business modules based on the first associated relationships, and generating a business detail table for each of the businesses based on the associated fields and data corresponding to the associated fields;

[0035] In one embodiment, each business module has its corresponding data table for storing the data obtained by the business module. The data table includes at least one field and the data corresponding to the field. For example, the fields that may be included in the data table corresponding to the customer information recording module include "name", "age", "gender", "contact information", etc. The data required for different businesses are not the same, so the associated fields are determined based on the first association relationship between the business modules. For example, the customer physical examination status analysis business needs to obtain the customer's name, so the "name" field is used as the associated field.

[0036] In one embodiment, the associated fields and data corresponding to the associated fields in the existing business module data table of each business are obtained to generate a business detail table corresponding to each business.

[0037] In one embodiment, an ETL (Extract, Transform, Load) tool or a pre-written script can be used to extract associated fields and their corresponding data from the data tables of each business module. In the process of extracting data, data cleaning is performed to ensure the quality and consistency of the data. Data from different business modules are merged together through associated fields. The integrated data is loaded into a unified table to obtain a business detail table.

[0038] S103, obtaining the primary key field of each of the businesses, and generating a summary data table based on the primary key field of each of the businesses and the business detail table;

[0039] In one embodiment, a primary key field for uniquely identifying a record is identified in the business detail table of each business module. For example, the order module may be an order ID, and the customer module may be a customer ID.

[0040] In one embodiment, an ETL tool or a data integration tool may be used to aggregate the business detail tables into a summary data table.

[0041] In one embodiment, the structure of the summary data table can be defined according to business requirements. For example, based on business requirements, the fields that the summary data table needs to include are defined. This usually includes key indicators of each business module, such as sales, cost, profit, etc.

[0042] In one embodiment, SQL (Structured Query Language) query or ETL tool is used to aggregate data according to the primary key fields in the business detail table, and the aggregated data is inserted into the summary data table.

[0043] Furthermore, after obtaining the primary key fields of each of the businesses and generating a summary data table based on the primary key fields of each of the businesses and the business details table, it also includes: when a new business module is detected, searching for the fields to be added and the data corresponding to the fields to be added in the data table of the new business module based on the second association relationship between the new business module and the existing business module; adding the fields to be added and the data of the fields to be added into the summary data table to update the summary data table.

[0044] In one embodiment, a newly added business module can be detected by a pre-established monitoring system. When a newly added business module is detected, the business logic relationship between the newly added business module and the existing business module is analyzed to determine the second association relationship between them. Specifically, a common field that can be used for association between the two modules is identified. For example, if the newly added module is "employee performance" and the existing module is "employee information", "employee ID" may be an association field.

[0045] In one embodiment, based on the second association relationship, the fields to be added and their corresponding data are queried from the data table of the newly added business module. The newly added data is integrated into the summary data table, including data mapping rules, conversion logic, etc. It is ensured that the data of the newly added business module is correctly integrated into the summary data table, thereby supporting data analysis and decision-making across business modules and improving data consistency and availability.

[0046] In one embodiment, the summary data table is stored in the Doris database, which is a high-performance, highly available, and scalable analytical database based on a large-scale parallel processing architecture. The query results can be obtained with only a sub-second response time, which not only supports high-concurrency query scenarios, but also supports complex analysis scenarios with high throughput. It can support more business query needs, such as real-time query of employee abnormalities in any time period. It is necessary to take the abnormal indicators of each person’s latest physical examination report within a specified time range for calculation, and the original BI (Business Intelligence) calculation solution can only budget limited results in advance, which is difficult to be dynamic. Based on the Doris solution query, such complex analysis scenarios can be calculated in real time.

[0047] S104: When data description information of the dimension table is detected, the data to be changed is determined based on the data description information, and the summary data table is updated based on the data to be changed to summarize the changed data.

[0048] In one embodiment, based on a preset application or other monitoring tools, the metadata and data description information of the dimension table are monitored, such as the addition or deletion of fields, the change of data types, the change of constraints, etc.

[0049] In one embodiment, when a change is detected, relevant change information is extracted, such as the changed field name, data types before and after the change, new default values ​​or constraints, the timestamp of the change, and operator information.

[0050] In one embodiment, based on the extracted change information, the summary data table is queried to find the data to be updated, determine which records' field values ​​need to be updated, construct an update statement based on the data to be updated, and use the SQL interface of the database to execute the update statement to achieve the summary of the changed data, that is, update the summary data table.

[0051] Furthermore, when the data description information of the dimension table is detected, the data to be changed is determined based on the data description information, and the summary data table is updated based on the data to be changed to summarize the changed data, and it also includes: when a data acquisition request is received, the data acquisition request is parsed to obtain the identity information of the data requester and the data to be acquired; the identity information is authenticated to obtain the data viewing permission of the data requester; when the data viewing permission includes the viewing permission of the data to be acquired, the data to be acquired is queried in the summary data table, and the data to be acquired is transmitted to the data requester.

[0052] In one embodiment, a data acquisition request is received based on a data interface or a data service, and the data acquisition request includes the identity information of the requester and the data that the requester wants to obtain, such as the requester ID, the requested data type or identifier, etc.

[0053] In one embodiment, identity information of the requester and the data that the requester wants to obtain, such as a user name, user ID or other authentication token, data field, data set identifier or query condition, are extracted from the data acquisition request.

[0054] In one embodiment, the extracted identity information is used for authentication. After successful identity authentication, the data viewing authority of the requesting party is retrieved. The data viewing authority can be stored in a authority database, access control list, or similar authority management system.

[0055] In one embodiment, the data viewing authority of the requesting party is compared with the data to be obtained to determine whether the requesting party has the authority to view the data to be obtained. If the requesting party has the corresponding authority, that is, the data viewing authority includes the viewing authority of the data to be obtained, access to the data is allowed. If not, an error message is returned indicating that the request is rejected and explaining the reason.

[0056] In one embodiment, if the authority permits, the data to be obtained is queried in the summary data table. The retrieved data is formatted into a user-readable format and sent to the requesting party through an API interface response or other data transmission mechanism. This ensures that only authorized users can access the corresponding data, thereby improving data security.

[0057] The above embodiment provides a data aggregation processing method, device, computer equipment and storage medium, obtains the business processing logic of at least one business, and determines the first association relationship between the existing business modules related to each of the businesses based on the business processing logic; extracts the association field and the data corresponding to the association field from the data table of each of the existing business modules based on the first association relationship, and generates the business detail table of each of the businesses based on the association field and the data corresponding to the association field; obtains the primary key field of each of the businesses, and generates a summary data table based on the primary key field of each of the businesses and the business detail table; when the data description information of the dimension table is detected, determines the data to be changed based on the data description information, and updates the summary data table based on the data to be changed to summarize the changed data. The present application can determine the first association relationship between the existing business modules according to the processing logic of each business, and extracts data from the data table corresponding to the business module based on the first association relationship, generates the business detail table corresponding to each business, obtains the primary key field of each business detail table, obtains the summary data table based on each primary key field and the business detail table, integrates the data of different business modules, avoids the situation where data is not circulated between business systems, and improves the consistency of data. In addition, the data to be changed is obtained in a timely manner through the data description information of the dimension table, and the summary data table is updated to ensure data consistency at all times.

[0058] See also Figure 2 , Figure 2 This is a schematic flow chart of a data aggregation processing method provided by an embodiment of the present application. The data aggregation processing method can be applied to a server to timely reflect the changes through data description information when the data changes, and update the aggregated data table, thereby supporting the needs of business decision-making and data analysis, and improving the accuracy and timeliness of the aggregated data table.

[0059] like Figure 2 As shown, the data aggregation processing method specifically includes steps S201 to S203.

[0060] S201, parsing the data description information to obtain the data to be changed, wherein the data to be changed includes a field to be updated and a value to be updated of the field to be updated;

[0061] S202, querying in the summary data table based on the field to be updated to obtain a query result;

[0062] S203: When the query result indicates that the field to be updated exists in the summary data table and the current value corresponding to the field to be updated is different from the value to be updated, update the current value to the value to be updated to update the summary data table.

[0063] In this embodiment, the data description information, that is, the ROMA information of the dimension table, generally includes relevant details of data update, such as field name, value to be updated, and possible data update conditions.

[0064] In one embodiment, the name of the field to be updated and the corresponding value to be updated are parsed from the data description information. For example, if the data description information is a JSON object: {"field":"price","new_value":19.99}, then the field to be updated obtained after parsing is price, and the value to be updated is 19.99.

[0065] In one embodiment, a query statement is constructed based on the parsed field to be updated and other query conditions specified by the user to find records containing the field in the summary data table, wherein the query conditions are used to limit the scope of the query results.

[0066] In one embodiment, a query statement is executed to obtain records related to the field to be updated in the summary data table. In one embodiment, if the query result exists, for each record in the query result, it is checked whether the current value of the field to be updated is different from the value to be updated. If the current value is different from the value to be updated, an update operation is performed to update the current value to the value to be updated, thereby achieving real-time update of data in the summary data table.

[0067] Exemplarily, after parsing the data description information, it is determined that the value of the business name field business org.name of the business org business has changed, then data is queried in the summary data table according to the business org.id business id, and the business org.name value of these data is modified to the value to be updated.

[0068] Furthermore, after querying in the summary data table based on the field to be updated, it also includes: when the query result is a null value, adding the field to be updated and the value to be updated corresponding to the field to be updated in the summary data table to update the summary data table.

[0069] In one embodiment, when the query result is empty, that is, when there is no corresponding field to be updated in the summary data table, an insert operation needs to be performed to add the field to be updated and its corresponding value to be updated.

[0070] In one embodiment, before inserting the fields to be updated and their values ​​to be updated into the summary data table, the data types and formats of the fields to be updated and the values ​​to be updated can be checked and processed, and the values ​​to be updated can be formatted into data types suitable for insertion, for example, converting a date string into a date type. Confirm the data type and format of the fields to be updated to ensure that they are compatible with the schema of the summary data table.

[0071] In one embodiment, an insert (INSERT) SQL statement is constructed according to the structure of the summary data table and the fields to be updated. The fields to be updated and the corresponding values ​​to be updated are added to the summary data table using the SQL insert command.

[0072] In the above embodiment, when data changes, these changes are reflected in a timely manner through data description information, and the summary data table is updated, thereby supporting business decision-making and data analysis needs and improving the accuracy and timeliness of the summary data table.

[0073] See also Figure 3 , Figure 3 This is a schematic flow chart of a data aggregation processing method provided by an embodiment of the present application. The data aggregation processing method can be applied to a server, and is used to ensure the integrity and traceability of data backup by backing up the data in the summary data table and recording the backup information, while providing necessary records for data recovery and auditing, thereby improving data security. Secondly, the backup table is cleaned and the status is updated regularly, the life cycle of the backup data is effectively managed, storage space is released, and the accuracy of the backup record table is ensured, which is conducive to maintaining the cleanliness and efficient operation of the data warehouse.

[0074] like Figure 3 As shown, the data aggregation processing method specifically includes steps S301 to S302.

[0075] S301, when a preset backup task is triggered, a backup table of the current date is created, and the data in the summary data table is copied to the backup table to obtain completion information of the backup task;

[0076] S302: Based on a preset backup record table, record the completion information of the backup task, and update the data status corresponding to the current date in the backup record to a data-available status.

[0077] In this embodiment, the backup task can be triggered by a scheduled task scheduler or a built-in scheduling function of a data management tool. For example, the backup task can be triggered according to a preset cycle or time (for example, 17:00 every day).

[0078] In a specific embodiment, when a backup task is triggered, a backup table of the current date is created in a data warehouse or database system. The name of the backup table may include the current date, such as backup_table_20241001. The structure of the backup table is the same as that of the summary data table.

[0079] In a specific embodiment, the data in the summary data table is copied to the newly created backup table. Specifically, it can be completed by a database statement (such as a SQL statement) or a backup tool provided by the database. After executing the data copy operation, the completion information of the backup task is obtained, including whether the backup is successful, the amount of data backed up, the backup time, the backup date, etc.

[0080] In this embodiment, the backup record table is used to record detailed information of each backup task, including recording time, backup date, backup table name, backup status, etc.

[0081] In one embodiment, the completion information of the backup task is inserted into the backup record table, and the data status corresponding to the current date in the backup record is updated to "data status", indicating that the data of the summary data table of the current date has been backed up in the backup table.

[0082] In one embodiment, for scenarios such as reconciliation where multiple parties need to compare data for consistency, the backup table data can be used for comparison when accurately obtaining data. Similarly, the real-time table can also be used to obtain data when real-time data is needed.

[0083] Furthermore, after recording the completion information of the backup task based on the preset backup record table and updating the data status corresponding to the current date in the backup record to a data-existing status, the method further includes: when triggering the preset backup table cleaning task, obtaining the backup records within a preset period based on the backup record table; determining, based on the backup record, at least one backup table to be deleted whose data status is a data-existing status; deleting the backup table to be deleted, and updating the data status corresponding to the backup date of the backup table to be deleted in the backup record table to a data-free status.

[0084] In this embodiment, the backup table cleaning task may be a scheduled task, and the backup table cleaning task is triggered according to a preset period or time (for example, every week or every month).

[0085] In one embodiment, the preset period is a time length set by the user according to actual needs. Based on the backup record table, all backup records within the preset period are queried. Backup records with a data status of "with data status" are filtered out from the query results. For each backup table determined to be in a "with data status", a deletion operation is performed.

[0086] In one embodiment, for each deleted backup table, the data state corresponding to its backup date is updated to "no data state" in the backup record table.

[0087] In the above embodiment, by backing up the data in the summary data table and recording the backup information, the integrity and traceability of the data backup can be ensured, and necessary records can be provided for data recovery and auditing, thereby improving data security. Secondly, the backup table is cleaned and the status is updated regularly to effectively manage the life cycle of the backup data, free up storage space, and ensure the accuracy of the backup record table, which is conducive to maintaining the cleanliness and efficient operation of the data warehouse.

[0088] See also Figure 4 , Figure 4 The embodiment of the present application provides a schematic block diagram of a data aggregation processing device, which is used to execute the aforementioned data aggregation processing method. The data aggregation processing device can be configured on a server.

[0089] like Figure 4 As shown, the data aggregation processing device 400 includes:

[0090] A first association relationship determination module 401 is used to obtain a business processing logic of at least one business, and determine a first association relationship between existing business modules related to each of the businesses based on the business processing logic;

[0091] A business detail table generating module 402 is used to extract the associated fields and the data corresponding to the associated fields from the data tables of the existing business modules based on the first association relationship, and generate a business detail table for each of the businesses based on the associated fields and the data corresponding to the associated fields;

[0092] A summary data table generation module 403 is used to obtain the primary key field of each of the businesses and generate a summary data table based on the primary key field of each of the businesses and the business detail table;

[0093] The data to be changed updating module 404 is used to determine the data to be changed based on the data description information when the data description information of the dimension table is detected, and to update the summary data table based on the data to be changed to summarize the changed data.

[0094] Furthermore, the data aggregation processing device 400 further includes a data adding module, and the data adding module includes:

[0095] A to-be-added field determining unit, configured to search for the to-be-added field and data corresponding to the to-be-added field in a data table of the newly added business module based on a second association relationship between the newly added business module and the existing business module when a newly added business module is detected;

[0096] The to-be-added field adding unit is used to add the to-be-added field and the data of the to-be-added field into the summary data table to update the summary data table.

[0097] Furthermore, the to-be-changed data updating module 404 includes:

[0098] A to-be-updated field obtaining unit, configured to parse the data description information to obtain the to-be-changed data, wherein the to-be-changed data includes the to-be-updated field and the to-be-updated value of the to-be-updated field;

[0099] A query result obtaining unit, used for performing a query in the summary data table based on the field to be updated to obtain a query result;

[0100] The to-be-updated value updating unit is used to update the current value to the to-be-updated value to update the summary data table when the query result shows that the to-be-updated field exists in the summary data table and the current value corresponding to the to-be-updated field is different from the to-be-updated value.

[0101] Furthermore, the to-be-changed data updating module 404 further includes:

[0102] The to-be-updated field adding unit is used to add the to-be-updated field and the to-be-updated value corresponding to the to-be-updated field to the summary data table when the query result is a null value, so as to update the summary data table.

[0103] Furthermore, the data aggregation processing device 400 further includes a data backup module, and the data backup module includes:

[0104] A backup task processing unit, used to create a backup table of the current date when a preset backup task is triggered, and copy the data in the summary data table to the backup table to obtain completion information of the backup task;

[0105] The backup task recording unit is used to record the completion information of the backup task based on a preset backup record table, and update the data state corresponding to the current date in the backup record to a data-existing state.

[0106] Furthermore, the data backup module further includes:

[0107] A backup record acquisition unit, configured to acquire backup records within a preset period based on the backup record table when a preset backup table cleanup task is triggered;

[0108] A backup date determination unit, configured to determine, based on the backup record, at least one backup table to be deleted whose data status is a state with data;

[0109] The backup data deleting unit is used to delete the backup table to be deleted, and update the data state corresponding to the backup date of the backup table to be deleted in the backup record table to a no-data state.

[0110] Furthermore, the data aggregation processing device 400 further includes a data acquisition request processing module, and the data acquisition request processing module includes:

[0111] A data acquisition request parsing unit, configured to parse the data acquisition request upon receipt of the data acquisition request, and obtain the identity information of the data requester and the data to be acquired;

[0112] A data viewing authority obtaining unit, used to authenticate the identity information and obtain the data viewing authority of the data requester;

[0113] A data transmission unit is used to query the data to be obtained in the summary data table when the data viewing permission includes the viewing permission of the data to be obtained, and transmit the data to be obtained to the data requester.

[0114] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0115] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0116] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0117] See also Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0118] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any data aggregation processing method.

[0119] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0120] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any data aggregation processing method.

[0121] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0122] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0123] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0124] Acquire the business processing logic of at least one business, and determine, based on the business processing logic, a first association relationship between existing business modules related to each of the businesses;

[0125] Extracting associated fields and data corresponding to the associated fields from the data tables of the existing business modules based on the first associated relationships, and generating a business detail table for each of the businesses based on the associated fields and data corresponding to the associated fields;

[0126] Obtaining the primary key fields of each of the businesses, and generating a summary data table based on the primary key fields of each of the businesses and the business detail table;

[0127] When the data description information of the dimension table is detected, the data to be changed is determined based on the data description information, and the summary data table is updated based on the data to be changed to summarize the changed data.

[0128] In one embodiment, after obtaining the primary key field of each of the businesses and generating a summary data table based on the primary key field of each of the businesses and the business detail table, the processor is further configured to implement:

[0129] When a new business module is detected, searching a data table of the new business module for a field to be added and data corresponding to the field to be added based on a second association relationship between the new business module and the existing business module;

[0130] The to-be-added field and the data of the to-be-added field are added into the summary data table to update the summary data table.

[0131] In one embodiment, when the processor determines the data to be changed based on the data description information, and updates the summary data table based on the data to be changed to summarize the changed data, it is used to implement:

[0132] Parsing the data description information to obtain the data to be changed, wherein the data to be changed includes a field to be updated and a value to be updated of the field to be updated;

[0133] Based on the field to be updated, query the summary data table to obtain query results;

[0134] When the query result indicates that the field to be updated exists in the summary data table and the current value corresponding to the field to be updated is different from the value to be updated, the current value is updated to the value to be updated to update the summary data table.

[0135] In one embodiment, after implementing the query in the summary data table based on the field to be updated, the processor is further configured to implement:

[0136] When the query result is a null value, the to-be-updated field and the to-be-updated value corresponding to the to-be-updated field are added to the summary data table to update the summary data table.

[0137] In one embodiment, the processor is further configured to implement, when detecting data description information of a dimension table, determining the data to be changed based on the data description information, and updating the summary data table based on the data to be changed to summarize the changed data, and further to implement:

[0138] When a preset backup task is triggered, a backup table for the current date is created, and the data in the summary data table is copied to the backup table to obtain completion information of the backup task;

[0139] Based on the preset backup record table, the completion information of the backup task is recorded, and the data status corresponding to the current date in the backup record is updated to a data-available status.

[0140] In one embodiment, after recording the completion information of the backup task based on a preset backup record table and updating the data state corresponding to the current date in the backup record to a data-available state, the processor is further configured to implement:

[0141] When a preset backup table cleanup task is triggered, the backup records within a preset period are obtained based on the backup record table;

[0142] Based on the backup record, determining that the data status is at least one backup table to be deleted that has data;

[0143] The backup table to be deleted is deleted, and the data state corresponding to the backup date of the backup table to be deleted in the backup record table is updated to a no-data state.

[0144] In one embodiment, the processor is further configured to implement, when detecting data description information of a dimension table, determining the data to be changed based on the data description information, and updating the summary data table based on the data to be changed to summarize the changed data, and further to implement:

[0145] When receiving a data acquisition request, parsing the data acquisition request to obtain the identity information of the data requester and the data to be acquired;

[0146] Authenticate the identity information and obtain data viewing authority of the data requester;

[0147] When the data viewing authority includes the viewing authority of the data to be obtained, the data to be obtained is searched in the summary data table, and the data to be obtained is transmitted to the data requester.

[0148] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the data aggregation processing methods provided in the embodiments of the present application.

[0149] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0150] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data aggregation processing method, characterized in that: include: Acquire the business processing logic of at least one business, and determine, based on the business processing logic, a first association relationship between existing business modules related to each of the businesses; Extracting associated fields and data corresponding to the associated fields from the data tables of the existing business modules based on the first associated relationships, and generating a business detail table for each of the businesses based on the associated fields and data corresponding to the associated fields; Obtaining the primary key fields of each of the businesses, and generating a summary data table based on the primary key fields of each of the businesses and the business detail table; When the data description information of the dimension table is detected, the data to be changed is determined based on the data description information, and the summary data table is updated based on the data to be changed to summarize the changed data.

2. The data aggregation processing method according to claim 1, characterized in that: After obtaining the primary key field of each of the businesses and generating a summary data table based on the primary key field of each of the businesses and the business detail table, the method further includes: When a new business module is detected, searching a data table of the new business module for a field to be added and data corresponding to the field to be added based on a second association relationship between the new business module and the existing business module; The to-be-added field and the data of the to-be-added field are added into the summary data table to update the summary data table.

3. The data aggregation processing method according to claim 1, characterized in that: The determining the data to be changed based on the data description information, and updating the summary data table based on the data to be changed to summarize the changed data, includes: Parsing the data description information to obtain the data to be changed, wherein the data to be changed includes a field to be updated and a value to be updated of the field to be updated; Based on the field to be updated, query the summary data table to obtain query results; When the query result indicates that the field to be updated exists in the summary data table and the current value corresponding to the field to be updated is different from the value to be updated, the current value is updated to the value to be updated to update the summary data table.

4. The data aggregation processing method according to claim 3, characterized in that: After querying the summary data table based on the field to be updated, the method further includes: When the query result is a null value, the to-be-updated field and the to-be-updated value corresponding to the to-be-updated field are added to the summary data table to update the summary data table.

5. The data aggregation processing method according to claim 1, characterized in that: After the method detects the data description information of the dimension table, determines the data to be changed based on the data description information, and updates the summary data table based on the data to be changed to summarize the changed data, the method further includes: When a preset backup task is triggered, a backup table for the current date is created, and the data in the summary data table is copied to the backup table to obtain completion information of the backup task; Based on the preset backup record table, the completion information of the backup task is recorded, and the data status corresponding to the current date in the backup record is updated to a data-available status.

6. The data aggregation processing method according to claim 5, characterized in that: After recording the completion information of the backup task based on the preset backup record table and updating the data state corresponding to the current date in the backup record to a data-available state, the method further includes: When a preset backup table cleanup task is triggered, the backup records within a preset period are obtained based on the backup record table; Based on the backup record, determining that the data status is at least one backup table to be deleted that has data; The backup table to be deleted is deleted, and the data state corresponding to the backup date of the backup table to be deleted in the backup record table is updated to a no-data state.

7. The data aggregation processing method according to any one of claims 1 to 6, characterized in that: After the method detects the data description information of the dimension table, determines the data to be changed based on the data description information, and updates the summary data table based on the data to be changed to summarize the changed data, the method further includes: When receiving a data acquisition request, parsing the data acquisition request to obtain the identity information of the data requester and the data to be acquired; Authenticate the identity information and obtain data viewing authority of the data requester; When the data viewing authority includes the viewing authority of the data to be obtained, the data to be obtained is searched in the summary data table, and the data to be obtained is transmitted to the data requester.

8. A data aggregation processing device, characterized in that: include: A first association relationship determination module, used to obtain a business processing logic of at least one business, and determine a first association relationship between existing business modules related to each of the businesses based on the business processing logic; A business detail table generating module, configured to extract associated fields and data corresponding to the associated fields from the data tables of the existing business modules based on the first associated relationships, and generate a business detail table for each of the businesses based on the associated fields and data corresponding to the associated fields; A summary data table generation module, used to obtain the primary key fields of each of the businesses, and generate a summary data table based on the primary key fields of each of the businesses and the business detail table; The module for updating data to be changed is used to determine the data to be changed based on the data description information when the data description information of the dimension table is detected, and to update the summary data table based on the data to be changed to summarize the changed data.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the data aggregation processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the data aggregation processing method according to any one of claims 1 to 7.

Images