Method and device for acquiring supervision submitted data, electronic equipment and program product

Abstract

The invention discloses a supervision submitted data acquisition method and device, electronic equipment and a program product, and relates to the field of financial science and technology, and the acquisition method comprises the steps: based on a preset triggering strategy, employing a data analysis tool to load business data of a plurality of business systems to a data lake of a distributed cluster, business data is stored in the data lake in a data table form; based on the business fields, processing each data table to obtain a processed data table, and based on all the processed data tables, determining a theme business table of each business field; and based on the supervision submission demand, aggregating the plurality of theme service tables to obtain aggregated data, and extracting data from the aggregated data to obtain supervision submission data. The technical problems that data cannot be accurately acquired and the quality of supervising submitted data is reduced due to the fact that the data are independently acquired by depending on each business product system in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and more specifically, to a method, apparatus, electronic device, and program product for obtaining regulatory reporting data. Background Technology

[0002] Due to differences in the construction period, technical architecture, and data standards of various business systems of financial institutions, the integration and sharing of data between different systems becomes difficult, and the transaction processing logic is complex. If the business systems are modified, the impact on the operation of financial institutions will be significant.

[0003] However, regulatory reporting requires data to be obtained from various business systems. Whether it is to complete basic business data or process reporting rules, the closed nature of each business system may lead to the repeated acquisition of the same data in different regulatory reporting scenarios. This not only increases the workload, but may also cause inconsistencies in data interpretation due to different data sources, affecting the accuracy of regulatory reporting data.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and program product for obtaining regulatory reporting data, to at least solve the technical problem in related technologies where data is obtained independently by various business product systems, resulting in inaccurate data acquisition and reduced quality of regulatory reporting data.

[0006] According to one aspect of the embodiments of this application, a method for obtaining regulatory reporting data is provided, comprising: loading business data from multiple business systems into a data lake of a distributed cluster using a data analysis tool based on a preset triggering strategy, wherein the business data is stored in the data lake in the form of data tables; processing each data table based on a business domain to obtain processed data tables, and determining a subject business table for each business domain based on all processed data tables; aggregating multiple subject business tables based on regulatory reporting requirements to obtain aggregated data, and extracting data from the aggregated data to obtain regulatory reporting data.

[0007] Furthermore, before using data analysis tools to load business data from multiple business systems into the data lake of the distributed cluster, the process also includes: generating an event identifier for each piece of business data and identifying the corresponding transaction business; recording information when the transaction business occurs, wherein the information includes at least: an Internet Protocol address and a terminal device identifier, wherein the Internet Protocol address is the address of the terminal device used when the transaction business occurs.

[0008] Furthermore, the step of using data analysis tools to load business data from multiple business systems into the data lake of the distributed cluster also includes: obtaining incremental data within a preset time period and adding incremental data to the data table using a first preset algorithm; or, if the business data has version configuration, loading all business data using a second preset algorithm and recording the change information of each business data to track the change status of the business data through the change information.

[0009] Furthermore, after using data analysis tools to load business data from multiple business systems into the data lake of the distributed cluster, the method further includes: when there are data updates in the business data, using a third preset algorithm to update the data in the data table and insert incremental data into the data table; or, using a fourth preset algorithm to clear the data in the data table and store all business data within a preset time period into the data table.

[0010] Furthermore, based on all processed data tables, the steps to determine the theme business table for each business domain include: determining the aggregate primary key of the theme business table based on the business domain, and determining multiple attribute information of the aggregate primary key; extracting the data fields corresponding to the aggregate primary key and all attribute information from multiple processed data tables; and constructing the theme business table based on all data fields, the aggregate primary key, and all attribute information.

[0011] According to one aspect of the embodiments of this application, a regulatory reporting data platform applied to the method for obtaining regulatory reporting data is also provided. The regulatory reporting data platform includes: a data loading layer for loading multiple business data into a data lake of a distributed cluster, wherein the business data is stored in the data lake in the form of data tables; a data topic processing layer for processing each data table to obtain processed data tables; and a topic application layer for aggregating all processed data tables based on regulatory reporting requirements to obtain aggregated data, and extracting data from the aggregated data to obtain regulatory reporting data.

[0012] Furthermore, the topic application layer includes: a professional topic aggregation layer, used to determine the topic business table for each business area based on the business area and all processed data tables; a general topic aggregation layer, used to aggregate multiple topic business tables based on regulatory reporting requirements to obtain aggregated data, and extract data from the aggregated data to obtain regulatory reporting data; and a reporting layer, used to process the regulatory reporting data to obtain target reporting topic data in a preset format.

[0013] According to another aspect of the embodiments of this application, a device for acquiring regulatory reporting data is also provided, comprising: a loading unit, configured to load business data from multiple business systems into a data lake of a distributed cluster using data analysis tools based on a preset triggering strategy, wherein the business data is stored in the data lake in the form of data tables; a processing unit, configured to process each data table based on a business domain to obtain processed data tables, and to determine a subject business table for each business domain based on all processed data tables; and an extraction unit, configured to aggregate multiple subject business tables based on regulatory reporting requirements to obtain aggregated data, and to extract data from the aggregated data to obtain regulatory reporting data.

[0014] Furthermore, the device for acquiring regulatory reporting data also includes: a first generation module, used to generate an event identifier for each business data and determine the transaction business corresponding to the business data before loading the business data of multiple business systems into the data lake of the distributed cluster using data analysis tools; and a first recording module, used to record information when the transaction business occurs, wherein the information includes at least: an Internet Protocol address and a terminal device identifier, wherein the Internet Protocol address is the address of the terminal device used when the transaction business occurs.

[0015] Furthermore, the loading unit includes: a first acquisition module, used to acquire incremental data within a preset time period and add incremental data to the data table using a first preset algorithm; and a second recording module, used to load all business data using a second preset algorithm when there is a version configuration for the business data, and record the change information of each business data, so as to track the change status of the business data through the change information.

[0016] Furthermore, the device for acquiring regulatory reporting data also includes: a first update module, used to update the data in the data table and insert incremental data into the data table when there are data updates in the business data after the business data of multiple business systems are loaded into the data lake of the distributed cluster using data analysis tools; and a first storage module, used to clear the data in the data table and store all business data within a preset time period into the data table using a fourth preset algorithm.

[0017] Furthermore, the processing unit includes: a first determining module, which determines the aggregated primary key of the subject business table based on the business domain, and determines multiple attribute information of the aggregated primary key; a first extraction module, which extracts the data fields corresponding to the aggregated primary key and all attribute information from multiple processed data tables; and a first construction module, which constructs the subject business table based on all data fields, the aggregated primary key, and all attribute information.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described methods for obtaining regulatory reporting data.

[0019] According to another aspect of the embodiments of this application, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors cause the one or more processors to implement any of the above-described methods for obtaining regulatory reporting data.

[0020] In this invention, based on a preset triggering strategy, data analysis tools are used to load business data from multiple business systems into a data lake in a distributed cluster. The business data is stored in the data lake in the form of data tables. Each data table is processed based on its business domain to obtain a processed data table. Based on all processed data tables, a thematic business table for each business domain is determined. Based on regulatory reporting requirements, multiple thematic business tables are aggregated to obtain aggregated data. Data is then extracted from the aggregated data to obtain the regulatory reporting data. This invention solves the technical problem in related technologies where reliance on independent data acquisition by each business product system leads to inaccurate data acquisition and reduces the quality of regulatory reporting data.

[0021] In this invention, firstly, based on a preset triggering strategy, efficient data analysis tools are used to batch load raw business data from multiple business systems into a data lake environment constructed by a distributed cluster. Then, for each data table in the data lake, preprocessing is performed according to the specific business domain, including but not limited to data cleaning and format standardization, to improve data quality and eliminate redundancy. Through preprocessing, the raw data of each business domain is transformed into processed data tables. Subsequently, all processed data tables are integrated and classified to obtain thematic business tables for each business domain. Multiple thematic business tables can be aggregated according to regulatory requirements to generate aggregated data, and data is extracted from the aggregated data to obtain high-quality regulatory reporting data information. This not only improves data processing efficiency and data integrity but also maximizes the reuse of data resources, reduces repetitive work, and shortens the data processing cycle, thereby improving the quality and efficiency of regulatory reporting. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0023] Figure 1A hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for acquiring regulatory reporting data is shown.

[0024] Figure 2 This is a flowchart of the method for obtaining regulatory reporting data according to Embodiment 1 of this application;

[0025] Figure 3 This is a schematic diagram of the regulatory reporting data platform according to Embodiment 2 of this application;

[0026] Figure 4 This is a schematic diagram of an optional regulatory reporting data acquisition device according to an embodiment of this application;

[0027] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] It should be noted that all related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected and involved in this invention are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and it does not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0031] In this invention, regulatory data, which is scattered across various business systems, is loaded and stored in a data aggregation layer instead of relying on individual business systems for processing. A private database tenant in the big data service cloud collects, cleans, models, processes, and refines large volumes of diverse information. Utilizing massive amounts of data from multiple sources, further mining and refining the data forms a systematic and long-term data governance mechanism, improving the quality of regulatory reporting. Furthermore, relying on the data platform, new general regulatory theme aggregation and professional theme aggregation layers are added to comprehensively and promptly gather business data from financial institutions and external data. After data extraction, cleaning, and refinement, the theme aggregation layer processes the data to form theme aggregation tables, further creating general and professional shared data services.

[0032] The present invention will now be described in detail with reference to various embodiments.

[0033] Example 1

[0034] According to an embodiment of this application, an embodiment of a method for obtaining regulatory reporting data is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for acquiring regulatory reporting data is shown. Figure 1 As shown, computer terminal 10 (or mobile device) may include one or more ( Figure 1The processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions may also be included. In addition, it may include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera, wherein the network interface can be connected to wired and / or wireless networks. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the regulatory reporting data acquisition method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned regulatory reporting data acquisition method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0040] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for obtaining regulatory reporting data is shown. Figure 2 This is a flowchart of the method for obtaining regulatory reporting data according to Embodiment 1 of this application, as follows: Figure 2 As shown, the method includes the following steps:

[0041] Step S201: Based on a preset triggering strategy, data analysis tools are used to load business data from multiple business systems into the data lake of the distributed cluster. The business data is stored in the data lake in the form of data tables.

[0042] In this embodiment of the invention, each job (i.e., a specific, executable data processing task or process, such as an ETL (Extract, Transform, Load) job) can be responsible for handling a specific file loading task. This involves reading data files from the source business system, performing necessary preprocessing (such as decompression and format conversion), and then loading the data into the corresponding data table in the data lake. The preset triggering strategy, i.e., the job triggering logic, can include mechanisms such as scheduled execution (e.g., execution every morning), data arrival events (i.e., starting the job when a new data file arrives), or manual triggering. Based on the preset triggering strategy, data analysis tools (i.e., tools that support ETL processes) can be used to load business data from multiple business systems into the distributed cluster's data lake (a repository for storing large amounts of raw data) to achieve timely and accurate data processing.

[0043] Step S202: Based on the business domain, process each data table to obtain a processed data table, and based on all the processed data tables, determine the theme business table for each business domain.

[0044] In this embodiment of the invention, the raw data (i.e., the raw data tables) in the data lake are processed in depth to remove unnecessary or erroneous data, resulting in processed data tables. The data in each processed data table is then categorized to determine the thematic business tables for each business domain. For example, the customer information thematic table may include basic customer information, transaction records, and contact information; the counterparty information thematic table may include counterparty information such as account number, account name, bank code, and bank name. Each thematic table is processed and categorized according to business domain.

[0045] Step S203: Based on regulatory reporting requirements, aggregate multiple subject business tables to obtain aggregated data, and extract data from the aggregated data to obtain regulatory reporting data.

[0046] In this embodiment of the invention, regulatory reporting requirements refer to the data types and formats that regulatory agencies stipulate must be submitted periodically or irregularly for financial supervision and compliance checks. Aggregated data is data extracted and integrated from multiple subject business tables to form a unified data view or meet specific analytical needs. All subject business table data related to regulatory reporting can be further aggregated and cleaned to meet the specific needs of regulatory agencies. For example, financial statistical regulatory reporting may require information such as basic customer information, counterparty information, transaction amount, transaction time, transaction location, and suspicious transaction identifiers. By extracting and integrating this information from different subject business tables to form a unified regulatory reporting data format, a data package (i.e., regulatory reporting data) that meets the reporting requirements of a specific regulatory agency is ultimately generated.

[0047] In summary, firstly, based on a preset trigger strategy, data analysis tools are used to automatically load transaction information from various business systems into a distributed cluster's data lake, forming a unified data storage pool. Then, the business data within the data lake is domain-specific, and through cleaning, organization, and aggregation, thematic business tables covering customer information, transaction details, and counterparty data are generated. Finally, according to regulatory reporting requirements, thematic business tables are integrated, and the required data is extracted and packaged to form target data that meets regulatory standards. This approach not only improves data completeness and accuracy but also optimizes the data reporting process, shortens the processing cycle, reduces the coupling between business systems, and minimizes redundant work. It also solves the technical problem of relying on independent data acquisition by various business product systems, which leads to inaccurate data acquisition and reduced quality of regulatory reporting data.

[0048] To achieve refined data management, in the regulatory reporting data acquisition method provided in Embodiment 1 of this application, an event identifier is generated for each business data and the corresponding transaction business is determined; information when the transaction business occurs is recorded, wherein the information includes at least: Internet Protocol address and terminal device identifier, wherein the Internet Protocol address is the address of the terminal device used when the transaction business occurs.

[0049] In this embodiment of the invention, when a transaction occurs, a unique event identifier can be generated for each piece of transaction data. This identifier can be a randomly generated, unique string or number combination. The event identifier serves as a key index for tracing transaction details throughout the entire data processing flow. The event identifier can uniquely identify a transaction throughout the entire data chain, regardless of which business system or channel the transaction data originates from, enabling fast and accurate information association. Furthermore, it can record environmental information at the time of the transaction, including at least the Internet Protocol address (IP address of the terminal device) and the terminal device identifier. Storing this information along with the transaction data and associating it through the event identifier enhances data traceability and integration.

[0050] In order to accurately store business data in a data table, the regulatory reporting data acquisition method provided in Embodiment 1 of this application acquires incremental data within a preset time period and adds incremental data to the data table using a first preset algorithm; or, if the business data has version configuration, a second preset algorithm is used to load all business data and record the change information of each business data to track the change status of the business data through the change information.

[0051] In this embodiment of the invention, the time window for data loading is typically set to the most recent one or more periods, such as the past day or week. For example, a preset time window of the past 24 hours can be used to extract all newly generated business data (i.e., incremental data) from various business systems since the last data loading, and the incremental data can be added to the data table using a first preset algorithm (i.e., the full load algorithm). For example, for an existing customer transaction data table, the newly extracted transaction data will be appended to the end of the table to maintain data continuity and integrity, while reducing the need to rebuild the entire data table.

[0052] Optionally, when the management requirements of business data take into account historical versions and change records, a second preset algorithm (such as the historical zipper algorithm) can be used to load all business data, and the version information of each business data can be saved during the loading process to track the change status of business data through change information.

[0053] Incremental data loading ensures that only the latest data changes are loaded, saving significant computing and storage resources and improving the real-time performance and efficiency of data processing. Version-based data loading strategies provide a complete change history for business data, helping to track data changes and providing data support for regulatory compliance, business decisions, and audit reviews.

[0054] To further improve the accuracy of the data stored in the data table, in the method for obtaining regulatory reporting data provided in Embodiment 1 of this application, when there is a data update in the business data, a third preset algorithm is used to update the data in the data table and insert incremental data into the data table; or, a fourth preset algorithm is used to clear the data in the data table and store all business data within a preset time period into the data table.

[0055] In this embodiment of the invention, the update status of business data can be monitored periodically or in real time. When business data is updated, a third preset algorithm (i.e., UPDATE+INSERT algorithm) is used to update the data in the data table and insert the newly generated incremental data into the data table, improving data continuity and avoiding unnecessary full data reconstruction. A fourth preset algorithm (i.e., full delete and full add) can be used periodically to clear all existing data in the data table and store all business data within a preset time period into the data table. For example, a financial institution's statistical system needs to update customer transaction information daily. During the morning batch processing, the system first clears the customer transaction data table stored in the data platform the previous day using the third preset algorithm. Then, it reloads all transaction records of all customers in the past 24 hours from the financial institution's transaction system, including newly occurring transactions and existing historical transaction records, ensuring that the data platform has the latest and most comprehensive transaction information. The full delete and full add method can synchronize the data status of the source system and the data platform in a short time; however, it consumes more resources and is usually used in situations with short data update cycles or moderate data volumes to balance data accuracy and system performance.

[0056] To improve the accuracy of constructing thematic business tables, in the regulatory reporting data acquisition method provided in Embodiment 1 of this application, the aggregated primary key of the thematic business table is determined based on the business domain, and multiple attribute information of the aggregated primary key is determined; data fields corresponding to the aggregated primary key and all attribute information are extracted from multiple processed data tables; and the thematic business table is constructed based on all data fields, the aggregated primary key, and all attribute information.

[0057] In this embodiment of the invention, it is necessary to identify an aggregate primary key that can serve as a data aggregation point based on a specific business domain. The aggregate primary key is a key field used to associate and aggregate related records in different data tables. For example, in the customer domain subject table, the customer number can be used as the aggregate primary key, and the customer name, document type, and contact information can be used as attribute information of the aggregate primary key. This achieves efficient classification, aggregation, and integration of data, and the data fields corresponding to the aggregate primary key and all attribute information can be extracted from multiple processed data tables. Based on all data fields, the aggregate primary key, and all attribute information, a subject business table is constructed, which not only improves data utilization and reduces repetitive data processing work, but also provides financial institutions with more powerful data management and analysis capabilities.

[0058] The regulatory reporting data acquisition method provided in this application, leveraging big data technology, can automatically identify and load incremental data within a preset time period, rapidly update data tables using data loading algorithms, and simultaneously record every change in business data, ensuring data integrity and traceability. Furthermore, by determining the aggregated primary key and attribute information of the thematic business table, it integrates large volumes of data across business domains, constructing a unified and in-depth thematic business view, thereby improving the accuracy and efficiency of data analysis.

[0059] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0060] Example 2

[0061] According to embodiments of this application, an embodiment of a regulatory reporting data platform applied to a method for obtaining regulatory reporting data is also provided. This application provides, as follows: Figure 3 The regulatory reporting data platform shown is Figure 3 This is a schematic diagram of the regulatory reporting data platform according to Embodiment 2 of this application.

[0062] exist Figure 3 The regulatory reporting data platform comprises: a data loading layer, used to load multiple business data into a distributed cluster's data lake, where the business data is stored in the data lake in the form of data tables; a data topic processing layer, used to process each data table to obtain processed data tables; and a topic application layer, used to aggregate all processed data tables based on regulatory reporting requirements to obtain aggregated data, and extract data from the aggregated data to obtain regulatory reporting data.

[0063] Optionally, the data loading layer can load source files from all front-end business systems and store them as business tables in a distributed database, enabling the access, transformation, and storage of business system data. By configuring job triggering logic, all jobs are invoked, each job corresponding to a file loading task. The corresponding loading algorithm is configured according to business requirements, and the ETL process is performed on the job ETL server. The final business tables are stored in the corresponding database in the distributed cluster.

[0064] Optionally, the data topic processing layer is used to process each data table, such as cleaning, formatting, standardizing, and performing preliminary analysis to obtain processed data tables. The topic application layer is used to aggregate all processed data tables based on regulatory reporting requirements (such as financial statistics reporting) to obtain aggregated data, and can extract data from the aggregated data to obtain regulatory reporting data.

[0065] In the regulatory reporting data platform provided in Embodiment 2 of this application, the subject application layer further includes: a professional subject aggregation layer, used to determine the subject business table for each business area based on the business area and all processed data tables; a general subject aggregation layer, used to aggregate multiple subject business tables based on regulatory reporting requirements to obtain aggregated data, and extract data from the aggregated data to obtain regulatory reporting data; and a reporting layer, used to process the regulatory reporting data to obtain target reporting subject data in a preset format.

[0066] In this embodiment of the invention, the professional theme aggregation layer can use the business tables in the data lake as source tables, and classify and process the processed data tables according to the various business objects of financial institutions and the unified reporting standards, so as to precipitate the theme business tables of the business domain.

[0067] Optionally, the professional theme aggregation layer can include theme business tables from multiple professional fields, such as customer information, personal banking, corporate banking, credit cards, and personal loans. For example, for the regulatory reporting of personal banking accounts in the retail sector, in addition to basic account and name information, regulatory reporting also requires processing of certain regulatory reporting-specific logics, such as calculating monthly balance accumulation, quarterly balance accumulation, and annual balance accumulation. In the professional theme aggregation layer, these computationally intensive logics are pre-processed in the background, so that downstream regulatory reporting layers can quickly extract data from the theme business tables of personal banking accounts in the retail sector for reporting during the reporting process.

[0068] Optionally, the general topic aggregation layer is used to integrate information across business areas to obtain aggregated data, and extract data from the aggregated data. For example, for a financial statistics regulatory reporting topic, basic information of the corresponding customer, other fund flow data, and transaction data such as customer transaction amount, transaction region, branch, and lending indicators can be obtained. This transaction data may be stored in topic business tables of different business areas. By extracting data and loading and merging it, the target data (i.e., regulatory reporting data) can be obtained. The reporting layer can process the regulatory reporting data to obtain the target reporting topic data in a preset format (i.e., a data format that meets the requirements of regulatory agencies). Through professional and general data aggregation, the diverse needs of different regulatory agencies are flexibly met. The preprocessing work of the reporting layer reduces the adjustment and repetitive work when submitting data to regulatory agencies. The data in the professional topic aggregation layer and the general topic aggregation layer does not have to be static tables. It can be provided to the outside world in the form of data services, that is, the data can be flexibly called by different applications and systems without directly accessing the underlying raw data or performing repetitive data processing. Such data services can be application programming interfaces, allowing professional applications such as customer identification models and personal loan management to directly call the required data, improving the efficiency and convenience of data use.

[0069] Example 3

[0070] This application also provides a device for acquiring regulatory reporting data. It should be noted that this device can be used to execute the method for acquiring regulatory reporting data provided in this application. The following describes the device for acquiring regulatory reporting data provided in this application.

[0071] According to an embodiment of this application, an apparatus for implementing the above-described method for obtaining regulatory reporting data is also provided. Figure 4 This is a schematic diagram of an optional regulatory reporting data acquisition device according to an embodiment of this application, such as... Figure 4 As shown, the device for acquiring regulatory reporting data may include: a loading unit 40, a processing unit 41, and an extraction unit 43.

[0072] Among them, the loading unit 40 is used to load business data from multiple business systems into the data lake of the distributed cluster based on a preset triggering strategy and using data analysis tools. The business data is stored in the data lake in the form of data tables.

[0073] Processing unit 41 is used to process each data table based on the business domain to obtain the processed data table, and to determine the subject business table for each business domain based on all the processed data tables.

[0074] Extraction unit 43 is used to aggregate multiple subject business tables based on regulatory reporting requirements to obtain aggregated data, and extract data from the aggregated data to obtain regulatory reporting data.

[0075] The regulatory reporting data acquisition device provided in this application embodiment can load business data from multiple business systems into the data lake of a distributed cluster using a data analysis tool based on a preset triggering strategy by a loading unit 40. It can process each data table based on a business domain by a processing unit 41 to obtain processed data tables, and determine the subject business tables for each business domain based on all processed data tables. It can also aggregate multiple subject business tables based on regulatory reporting requirements by an extraction unit 43 to obtain aggregated data, and extract data from the aggregated data to obtain regulatory reporting data.

[0076] Optionally, the device for acquiring regulatory reporting data further includes: a first generation module, used to generate an event identifier for each business data and determine the transaction business corresponding to the business data before loading the business data of multiple business systems into the data lake of the distributed cluster using data analysis tools; and a first recording module, used to record information when the transaction business occurs, wherein the information includes at least: an Internet Protocol address and a terminal device identifier, wherein the Internet Protocol address is the address of the terminal device used when the transaction business occurs.

[0077] Optionally, the loading unit 40 includes: a first acquisition module, used to acquire incremental data within a preset time period and add incremental data to the data table using a first preset algorithm; and a second recording module, used to load all business data using a second preset algorithm when there is a version configuration for the business data, and record the change information of each business data, so as to track the change status of the business data through the change information.

[0078] Optionally, the device for acquiring regulatory reporting data further includes: a first update module, used to update the data in the data table and insert incremental data into the data table when there are data updates in the business data after the business data of multiple business systems are loaded into the data lake of the distributed cluster using data analysis tools; and a first storage module, used to clear the data in the data table and store all business data within a preset time period into the data table using a fourth preset algorithm.

[0079] Optionally, the processing unit 41 includes: a first determining module, which determines the aggregate primary key of the subject business table based on the business domain, and determines multiple attribute information of the aggregate primary key; a first extraction module, which extracts the data fields corresponding to the aggregate primary key and all attribute information from multiple processed data tables; and a first construction module, which constructs the subject business table based on all data fields, the aggregate primary key, and all attribute information.

[0080] The aforementioned device for acquiring regulatory reporting data may also include a processor and a memory. The loading unit 40, processing unit 41, extraction unit 43, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0081] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and by adjusting kernel parameters, multiple thematic business tables can be aggregated based on regulatory reporting requirements to obtain aggregated data. Data is then extracted from the aggregated data to obtain the regulatory reporting data.

[0082] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0083] It should be noted that the loading unit 40, processing unit 41, and extraction unit 43 mentioned above correspond to steps S201 to S203 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0084] Example 4

[0085] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced with a mobile terminal or an electronic device, etc.

[0086] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0087] In this embodiment, the aforementioned computer terminal can execute the program code for the following steps in the method for obtaining regulatory reporting data: based on a preset triggering strategy, using data analysis tools to load business data from multiple business systems into a data lake of a distributed cluster, wherein the business data is stored in the data lake in the form of data tables; based on business domains, processing each data table to obtain processed data tables, and based on all processed data tables, determining the subject business table for each business domain; based on regulatory reporting requirements, aggregating multiple subject business tables to obtain aggregated data, and extracting data from the aggregated data to obtain regulatory reporting data.

[0088] Optionally, the aforementioned computer terminal may execute program code for the following steps in the method for obtaining regulatory reporting data: generating an event identifier for each business data and determining the transaction business corresponding to the business data; recording information when the transaction business occurs, wherein the information includes at least: an Internet Protocol address and a terminal device identifier, wherein the Internet Protocol address is the address of the terminal device used when the transaction business occurs.

[0089] Optionally, the aforementioned computer terminal may execute program code for the following steps in the method for obtaining regulatory reporting data: obtaining incremental data within a preset time period and adding incremental data to the data table using a first preset algorithm; or, if the business data has version configuration, loading all business data using a second preset algorithm and recording the change information of each business data to track the change status of the business data through the change information.

[0090] Optionally, the aforementioned computer terminal may execute program code for the following steps in the method for obtaining regulatory reporting data: when there is data update in the business data, use a third preset algorithm to update the data in the data table and insert incremental data into the data table; or, use a fourth preset algorithm to clear the data in the data table and store all business data within a preset time period into the data table.

[0091] Optionally, the aforementioned computer terminal may execute program code for the following steps in the method for obtaining regulatory reporting data: determining the aggregate primary key of the subject business table based on the business domain, and determining multiple attribute information of the aggregate primary key; extracting data fields corresponding to the aggregate primary key and all attribute information from multiple processed data tables; and constructing the subject business table based on all data fields, the aggregate primary key, and all attribute information.

[0092] Optionally, Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5(Only one is shown) processor 502, memory 504, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0093] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the regulatory reporting data acquisition method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned regulatory reporting data acquisition method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0094] The processor can access the information and application programs stored in the memory via the transmission device to execute the steps described above in the method for obtaining regulatory reporting data.

[0095] The embodiments of this application provide a solution for obtaining regulatory reporting data. By using big data technology, a data platform including a professional topic aggregation layer and a general topic aggregation layer is constructed. This platform can efficiently integrate and process regulatory reporting data from different business systems, achieving standardized and automated data collection and analysis. This solves the technical problem in related technologies where data is obtained independently by each business product system, resulting in inaccurate data acquisition and reduced quality of regulatory reporting data.

[0096] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be terminal devices such as smartphones, tablets, PDAs, and mobile internet devices (MIDs). Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0097] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0098] Example 5

[0099] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the regulatory reporting data acquisition method provided in Embodiment 1.

[0100] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0101] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing the steps of a method for obtaining regulatory reporting data.

[0102] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0103] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0108] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for obtaining regulatory reporting data, characterized in that, include: Based on a preset triggering strategy, data analysis tools are used to load business data from multiple business systems into a data lake in a distributed cluster. The business data is stored in the data lake in the form of data tables. Based on the business domain, each of the data tables is processed to obtain a processed data table, and based on all the processed data tables, a theme business table for each business domain is determined. Based on regulatory reporting requirements, multiple subject business tables are aggregated to obtain aggregated data, and data is extracted from the aggregated data to obtain regulatory reporting data.

2. The method for obtaining regulatory reporting data according to claim 1, characterized in that, Before using data analytics tools to load business data from multiple business systems into a distributed cluster's data lake, the process also includes: Generate an event identifier for each piece of business data and determine the transaction business corresponding to the business data; Record information when the transaction occurs, wherein the information includes at least: Internet Protocol address and terminal device identifier, wherein the Internet Protocol address is the address of the terminal device used when the transaction occurs.

3. The method for obtaining regulatory reporting data according to claim 1, characterized in that, The steps of using data analytics tools to load business data from multiple business systems into a data lake in a distributed cluster also include: Obtain incremental data within a preset time period, and add the incremental data to the data table using a first preset algorithm; or, If the business data has a version configuration, a second preset algorithm is used to load all the business data and record the change information of each business data to track the change status of the business data through the change information.

4. The method for obtaining regulatory reporting data according to claim 3, characterized in that, After using data analytics tools to load business data from multiple business systems into a distributed cluster's data lake, the process also includes: If the business data is updated, a third preset algorithm is used to update the data in the data table and insert the incremental data into the data table; or, The fourth preset algorithm is used to clear the data in the data table and store all the business data within the preset time period in the data table.

5. The method for obtaining regulatory reporting data according to claim 1, characterized in that, The step of determining the subject business table for each business domain based on all the processed data tables includes: Based on the business domain, determine the aggregate primary key of the topic business table, and determine multiple attribute information of the aggregate primary key; Extract the aggregated primary key and the data fields corresponding to all the attribute information from the multiple processed data tables; The subject business table is constructed based on all the data fields, the aggregate primary key, and all the attribute information.

6. A regulatory reporting data platform applied to the regulatory reporting data acquisition method according to any one of claims 1 to 5, characterized in that, The regulatory reporting data platform includes: A data loading layer is used to load multiple business data into a data lake of a distributed cluster, wherein the business data is stored in the data lake in the form of data tables; The data topic processing layer is used to process each of the data tables to obtain the processed data tables; The subject application layer is used to aggregate all the processed data tables based on regulatory reporting requirements to obtain aggregated data, and extract data from the aggregated data to obtain regulatory reporting data.

7. The regulatory reporting data platform according to claim 6, characterized in that, The topic application layer includes: A specialized topic aggregation layer is used to determine the topic business table for each business domain based on the business domain and all the processed data tables. A general topic aggregation layer is used to aggregate multiple topic business tables based on regulatory reporting requirements to obtain aggregated data, and to extract data from the aggregated data to obtain the regulatory reporting data. The reporting layer is used to process the regulatory reporting data to obtain target reporting topic data in a preset format.

8. A device for acquiring regulatory reporting data, characterized in that, include: The loading unit is used to load business data from multiple business systems into the data lake of a distributed cluster based on a preset triggering strategy and using data analysis tools. The business data is stored in the data lake in the form of data tables. The processing unit is configured to process each of the data tables based on the business domain to obtain a processed data table, and to determine the subject business table for each of the business domains based on all the processed data tables. The extraction unit is used to aggregate multiple subject business tables based on regulatory reporting requirements to obtain aggregated data, and to extract data from the aggregated data to obtain regulatory reporting data.

9. A computer program product, characterized in that, The method includes a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for obtaining regulatory reporting data as described in any one of claims 1 to 5.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for obtaining regulatory reporting data as described in any one of claims 1 to 5.

Images