Financial data management method and device, equipment and storage medium
Through the standardization processing and abnormal types of financial data of bank financial data, combined with metadata management and blood relationship algorithms, the problem of long -term analysis of abnormal data in financial data has been solved, and rapid analysis and efficient governance have been achieved.
Patent Information
- Application Number
- CN202411426940.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2025-01-24
AI Technical Summary
Due to the huge amount of financial data in banks, the number of abnormal data increases. Manually analyzing abnormal data one by one is a lot of work and takes a long time.
By obtaining the financial data set, the sample data is standardized, the abnormal type corresponding to the abnormal value is determined, and the data management algorithm and the blood relationship algorithm are used for data control, and an abnormal type is automatically matched to reduce artificial participation.
Realize rapid analysis of abnormal data, reduce manual participation in the data governance process, and improve analysis efficiency.
Smart Images
Figure CN119357169A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing, and in particular to a financial data management method, device, equipment and storage medium. Background Art
[0002] Data governance is a comprehensive data management method that is mainly used to achieve data standardization and improve data quality, thereby improving data usability and further improving business processing efficiency, providing a reliable foundation for subsequent data analysis.
[0003] In the related technology, the data governance method includes: setting data quality standards; treating data that does not meet the data quality standards as abnormal data and tracing the source to obtain tracing results; manually analyzing the tracing results to determine the cause of the abnormal data and the type of abnormality to which the abnormal data belongs.
[0004] However, due to the huge amount of financial data in banks, the number of possible abnormal data has also greatly increased. Manual analysis of abnormal data one by one is labor-intensive and time-consuming. Summary of the invention
[0005] The present disclosure provides a financial data governance method, device, equipment and storage medium, which can realize rapid analysis of abnormal data. The technical solution at least includes the following solutions: In a first aspect, a financial data governance method is provided, comprising: obtaining a first financial data set, wherein the first financial data set includes financial data from multiple data sources; performing standardization processing on sample data, wherein the standardization processing includes determining multiple first anomaly types corresponding to multiple first anomaly values existing in the sample data, wherein the sample data is data extracted from the first financial data set; performing standardization processing on the financial data in the first financial data set according to a process for performing standardization processing on the sample data, and storing the financial data in the first financial data set after standardization processing in a data warehouse; performing data governance on the data in the data warehouse using a metadata management algorithm and a lineage algorithm, wherein the data governance includes determining, when there is a data error in the data in the data warehouse, the anomaly type corresponding to the data error based on the metadata management algorithm, the lineage algorithm and the multiple first anomaly types.
[0006] Optionally, determining the exception type corresponding to the data error based on the metadata management algorithm, the blood relationship algorithm and the multiple first exception types includes: tracing the data error using the metadata management algorithm and the blood relationship algorithm to determine a tracing result, wherein the tracing result is used to indicate the abnormal value that produces the data error; matching the tracing result with the multiple first exception types one by one to determine the exception type of the tracing result, wherein, if the match is successful, the exception type of the tracing result is the first exception type that matches the tracing result.
[0007] Optionally, the standardization processing of the sample data includes: preprocessing the sample data according to standardization rules; using data quality standards to detect the preprocessed sample data to determine the multiple first outlier values, and the multiple first outlier values are used to indicate data that does not meet the data quality standards; performing root cause analysis on the multiple first outlier values to determine the multiple first outlier types, each first outlier type corresponds to a first outlier value.
[0008] Optionally, storing the financial data in the first financial data set after standardization in a data warehouse includes: based on a data modeling method, dividing the data warehouse into multiple layers, the multiple layers including an operation data storage layer, a data detail layer, a data intermediate layer, a data service layer and a data application layer.
[0009] Optionally, the financial data in the first financial data set includes: basic customer information, customer credit history, customer loan characteristics, customer debt situation, customer socioeconomic factors, customer financial behavior data, customer risk score, customer overdue and default data, third-party financial data, and credit product characteristics.
[0010] In a second aspect, a financial data management device is provided, comprising: an acquisition module, configured to acquire a first financial data set, wherein the first financial data set includes financial data from multiple data sources; A first standardization module is used to perform standardization processing on sample data, wherein the standardization processing includes determining multiple first abnormality types corresponding to multiple first abnormal values existing in the sample data, and the sample data is data extracted from the first financial data set; a second standardization module is used to perform standardization processing on the financial data in the first financial data set according to the process of performing standardization processing on the sample data, and store the financial data in the first financial data set after standardization processing in a data warehouse; a data governance module is used to perform data governance on the data in the data warehouse using a metadata management algorithm and a blood relationship algorithm, wherein the data governance includes, when there is a data error in the data in the data warehouse, determining the abnormality type corresponding to the data error based on the metadata management algorithm, the blood relationship algorithm and the multiple first abnormality types.
[0011] Optionally, the data governance module is also used to use the metadata management algorithm and the blood relationship algorithm to trace the data error and determine the tracing result, wherein the tracing result is used to indicate the abnormal value that caused the data error; the tracing result is matched with the multiple first abnormality types one by one to determine the abnormality type of the tracing result, wherein, if the match is successful, the abnormality type of the tracing result is the first abnormality type that matches the tracing result.
[0012] Optionally, the first standardization module is also used to preprocess the sample data according to standardization rules; use data quality standards to detect the preprocessed sample data to determine the multiple first outlier values, and the multiple first outlier values are used to indicate data that does not meet the data quality standards; perform root cause analysis on the multiple first outlier values to determine the multiple first outlier types, each first outlier type corresponds to a first outlier value.
[0013] Optionally, the second standardization module is also used to divide the data warehouse into multiple layers based on a data modeling method, and the multiple layers include an operation data storage layer, a data detail layer, a data intermediate layer, a data service layer and a data application layer.
[0014] In a third aspect, a computer device is also provided, comprising: a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor, thereby executing the financial data governance method described in the above embodiment.
[0015] In a fourth aspect, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to execute the financial data governance method described in the above embodiment.
[0016] In a fifth aspect, a computer program product is also provided, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method described in the first aspect is implemented.
[0017] The beneficial effects brought by the technical solution provided by the embodiments of the present disclosure include at least: In the embodiment of the present disclosure, by determining the multiple first abnormality types corresponding to the multiple first abnormality values existing in the sample data, it is equivalent to storing the multiple first abnormality types as a template; when encountering other abnormality values of the same abnormality type as a certain abnormality value among the multiple first abnormality values, the abnormality value does not need to be analyzed for the root cause again by the staff, but only needs to match the abnormality value with the multiple first abnormality types to determine the abnormality type of the abnormality value. This reduces the manual participation in analyzing abnormal data in the data governance process, and can achieve rapid analysis of abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 A flowchart of a financial data governance method provided by an exemplary embodiment of the present disclosure is shown; Figure 2 A flowchart of a financial data governance method provided by another exemplary embodiment of the present disclosure is shown; Figure 3 A schematic diagram of the structure of a financial data management device provided by an exemplary embodiment of the present disclosure is shown; Figure 4 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0020] Unless otherwise defined, the technical or scientific terms used herein shall have the usual meanings understood by persons of ordinary skill in the field to which the present disclosure belongs. The words "first", "second", "third" and similar words used in the patent application specification and claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" and similar words mean that the elements or objects appearing before "include" or "comprises" cover the elements or objects listed after "include" or "comprises" and their equivalents, and do not exclude other elements or objects.
[0021] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.
[0022] Figure 1 A flowchart of a financial data governance method provided by an exemplary embodiment of the present disclosure is shown, and the method can be executed by a computer device. Figure 1 , the method comprising: In step 101, a first financial data set is obtained.
[0023] The first financial data set includes financial data from multiple data sources.
[0024] In the disclosed embodiments, financial data includes: basic customer information, customer credit history, customer loan characteristics, customer debt situation, customer socioeconomic factors, customer financial behavior data, customer risk score, customer overdue and default data, third-party financial data, and credit product characteristics.
[0025] Here, the customer refers to the customer who conducts financial business in the bank. The basic information of the customer is, for example, the basic personal information of the customer, including name, contact information, etc. The customer credit history is used to indicate the customer's credit history in finance, such as the customer's loan history and repayment history, credit data, etc. The customer loan characteristics are used to indicate the type of loan (such as mortgage or car loan) and loan amount (such as large or small amount) selected by the customer in the bank. The customer's debt situation refers to whether the customer is in debt, and the specific amount of the customer's debt if the customer is in debt. The customer's socioeconomic factors include the customer's occupation, people who have social relations with the customer (such as the customer's relatives and friends), etc. The customer's financial behavior data includes the customer's consumption behavior. The customer risk score is a preliminary assessment made by the bank on the customer's financial risk. If the value of a customer's customer risk score is larger, it means that the customer has a greater financial risk (for example, customers who often default on repayments have greater financial risks and their risk scores will also be higher). Customer overdue and default data is the record of the customer's overdue or default behavior in financial business. Third-party financial data refers to financial data from external systems other than the bank itself and customers who conduct business in the bank. Credit product characteristics refer to the types and risks of credit products launched by banks, as well as data such as the sales volume of credit products among customers.
[0026] Multiple data sources include: the bank's internal business system, the bank's salesperson operation data that has not been entered into the business system, and these operation data include table data or text data. Multiple data sources also include public data platforms, the People's Bank of China credit reporting system, etc. Some of these data sources can only provide some types of financial data, for example, the public data platform can only provide customer credit history data.
[0027] In step 102, the sample data is standardized.
[0028] The standardization process includes determining a plurality of first abnormality types corresponding to a plurality of first abnormal values existing in the sample data, where the sample data is data extracted from the first financial data set.
[0029] In step 103, the financial data in the first financial data set is standardized according to the process of standardizing the sample data, and the financial data in the first financial data set after the standardization is stored in the data warehouse.
[0030] In step 104, metadata management algorithms and blood relationship algorithms are used to perform data governance on the data in the data warehouse.
[0031] Data governance includes determining, when data errors exist in the data in the data warehouse, corresponding exception types of the data errors based on a metadata management algorithm, a blood relationship algorithm, and multiple first exception types.
[0032] In the embodiment of the present disclosure, by determining the multiple first abnormality types corresponding to the multiple first abnormality values existing in the sample data, it is equivalent to storing the multiple first abnormality types as a template; when encountering other abnormality values of the same abnormality type as a certain abnormality value among the multiple first abnormality values, the abnormality value does not need to be analyzed for the root cause again by the staff, but only needs to match the abnormality value with the multiple first abnormality types to determine the abnormality type of the abnormality value. This reduces the manual participation in analyzing abnormal data in the data governance process, and can achieve rapid analysis of abnormal data.
[0033] Figure 2 A flowchart of a financial data governance method provided by another exemplary embodiment of the present disclosure is shown, and the method can be executed by a computer device. Figure 2 , the method comprising: In step 201, a first financial data set is obtained.
[0034] The relevant contents of the first financial data set refer to the aforementioned step 101, and detailed description is omitted here.
[0035] Optionally, step 201 includes: determining a synchronization strategy for each data source; and acquiring a first financial data set according to the synchronization strategy for each data source.
[0036] The synchronization strategy includes: synchronization time, synchronization time range and synchronization frequency.
[0037] The process of obtaining data from multiple data sources is also called synchronization. The synchronization time is the time at which data is obtained from multiple data sources.
[0038] The synchronization time range refers to the acquisition of data generated by a data source within a certain time range when acquiring data from a certain data source. Here, since the data stored in the data source often contains data from multiple years, and some data from earlier years have lost their function and do not need to be acquired, it is necessary to set a suitable synchronization time range for each data source. After setting the synchronization time range, only the data generated by the data source within the synchronization time range needs to be acquired, thereby reducing the possibility of acquiring redundant data.
[0039] The synchronization frequency refers to the interval at which the synchronization operation is performed.
[0040] The update time of data in different data sources is different. Therefore, after selecting multiple data sources, you can choose an appropriate synchronization strategy according to the characteristics of the data source itself.
[0041] For example, a data source updates data at 9:00 a.m. on the 1st of every month, and only data within six months of the data source needs to be obtained. For this data source, the synchronization time is 9:00 a.m., the synchronization time range is from six months ago to the present, and the synchronization frequency is one month.
[0042] After the synchronization strategy of each data source is determined, financial data can be obtained from each data source according to the synchronization strategy, thereby obtaining a financial data set.
[0043] Optionally, the synchronization strategy also includes a synchronization path. Different data sources may have different data volumes, and different synchronization paths may be set according to the data volumes in the data sources to ensure maximum utilization of the storage space of the memory.
[0044] In step 202, the sample data is standardized.
[0045] The sample data is data extracted from the first financial data set.
[0046] When extracting sample data, you can extract randomly or by time range. When randomly extracting sample data, you can set a data volume threshold, and then randomly extract data from the first financial data set until the data volume threshold is reached. When extracting sample data by time range, you can set a time range, and then use the data in the first financial data set within the time as sample data.
[0047] Optionally, step 202 includes the following steps ac.
[0048] Step a: preprocess the sample data according to the standardization rules.
[0049] Optionally, step a includes: encoding and naming the sample data according to set encoding rules and naming rules, and performing deduplication and denoising processing on the sample data.
[0050] Unified coding and naming of sample data can improve the uniformity of data format and facilitate subsequent data exchange and integration; deduplication and denoising of sample data can improve the readability of sample data.
[0051] Step b: using data quality standards to detect the preprocessed sample data to determine a plurality of first outliers.
[0052] The first plurality of outliers is used to indicate data that does not meet data quality standards.
[0053] Optionally, data quality standards include: data accuracy, completeness, consistency, timeliness and reliability, as well as data quality monitoring methods (e.g., what methods are used to test data quality standards) and improvement measures (e.g., how to optimize data quality annotation). That is, the values of the above data quality standards are set in advance, and the sample data preprocessed based on the above data quality standards are tested. If a certain data does not meet any of the data quality standards, it means that the data is the first abnormal value.
[0054] The value of the pre-set data quality standard is an empirical value, which is not limited in the embodiment of the present disclosure.
[0055] Step c: performing root cause analysis on the plurality of first abnormal values to determine a plurality of first abnormal types.
[0056] Each first exception type corresponds to a first exception value.
[0057] The root cause analysis is mainly used to determine which type of anomaly the first abnormal value belongs to. After determining the existing first abnormality type, the cause of the abnormality can be analyzed based on the existing first abnormality type and the abnormal value can be processed.
[0058] There are many methods for implementing root cause analysis in the related art, and detailed description is omitted here.
[0059] Usually, the root cause analysis is implemented by the staff using different algorithms. In the embodiment of the present disclosure, by determining the first abnormality type corresponding to multiple first abnormal values, it is equivalent to storing the multiple first abnormality types as a template. When encountering other abnormal values that belong to the same abnormality type as a certain abnormal value in the multiple first abnormal values, the abnormal value does not need to be root-caused by the staff again, but only needs to match the abnormal value with the multiple first abnormality types to determine the abnormality type of the abnormal value.
[0060] That is, for any one of the multiple first abnormality types, the staff only needs to manually perform a root cause analysis once, without having to perform root cause analysis on abnormal values of the same type one by one, thereby greatly reducing the workload of the staff's repetitive work.
[0061] For example, there are three first abnormal values ABC. After performing root cause analysis on the first abnormal value A, the staff found that it belongs to the first abnormal type a. After performing root cause analysis on the first abnormal value B, the staff found that it belongs to the first abnormal type b. After performing root cause analysis on the first abnormal value C, the staff found that it belongs to the first abnormal type c, thereby obtaining the template corresponding to the first abnormal type abc. Subsequently, a new abnormal value D is found. At this time, the abnormal value D is first matched with the first abnormal types abc. For example, if the matching result is that the abnormal value D belongs to the first abnormal type a, the abnormal type of the abnormal value D is directly determined without the staff having to perform manual root cause analysis on the abnormal value D.
[0062] Optionally, the standardization process further includes an outlier processing rule. After the first outlier types corresponding to the plurality of first outliers are determined, the first outliers belonging to different first outlier types need to be processed according to the outlier processing rule.
[0063] The outlier processing rules are set based on experience, and the embodiments of the present disclosure do not limit this.
[0064] In step 203, the financial data in the first financial data set is standardized according to the process of standardizing the sample data, and the financial data in the first financial data set after the standardization is stored in the data warehouse.
[0065] In the process of standardizing the sample data, the rules and standards that need to be set have been set. At this time, the financial data in the first financial data set can be directly standardized according to the same process. In the process of standardizing the financial data in the first financial data set, if an abnormal value is found, the abnormal value is first matched with multiple first abnormal types. If the match is successful, the type of the abnormal value is directly determined. If the match fails, the staff will perform manual root cause analysis to obtain the abnormal type of the abnormal value. At this time, the abnormal value can also be regarded as a first abnormal value, and the abnormal type corresponding to the abnormal value can also be regarded as a first abnormal type for processing.
[0066] Optionally, a data modeling method may be used to manage the data in the data warehouse, the method comprising: defining the logical structure and physical structure of the data by using the data modeling method, and dividing the data warehouse into multiple layers based on the data modeling method.
[0067] Among them, the multiple layers include: operational data storage layer (Operational Data Store, ODS), data detail layer (Data Warehouse Detail, DWD), data middle layer (Data Warehouse Mart, DWM), data service layer (Data Warehouse Store, DWS) and data application layer (Analytical Data Store, ADS).
[0068] There are many related technologies on how to manage data in a data warehouse using data modeling methods, so detailed description is omitted here.
[0069] Optionally, after the financial data in the first financial data set is standardized and before the data is stored in the data warehouse, the method further includes: processing the financial data by means of horizontal and vertical data fusion. Here, the horizontal data is used to indicate different types of financial data related to the customer, such as the customer's loan data, fund data, etc. The vertical data corresponds to the horizontal data one by one, and the vertical data is the data generated in the process of a certain horizontal data or data directly related to the horizontal data changing over time. For example, when the horizontal data is loan data, the vertical data is the repayment record of each loan of the user, etc.; or when the horizontal data is fund data, the vertical data is a trend chart of a certain fund changing over time.
[0070] By processing financial data by means of horizontal and vertical data fusion, the readability of financial data can be further improved.
[0071] Optionally, the data warehouse is, for example, a corporate credit database.
[0072] Optionally, in the data warehouse, the standardized financial data is divided into standardized data and feature data, and the two are stored separately to facilitate special processing of data logic and caliber in the subsequent business indicator data calculation process.
[0073] In step 204, metadata management algorithms and blood relationship algorithms are used to perform data governance on the data in the data warehouse.
[0074] Data governance includes determining, when data errors exist in the data in the data warehouse, corresponding exception types of the data errors based on a metadata management algorithm, a blood relationship algorithm, and multiple first exception types.
[0075] There are many ways to implement metadata management algorithms and blood relationship algorithms in related technologies, so detailed description is omitted here. By adopting metadata management algorithms and data blood relationship algorithms, data transparency is improved, so that traceability results can be quickly located.
[0076] Optionally, step 204 includes step de.
[0077] Step d: Use metadata management algorithm and blood relationship algorithm to trace the data error and determine the tracing result.
[0078] Provenance results are used to indicate outliers that produce data errors.
[0079] After the financial data is stored in the data warehouse, data errors may occur in the subsequent call to the data warehouse. At this time, the outlier value that caused the data error can be located based on the metadata management algorithm and the blood relationship algorithm, that is, the traceability result. At this time, the outlier type of the outlier can be determined through step e.
[0080] Step e: Match the tracing result with the multiple first anomaly types one by one to determine the anomaly type of the tracing result.
[0081] Wherein, in the case of successful matching, the anomaly type of the traceability result is the first anomaly type that matches the traceability result.
[0082] Since the financial data is standardized through steps 202-203 and multiple first anomaly types that may exist in the first financial data set are identified, the match can usually be directly successful when executing step e. At this time, the first anomaly type that matches the tracing result is the anomaly type of the tracing result.
[0083] If the match is not successful, the staff will manually analyze the traceability result to determine the abnormal type of the traceability result.
[0084] Optionally, the method further includes: establishing a plurality of directories related to the first financial data set.
[0085] Multiple directories include: data source directory, basic data directory, and data asset directory.
[0086] The data source directory is used to count the multiple data sources in step 101; the basic data directory is used to count the data in the data warehouse; the data asset directory includes the authority data of each department and each institution, and the authority data is used to indicate the authority of each department and each institution to access the basic data directory. The data asset directory supports data sharing, automated data application, and automated data use. The data asset directory improves the transparency and accessibility of financial data within a safe range.
[0087] By establishing a data source catalog and a basic data catalog, data can be made easier to manage and access, making it easier for staff to find data. By establishing a data asset catalog, data sharing and collaboration across departments and institutions can be promoted.
[0088] Optionally, the method further includes: establishing a normalized data governance organization, which includes a decision-making layer, a management layer and an execution layer. The decision-making layer is used to issue decisions related to data governance, the management layer is used to manage the decision-making layer and supervise the operations of the decision-making layer, and the execution layer is used to perform data governance on financial data.
[0089] In the disclosed embodiments, the above-mentioned matching process can be completed automatically by computer equipment, which effectively reduces manual intervention and reduces the error correction cost and management cost caused by data problems.
[0090] The following are device embodiments of the present application. For details not described in detail in the device embodiments, reference may be made to the above method embodiments.
[0091] Figure 3 A schematic diagram of the structure of a financial data management device provided by an exemplary embodiment of the present disclosure is shown. Figure 3 The financial data governance device 300 includes: an acquisition module 301, a first standardization module 302, a second standardization module 303 and a data governance module 304.
[0092] The acquisition module 301 is used to acquire a first financial data set, where the first financial data set includes financial data from multiple data sources; The first standardization module 302 is used to perform standardization processing on the sample data, wherein the standardization processing includes determining a plurality of first abnormality types corresponding to a plurality of first abnormal values existing in the sample data, wherein the sample data is data extracted from the first financial data set; The second standardization module 303 is used to standardize the financial data in the first financial data set according to the process of standardizing the sample data, and store the financial data in the first financial data set after standardization in the data warehouse; The data governance module 304 is used to perform data governance on the data in the data warehouse using a metadata management algorithm and a lineage algorithm. The data governance includes, when there is a data error in the data in the data warehouse, determining the exception type corresponding to the data error based on the metadata management algorithm, the lineage algorithm and the multiple first exception types.
[0093] Optionally, the data governance module 304 is also used to use the metadata management algorithm and the blood relationship algorithm to trace the data error and determine the tracing result, wherein the tracing result is used to indicate the abnormal value that caused the data error; the tracing result is matched with the multiple first abnormality types one by one to determine the abnormality type of the tracing result, wherein, if the match is successful, the abnormality type of the tracing result is the first abnormality type that matches the tracing result.
[0094] Optionally, the first standardization module 302 is also used to preprocess the sample data according to standardization rules; use data quality standards to detect the preprocessed sample data to determine the multiple first outlier values, and the multiple first outlier values are used to indicate data that does not meet the data quality standards; perform root cause analysis on the multiple first outlier values to determine the multiple first outlier types, each first outlier type corresponds to a first outlier value.
[0095] Optionally, the second standardization module 303 is further used to divide the data warehouse into multiple layers based on a data modeling method, and the multiple layers include an operation data storage layer, a data detail layer, a data intermediate layer, a data service layer and a data application layer.
[0096] It should be noted that: when the financial data management device provided in the above embodiment performs financial data management, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the financial data management device provided in the above embodiment and the financial data management method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0097] The division of modules in the embodiments of the present disclosure is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present disclosure may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.
[0098] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a terminal device (which can be a personal computer, mobile phone, or communication device, etc.) or a processor (processor) to execute all or part of the steps of the method of each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0099] Figure 4 Schematic diagram of the structure of the computer device provided by the embodiment of the present disclosure. Figure 4 As shown, the computer device 400 includes: a processor 401 and a memory 402 .
[0100] The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0101] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is used to be executed by the processor 401 to implement the financial data governance method provided in the embodiments of the present disclosure.
[0102] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the computer device 400, and the computer device 400 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.
[0103] The embodiments of the present disclosure also provide a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of a computer device, the computer device is enabled to execute the financial data governance method provided in the embodiments of the present disclosure.
[0104] The embodiments of the present disclosure also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the financial data governance method provided in the embodiments of the present disclosure.
[0105] The above description is only an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A financial data governance method, characterized in that: The method comprises: Acquire a first financial data set, wherein the first financial data set includes financial data from multiple data sources; Performing standardization processing on the sample data, the standardization processing comprising determining a plurality of first abnormality types corresponding to a plurality of first abnormal values existing in the sample data, the sample data being data extracted from the first financial data set; Standardize the financial data in the first financial data set according to the process of standardizing the sample data, and store the standardized financial data in the first financial data set in a data warehouse; A metadata management algorithm and a lineage algorithm are used to perform data governance on the data in the data warehouse. The data governance includes, when there is a data error in the data in the data warehouse, determining the exception type corresponding to the data error based on the metadata management algorithm, the lineage algorithm and the multiple first exception types.
2. The method according to claim 1, characterized in that The determining, based on the metadata management algorithm, the blood relationship algorithm and the plurality of first exception types, the exception type corresponding to the data error comprises: Using the metadata management algorithm and the blood relationship algorithm to trace the data error and determine a tracing result, wherein the tracing result is used to indicate an abnormal value that causes the data error; The tracing result is matched with the multiple first exception types one by one to determine the exception type of the tracing result, wherein, if the match is successful, the exception type of the tracing result is the first exception type that matches the tracing result.
3. The method according to claim 1 or 2, characterized in that: The standardization of the sample data includes: Preprocessing the sample data according to standardized rules; Using a data quality standard to detect the preprocessed sample data to determine the plurality of first outliers, wherein the plurality of first outliers are used to indicate data that does not meet the data quality standard; A root cause analysis is performed on the multiple first abnormal values to determine the multiple first abnormal types, each of the first abnormal types corresponding to one first abnormal value.
4. The method according to claim 1 or 2, characterized in that: The storing of the standardized financial data in the first financial data set into the data warehouse includes: Based on the data modeling method, the data warehouse is divided into multiple layers, including an operation data storage layer, a data detail layer, a data intermediate layer, a data service layer, and a data application layer.
5. The method according to claim 1 or 2, wherein the financial data in the first financial data set comprises: Customer basic information, customer credit history, customer loan characteristics, customer debt situation, customer socioeconomic factors, customer financial behavior data, customer risk score, customer overdue and default data, third-party financial data, and credit product characteristics.
6. A financial data management device, characterized in that: The device comprises: An acquisition module, configured to acquire a first financial data set, wherein the first financial data set includes financial data from multiple data sources; a first standardization module, configured to perform standardization processing on the sample data, wherein the standardization processing includes determining a plurality of first abnormality types corresponding to a plurality of first abnormal values existing in the sample data, wherein the sample data is data extracted from the first financial data set; A second standardization module, configured to standardize the financial data in the first financial data set according to a process for standardizing the sample data, and store the financial data in the first financial data set after standardization in a data warehouse; A data governance module is used to perform data governance on the data in the data warehouse using a metadata management algorithm and a lineage algorithm. The data governance includes, when there is a data error in the data in the data warehouse, determining the exception type corresponding to the data error based on the metadata management algorithm, the lineage algorithm and the multiple first exception types.
7. A computer device, characterized in that: The computer device comprises: a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the method according to any one of claims 1 to 5.
9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.