Risk data processing method, apparatus, device, medium, and computer program product

Abstract

The application relates to a risk data processing method, device, equipment, medium and computer program product, and relates to the field of big data, comprising the following steps: determining a first risk feature according to first risk data and a first standard in an intermediate database, and migrating the first risk feature to a target database; determining a second risk feature according to second risk data and a second standard in the target database; converting the second risk feature of a target object into a third risk feature according to a mapping relationship between the risk feature under the first standard and the risk feature under the second standard; integrating the first risk feature and the third risk feature to obtain an integrated risk feature; and using the integrated risk feature to evaluate the risk degree of the target object. The mapping relationship between the risk feature under the second standard and the risk feature under the first standard is combined, the migration of the risk feature under the first standard is more complete, the calculation amount of the database is reduced, and the calculation speed of the risk feature is improved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a risk data processing method, apparatus, equipment, medium, and computer program product. Background Technology

[0002] With technological advancements and enhanced product innovation capabilities, financial institutions face increasingly diverse and complex risks, leading to a variety of methods for measuring various risk exposures. Furthermore, international and domestic financial risk regulatory systems are constantly evolving. This necessitates that financial institutions rapidly update and iterate their risk measurement methods in accordance with regulatory requirements. However, the development cycle for rating systems and models is lengthy. Therefore, a crucial issue that needs to be addressed is how to quickly and accurately transform the measurement results of various risks under the old standards for existing products, clients, or institutions into measurement results under the new standards. Summary of the Invention

[0003] Therefore, it is necessary to provide a risk data processing method, apparatus, equipment, medium, and computer program product that can quickly convert and migrate existing data under the old standard to existing data under the new standard, addressing the aforementioned technical problems.

[0004] Firstly, this application provides a risk data processing method. The method includes:

[0005] Retrieve the first and second risk data of the target object from the source database;

[0006] The first risk data is migrated to an intermediate database. In the intermediate database, a first risk characteristic of the target object is determined based on the first risk data and a first standard, and the first risk characteristic is migrated to the target database.

[0007] The second risk data is migrated to the target database, and the second risk characteristic of the target object is determined in the target database based on the second risk data and the second standard.

[0008] Based on the mapping relationship between the risk characteristics under the first standard and the risk characteristics under the second standard, the second risk characteristics of the target object are converted into the third risk characteristics of the target object; the third risk characteristics are the risk characteristics under the first standard.

[0009] The first risk feature and the third risk feature are integrated according to the first standard to obtain an integrated risk feature; the integrated risk feature is used to assess the risk level of the target object.

[0010] In one embodiment, the step of integrating the first risk feature and the third risk feature according to the first standard to obtain an integrated risk feature includes:

[0011] If the first risk characteristic and the third risk characteristic of the target object are the same, then either the first risk characteristic or the third risk characteristic is deleted to obtain the integrated risk characteristic;

[0012] If the first risk feature and the third risk feature of the target object are different, then the first risk feature and the third risk feature are combined to obtain the integrated risk feature.

[0013] In one embodiment, the first risk data includes multiple types of risk data; determining the first risk characteristic of the target object based on the first risk data and the first standard includes:

[0014] Based on the first standard and the first risk data for each category, determine the risk characteristics corresponding to each category of first risk data;

[0015] Based on the first standard, the risk characteristics corresponding to each type of first risk data are integrated to obtain multiple first risk characteristics; wherein, the number of features of the multiple first risk characteristics does not exceed the number of categories of the multiple types of risk data;

[0016] The plurality of first risk characteristics are determined as the first risk characteristics of the target object.

[0017] In one embodiment, the target object includes multiple objects, and before migrating the first risk feature to the target database, it further includes:

[0018] The object to be verified is determined from the target object;

[0019] For each object to be verified, the accuracy of the integration of the first risk features of the target object is verified based on the number of categories of the first risk data of the object to be verified and the number of features of the first risk features of the object to be verified. If the number of categories is equal to the number of features, or if the number of categories is not equal to the number of features but the integration result of the risk features corresponding to each category of the first risk data of the object to be verified meets the first standard, then the integration of the first risk features of the object to be verified is determined to be accurate.

[0020] If the first risk characteristics of all objects to be verified are accurately integrated, then the integration accuracy verification is successful.

[0021] If the accuracy verification is successful, the first risk feature will be migrated to the target database.

[0022] In one embodiment, after performing an accuracy verification on the first risk features of the target object based on the number of categories of the first risk data of the object to be verified and the number of features of the first risk features of the object to be verified, the method further includes:

[0023] If the number of categories is not equal to the number of features and the integration result of the risk features corresponding to each category of the first risk data of the object to be verified does not meet the first standard, then the difference features between the integration result and the standard result corresponding to the object to be verified are analyzed according to the first standard.

[0024] Correct the first risk feature of the target object that has the difference feature based on the difference feature.

[0025] In one embodiment, before integrating the risk features corresponding to each type of first risk data according to the first standard to obtain multiple first risk features, the method further includes:

[0026] The accuracy of the calculation of risk characteristics corresponding to each type of first risk data is verified based on the first risk data and the first standard.

[0027] If the accuracy of the calculation of the risk features corresponding to the first risk data in all categories is successfully verified, the risk features corresponding to each category of first risk data are integrated according to the first standard to obtain multiple first risk features.

[0028] In one embodiment, before determining the first risk characteristic of the target object based on the first risk data and the first standard, the method further includes:

[0029] The integrity and consistency of the first risk data of the target object in the intermediate database are verified based on the first risk data of the target object in the source database.

[0030] If the integrity and consistency verification is successful, the first risk characteristic of the target object is determined based on the first risk data and the first standard.

[0031] Secondly, this application also provides a risk data processing apparatus. The apparatus includes:

[0032] The acquisition module is used to retrieve the first risk data and the second risk data of the target object from the source database;

[0033] The first determining module is used to migrate the first risk data to an intermediate database, and in the intermediate database, determine the first risk characteristic of the target object based on the first risk data and the first standard, and migrate the first risk characteristic to the target database.

[0034] The second determining module is used to migrate the second risk data to the target database, and determine the second risk characteristics of the target object in the target database based on the second risk data and the second standard.

[0035] The conversion module is used to convert the second risk feature of the target object into a third risk feature of the target object based on the mapping relationship between the risk features under the first standard and the risk features under the second standard; the third risk feature is the risk feature under the first standard.

[0036] An integration module is used to integrate the first risk feature and the third risk feature according to the first standard to obtain an integrated risk feature; the integrated risk feature is used to assess the risk level of the target object.

[0037] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0038] Retrieve the first and second risk data of the target object from the source database;

[0039] The first risk data is migrated to an intermediate database. In the intermediate database, a first risk characteristic of the target object is determined based on the first risk data and a first standard, and the first risk characteristic is migrated to the target database.

[0040] The second risk data is migrated to the target database, and the second risk characteristic of the target object is determined in the target database based on the second risk data and the second standard.

[0041] Based on the mapping relationship between the risk characteristics under the first standard and the risk characteristics under the second standard, the second risk characteristics of the target object are converted into the third risk characteristics of the target object; the third risk characteristics are the risk characteristics under the first standard.

[0042] The first risk feature and the third risk feature are integrated according to the first standard to obtain an integrated risk feature; the integrated risk feature is used to assess the risk level of the target object.

[0043] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0044] Retrieve the first and second risk data of the target object from the source database;

[0045] The first risk data is migrated to an intermediate database. In the intermediate database, a first risk characteristic of the target object is determined based on the first risk data and a first standard, and the first risk characteristic is migrated to the target database.

[0046] The second risk data is migrated to the target database, and the second risk characteristic of the target object is determined in the target database based on the second risk data and the second standard.

[0047] Based on the mapping relationship between the risk characteristics under the first standard and the risk characteristics under the second standard, the second risk characteristics of the target object are converted into the third risk characteristics of the target object; the third risk characteristics are the risk characteristics under the first standard.

[0048] The first risk feature and the third risk feature are integrated according to the first standard to obtain an integrated risk feature; the integrated risk feature is used to assess the risk level of the target object.

[0049] Fifthly, this application also provides a computer program product, including a computer program, characterized in that, when executed by a processor, the computer program performs the following steps:

[0050] Retrieve the first and second risk data of the target object from the source database;

[0051] The first risk data is migrated to an intermediate database. In the intermediate database, a first risk characteristic of the target object is determined based on the first risk data and a first standard, and the first risk characteristic is migrated to the target database.

[0052] The second risk data is migrated to the target database, and the second risk characteristic of the target object is determined in the target database based on the second risk data and the second standard.

[0053] Based on the mapping relationship between the risk characteristics under the first standard and the risk characteristics under the second standard, the second risk characteristics of the target object are converted into the third risk characteristics of the target object; the third risk characteristics are the risk characteristics under the first standard.

[0054] The first risk feature and the third risk feature are integrated according to the first standard to obtain an integrated risk feature; the integrated risk feature is used to assess the risk level of the target object.

[0055] The aforementioned risk data processing method, apparatus, computer equipment, and storage medium calculate the first risk characteristic of the target object under the first standard and the second risk characteristic under the second standard by calculating the first risk data and the second risk data under different standards in different databases. The second risk characteristic is then transformed into a third risk characteristic under the first standard, and finally, the first and third risk characteristics are integrated into the final integrated risk characteristic. This scheme combines the mapping relationship between the risk characteristics of the second standard and the risk characteristics of the first standard, making the transfer of risk characteristics under the first standard more complete. Furthermore, by using two databases to calculate the risk characteristics under the first standard and the risk characteristics under the second standard respectively, the computational load of the database is reduced, and the calculation speed of the risk characteristics is improved. Therefore, this scheme can quickly and accurately obtain all the risk characteristics of the target object under the first standard. Attached Figure Description

[0056] Figure 1 This is an application environment diagram of a risk data processing method in one embodiment;

[0057] Figure 2 This is a flowchart illustrating a risk data processing method in one embodiment;

[0058] Figure 3 This is a technology roadmap for risk data processing in another embodiment;

[0059] Figure 4 This is a flowchart illustrating the data backup steps in one embodiment;

[0060] Figure 5 This is a flowchart illustrating the data cleaning and table reconstruction steps in one embodiment;

[0061] Figure 6 This is a flowchart illustrating the first risk feature calculation step in one embodiment;

[0062] Figure 7 This is a flowchart illustrating the calculation steps for the second and third risk features in one embodiment;

[0063] Figure 8 This is a flowchart illustrating the risk feature integration steps in one embodiment;

[0064] Figure 9 This is a flowchart illustrating the risk feature derivation steps in one embodiment;

[0065] Figure 10 This is a flowchart illustrating the risk feature import steps in one embodiment;

[0066] Figure 11 This is a schematic diagram of the risk data processing device in one embodiment;

[0067] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] The risk data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0070] In one embodiment, such as Figure 2 As shown, a risk data processing method is provided, the purpose of which is to transform risk characteristics under a second standard into risk characteristics under a first standard using raw risk data, ultimately obtaining the risk characteristics under the first standard. The method includes the following steps:

[0071] Step 202: Obtain the first risk data and the second risk data of the target object from the source database.

[0072] The source database is a pre-stored database containing primary and secondary risk data for the target object. The target object refers to the entity or individual subject to risk assessment, which can be any organization or individual. The primary and secondary risk data are raw data related to the risk characteristics of the target object and used to determine those characteristics. For example, in the case of migrating and updating existing data after a change in customer risk rating rules, the primary risk data might include the old rules' customer risk level table, customer suspiciousness table, etc. The primary risk data is used to determine the primary risk characteristic, and the secondary risk data is used to determine the secondary risk characteristic. The primary and secondary risk data may contain identical or different data. Risk characteristics are one of the assessment factors used to evaluate the risk of the target object.

[0073] Step 204: Migrate the first risk data to the intermediate database. In the intermediate database, determine the first risk characteristics of the target object based on the first risk data and the first standard, and then migrate the first risk characteristics to the target database.

[0074] The first standard specifies the criteria for determining risk characteristics. The first risk characteristic of the target object under the first standard is calculated in an intermediate database. The intermediate database can be any database capable of data computation and processing. Risk characteristics represent the risk factors of the target object and the corresponding risk level, and are used to determine the final risk level of the target object. For example, determining a customer's risk level under the first standard requires considering the customer's suspiciousness, whether they are a sensitive customer, etc., and using suspiciousness and sensitiveness characteristics as risk characteristics. The target database is the database that ultimately stores the risk characteristics. In some embodiments, the intermediate database is a Hadoop framework database; due to the large customer base and the complex characteristics of the risk data, complex calculations are required in the context of rule changes. The features of the Hadoop framework can well meet these needs, as it distributes and completes computation tasks across computer clusters, dynamically moves data between nodes, and has a fast processing speed. In other embodiments, both the source database and the target database are Oracle databases. Because risk data has a strong dependency on underlying data, and complex mappings need to be considered when rules change, Oracle, based on the characteristics of relational storage, has strong storage and management capabilities, which can better meet the needs of this scenario.

[0075] Specifically, the first risk characteristic is the risk characteristic of the target object determined based on the risk characteristic determination standard in the first standard and the first risk data. The risk data can be categorized into one or multiple categories. Each category of risk data can determine the initial risk characteristic of the target object. If there is only one category of risk data, then that initial risk characteristic is the first risk characteristic. If there are multiple categories of risk data, then all the initial risk characteristics need to be integrated according to the first standard to obtain the first risk characteristic of the target object. For example, based on the initial risk characteristics a, b, and c corresponding to each of the three categories of risk data, the first standard stipulates that a, b, and c are all represented by 'a'. Therefore, during integration, only 'a' is retained, and b and c are deleted. After determining the first risk characteristic of the target object, the first risk characteristic is migrated to the target database, where further data processing is performed.

[0076] Step 206: Migrate the second risk data to the target database, and determine the second risk characteristics of the target object in the target database based on the second risk data and the second standard.

[0077] Specifically, the second risk data is migrated to the target database, where the second risk characteristics of the target object under the second standard are determined. Based on the calculation criteria for risk characteristics in the second standard, the second risk data is used to determine the second risk characteristics of the target object.

[0078] Step 208: Based on the mapping relationship between the risk characteristics under the first standard and the risk characteristics under the second standard, the second risk characteristics of the target object are converted into the third risk characteristics of the target object; the third risk characteristics are the risk characteristics under the first standard.

[0079] Specifically, a mapping relationship is established between risk characteristics under the first standard and risk characteristics under the second standard. This mapping relationship can be determined manually based on actual circumstances. That is, each risk characteristic under the second standard can be replaced by a risk characteristic under the first standard, thus converting each second risk characteristic into a risk characteristic under the first standard, i.e., a third risk characteristic. Since the first and second standards have different parts, and the first and second risk data have both similarities and differences, converting the second risk characteristics into third risk characteristics allows for a complete understanding of the target object's risk characteristics under the first standard.

[0080] Step 210: Integrate the first risk characteristic and the third risk characteristic according to the first standard to obtain the integrated risk characteristic; the integrated risk characteristic is used to assess the risk level of the target object.

[0081] Specifically, both the third risk feature and the first risk feature are risk features under the first standard, but they have some similarities and some differences. The third risk feature and the first risk feature are integrated and stored in the same table in the target database.

[0082] In the aforementioned risk data processing method, the first risk data and the second risk data are calculated separately in different databases under different standards to obtain the first risk characteristic of the target object under the first standard and the second risk characteristic under the second standard. The second risk characteristic is then transformed into a third risk characteristic under the first standard. Finally, the first and third risk characteristics are integrated into the final integrated risk characteristic. This scheme combines the mapping relationship between the risk characteristics of the second standard and the risk characteristics of the first standard, making the transfer of risk characteristics under the first standard more complete. Furthermore, by using two databases to calculate the risk characteristics under the first standard and the risk characteristics under the second standard respectively, the computational load of the database is reduced, and the calculation speed of the risk characteristics is improved. Therefore, this scheme can quickly and accurately obtain all the risk characteristics of the target object under the first standard.

[0083] In one embodiment, the first risk characteristic and the third risk characteristic are integrated according to a first criterion to obtain an integrated risk characteristic, including:

[0084] Step 302: If the first risk feature and the third risk feature of the target object are the same, then delete any one of the risk features from the first risk feature and the third risk feature to obtain the integrated risk feature;

[0085] Step 304: If the first risk feature and the third risk feature of the target object are different, then the first risk feature and the third risk feature are combined to obtain the integrated risk feature.

[0086] Specifically, the integration of the first and third risk features mainly involves merging and deleting risk features. Inevitably, the first and third risk features may overlap, but the risk features corresponding to the target object must not overlap. If the first and second risk features overlap, one of them is deleted, retaining only the other. If the first and third risk features are different, both the first and second risk features must be retained. In this embodiment, by integrating the first and third risk features, the risk features of the final target object are more complete and accurate.

[0087] In some embodiments, the uniqueness of the integrated risk features is verified to ensure that there are no duplicate integrated risk features corresponding to the target object.

[0088] In one embodiment, the first risk data is multi-category risk data; based on the first risk data and a first standard, the first risk characteristics of the target object are determined, including:

[0089] Step 402: Determine the risk characteristics corresponding to each type of first risk data based on the first standard and each type of first risk data;

[0090] Step 404: Integrate the risk features corresponding to each type of first risk data according to the first standard to obtain multiple first risk features; wherein the number of features of the multiple first risk features does not exceed the number of categories of multiple risk data.

[0091] Step 406: Identify multiple first risk characteristics as the first risk characteristics of the target object.

[0092] Specifically, the first risk data of the target object includes risk data of multiple categories, that is, multi-class risk data. Based on the first standard and according to the first risk data of each category, the risk characteristics corresponding to each category of first risk data are calculated. Then, according to the first standard, the risk characteristics corresponding to each category of first risk data are integrated, and the final first risk characteristics include multiple risk characteristics. According to the first standard, in the process of integrating the risk labels corresponding to each category of first risk data, some risk characteristics can be merged into the same characteristic. The number of multiple first risk characteristics can be equal to or unequal to the number of categories of multi-class risk data. That is, each category of first risk data can correspond one-to-one with each first risk characteristic, or multiple categories of first risk data can correspond to one first risk characteristic. The resulting multiple first risk characteristics are determined as the first risk characteristics of the target object.

[0093] In one embodiment, the second risk data consists of multiple types of second risk data. Based on the second standard and each type of second risk data, the risk characteristics corresponding to each type of second risk data are determined. The risk characteristics corresponding to each type of second risk data are integrated according to the second standard to obtain multiple second risk characteristics. The number of features of the multiple second risk characteristics does not exceed the number of categories of the multiple types of second risk data. The multiple second risk characteristics are determined as the second risk characteristics of the target object.

[0094] In one embodiment, the target object includes multiple objects, and before migrating the first risk feature to the target database, it also includes:

[0095] Step 502: Determine the object to be verified from the target objects;

[0096] Step 504: For each object to be verified, the accuracy of the integration of the first risk features of the target object is verified based on the number of categories of the first risk data of the object to be verified and the number of features of the first risk features of the object to be verified. If the number of categories is equal to the number of features, or if the number of categories is not equal to the number of features but the integration result of the risk features corresponding to each category of the first risk data of the object to be verified meets the first standard, then the integration of the first risk features of the object to be verified is determined to be accurate.

[0097] Step 506: If the first risk characteristics of all objects to be verified are accurately integrated, then the integration accuracy verification is successful.

[0098] Step 508: If the accuracy verification is successful, migrate the first risk feature to the target database.

[0099] Specifically, before migrating the first risk features to the target database, it is necessary to verify the multiple first risk features obtained through integration to check whether any first risk features of the target objects are missing. In this embodiment, the verification of risk features is to verify whether the first risk features corresponding to each target object are correctly integrated. The objects to be verified can be determined according to the actual data volume. If the data volume is small, all target objects can be used as objects to be verified. If the data volume is large, a portion of the target objects can be extracted as objects to be verified. The integration accuracy of the first risk features of all target objects is verified by checking the integration accuracy of the first risk features of the extracted objects to be verified. In some embodiments, stratified sampling is used to determine the objects to be verified from the target objects. The basis for stratified sampling can be the data characteristics of the first risk data of the target objects. The first risk data of different target objects may have the same data characteristics or different data characteristics. Specifically, stratification can be carried out according to data characteristics, and each stratum extracts objects to be verified according to a preset sampling ratio.

[0100] The accuracy of integrating the first risk feature for each object to be verified is checked by comparing the number of categories of the first risk data for the object to be verified with the number of features of the risk features. If the number of categories and features are equal, the system has concatenated the risk features corresponding to the first risk data of all categories of the object to be verified, and no risk features are missing. The first risk feature integration of the object to be verified is accurate, and the verification is successful. If the number of categories and features are not equal, it indicates that there may be missing risk features or redundant risk features (e.g., duplicate risk features). In this case, it is necessary to check whether the integration result of the risk features corresponding to each category of the first risk data of the object to be verified meets the first standard. If the integration result meets the first standard, it means that the change in the number of categories and features is normal, and the first risk feature integration of the object to be verified is correct. If the first risk features of all objects to be verified are integrated accurately, it means that the overall integration accuracy of the data is high, and the accuracy of the first risk feature integration of the target object can be determined to be successful. The integrated first risk features can be migrated to the target database for the next data processing step.

[0101] For example, a customer is simultaneously in three categories of risk data. Each category of risk data determines the initial risk characteristics corresponding to the three categories of risk data as a, b, and c, respectively. In this case, the number of categories n1 is 3, and the customer's first risk characteristic is a. In this case, the number of characteristics n2 is 1, and n1 ≠ n2. If the first standard stipulates that a, b, and c are all represented by a to describe their risk characteristics, then it means that the customer's first risk characteristic is correct, and the verification is successful. If there is no such stipulation, then the verification fails. In this case, n1 > n2, and the difference items b and c can be determined as the customer's first risk characteristics.

[0102] In the above embodiments, by extracting the objects to be verified and verifying the integration accuracy of the first risk features of all target objects by checking the number of categories of the first risk data of the objects to be verified and the number of features of the first risk features of the objects to be verified, it is possible to quickly and accurately determine whether errors have occurred in the integration process of the risk features corresponding to each category of first risk data. At the same time, estimating the whole by sample can more quickly verify all first risk features, thus improving the verification speed. Furthermore, it reduces the subjectivity and uncertainty that may exist when manually verifying data.

[0103] In one embodiment, after performing an integrated accuracy verification of the first risk features of the target object based on the number of categories of the first risk data of the object to be verified and the number of features of the first risk features of the object to be verified, the method further includes:

[0104] Step 602: If the number of categories is not equal to the number of features and the integration result of the risk features corresponding to each category of the first risk data of the object to be verified does not meet the first standard, then analyze the difference features between the integration result and the standard result corresponding to the object to be verified according to the first standard.

[0105] Step 604: Correct the first risk feature with differential characteristics in the target object based on the differential characteristics.

[0106] Specifically, if the number of categories is not equal to the number of features, and the integrated result of the risk features corresponding to each category of the first risk data of the object to be verified does not meet the first standard, it indicates that the integration of the first risk features of the object to be verified is inaccurate. This suggests that there may be many cases of inaccurate integration among the first risk features of all target objects, and the inaccurate first risk features need to be corrected. The difference between the integrated result of the first risk features of the object to be verified (i.e., the integrated result corresponding to the object to be verified) and the standard result is analyzed according to the first standard. The standard result is the re-determined integration result based on the first standard and the first risk data of each category of the object to be verified that contains errors. The integration result to be verified is compared with the standard result. After determining the difference characteristics, the first risk feature with the difference characteristics can be found. Based on the first risk data of the target object corresponding to the first risk feature with the difference characteristics and the first standard, the first risk feature of the target object is re-determined, and the original first risk feature is corrected based on the re-determined first risk feature. In some embodiments, machine learning can be used to find the first risk features with the difference characteristics.

[0107] In the above embodiments, by analyzing the difference characteristics, the first risk characteristics of differences with the same features are found, and problematic first risk characteristics are corrected. Automatic updates are made for existing differences, reducing the workload of manual problem confirmation and correction. Real-time verification and automatic data correction mechanisms avoid duplicate program execution and resource waste.

[0108] In one embodiment, the second risk characteristic of the target object is integrated for accuracy verification and correction in the same manner as in the corresponding embodiments described above. For details, please refer to the integration accuracy verification process for the second risk characteristic, which will not be repeated here.

[0109] In one embodiment, before integrating the risk features corresponding to each type of first risk data according to a first criterion to obtain multiple first risk features, the method further includes:

[0110] Step 702: Calculate and verify the accuracy of the risk characteristics corresponding to each type of first risk data based on each type of first risk data and the first standard;

[0111] Step 704: If the accuracy of the calculation of the risk features corresponding to the first risk data of all categories is successfully verified, the risk features corresponding to each category of first risk data are integrated according to the first standard to obtain multiple first risk features.

[0112] Specifically, before integrating the risk features corresponding to each type of first risk data, it is necessary to verify the risk features corresponding to each type of first risk data to determine whether the risk data calculated based on the first risk data and the first standard is accurate. Based on each type of first risk data and the first standard, the risk features corresponding to each type of first risk data are recalculated. If the recalculated risk features are consistent with the original risk features corresponding to that type of first risk data, the accuracy verification of the calculation of the risk features corresponding to that type of first risk data is successful. If the accuracy verification of the calculation of the risk features corresponding to all types of first risk data is successful, the risk features corresponding to each type of first risk data are further processed. In some embodiments, if the recalculated risk features are inconsistent with the original risk features corresponding to that type of first risk data, the original risk features are corrected using the recalculated risk features. In other embodiments, the objects to be verified can be determined by using either full verification or sampling verification based on the number of target objects. The accuracy of the calculation of the risk features corresponding to each type of first risk data of the target objects is verified by verifying the accuracy of the calculation of the risk features corresponding to each type of first risk data of the target objects. The sampling verification can be performed using random sampling or stratified sampling. The stratified sampling method is the same as that described in the above embodiments and will not be repeated here. This embodiment verifies the accuracy of the calculated risk features corresponding to each type of first risk data before integrating them, thus avoiding the duplication of subsequent steps and the waste of resources.

[0113] In one embodiment, the accuracy of the calculation of the risk features corresponding to each type of second risk data is also verified based on each type of second risk data and the second standard; if the accuracy verification of the calculation of the risk features corresponding to all types of second risk data is successful, the risk features corresponding to each type of second risk data are integrated according to the second standard to obtain multiple second risk features.

[0114] In one embodiment, before determining the first risk characteristic of the target object based on the first risk data and the first criterion, the method further includes:

[0115] Step 802: Perform integrity and consistency verification on the first risk data of the target object in the intermediate database based on the first risk data of the target object in the source database;

[0116] Step 804: If the integrity and consistency verification is successful, determine the first risk characteristic of the target object based on the first risk data and the first standard.

[0117] Specifically, data integrity verification is mainly accomplished through row count comparison and primary key count. Data consistency verification mainly verifies whether the data type, data length, and data format of the same fields are consistent. After migrating the first-risk data to the intermediate database, to determine whether the migrated first-risk data has any problems, data integrity and consistency verification needs to be performed before processing the first-risk data. If the first-risk data of the target object in the intermediate database is complete and consistent with the first-risk data of the target object in the source database, then the integrity and consistency verification is successful, and the first-risk data can proceed to the next step of processing. In some embodiments, if the first-risk data of the target object in the intermediate database is incomplete (e.g., some first-risk data of the target object is missing), then the missing data is retrieved from the source database and supplemented in the intermediate database. If the first-risk data of the target object in the intermediate database is inconsistent with the first-risk data of the target object in the source database, then the correct first-risk data can be retrieved from the source database based on the target object, and the retrieved first-risk data is supplemented into the intermediate database. In other embodiments, the objects to be verified can be determined by using either full verification or sampling verification based on the number of target objects. The consistency of the first risk data of the target objects is verified by verifying the first risk data of the objects to be verified in the intermediate database. This embodiment avoids the duplication of subsequent steps and the waste of resources by performing data consistency verification before calculating the first risk data in the intermediate database.

[0118] In one embodiment, before determining the second risk characteristic of the target object in the target database based on the second risk data and the second standard, the method further includes: performing integrity and consistency verification on the second risk data of the target object in the target database based on the second risk data of the target object in the source database; and if the integrity and consistency verification is successful, determining the second risk characteristic of the target object based on the second risk data and the second standard.

[0119] The following specific embodiment illustrates this solution: The technical roadmap is as follows: Figure 3 As shown.

[0120] S1, Data Backup (e.g.) Figure 4 (As shown)

[0121] The source database (Oracle database 1) contains n primary risk data tables A1.REL1, A1.REL2...A1.RELn, each recording primary risk data for multiple customers of the same type. Export A1.REL1, A1.REL2...A1.RELn from the source database and generate corresponding files A1_BIN1, A1_BIN2...A1_BINn. Verify the existence and emptiness of the files, and check file integrity by comparing row counts. If the verification fails, repeat the corresponding steps; if the verification still fails, issue an error warning and output detailed results.

[0122] S2, Data cleaning, rebuilding a new table (e.g.) Figure 5 (As shown)

[0123] Data cleanup is performed on the intermediate database (Hadoop database) and the target database (Oracle database 2). If historical data exists in the target database, the TRUNCATE statement is used to clean it first, and the indexes are rebuilt. New tables are rebuilt in both the target and intermediate databases. These new tables mainly include data source tables, result tables, and statistical reports. The data source tables store risk data, the result tables store data validation results, and the statistical reports store relevant data validation information, including validation results, difference characteristics, and accuracy. Various parameter tables may also be included, used to calculate the final risk level. An exact match is performed between the table list and all tables in the target and intermediate databases. The existence and empty status of tables of the corresponding types in the table list are verified in both databases. If the validation fails, the corresponding steps are re-executed. If the validation still fails, an error warning is issued, and detailed results are output. If the validation passes, the process proceeds to the next step.

[0124] S3, HADOOP initialization, first save (e.g.) Figure 6 (As shown)

[0125] Load the exported files A1_BIN1, A1_BIN2...A1_BINn from S1 into the corresponding tables in the intermediate database, forming tables B.TMP1, B.TMP2...B.TMPn in the intermediate database. Perform validation on tables B.TMP1, B.TMP2...B.TMPn: Using the primary key (e.g., customer number) as the search term, perform integrity validation on the corresponding tables B.TMP1, B.TMP2...B.TMPn in the intermediate database (hereinafter referred to as target files) based on tables A1.REL1, A1.REL2...A1.RELn in the source database (hereinafter referred to as source files). If there are discrepancies (e.g., inconsistent row counts), search tables A1.REL1, A1.REL2...A1.RELn and supplement the data in the target files; [Further details regarding data entry are needed]. The values ​​of key fields are checked for consistency and validity (e.g., in table B.TMP1, the dictionary values ​​for the "suspicion level" field are 01, 02, and 03; if some values ​​are initialized to 1, it indicates inconsistency). Stratified sampling is performed based on object characteristics to check the consistency between tables A1.REL1, A1.REL2...A1.RELn and tables B.TMP1, B.TMP2...B.TMPn. If discrepancies are found, a search is performed in tables A1.REL1, A1.REL2...A1.RELn, and the results are supplemented in tables B.TMP1, B.TMP2...B.TMPn. All validation statistics are compiled into a statistical report and exported.

[0126] After verification and supplementary entry, based on the new standard and the first risk data in tables B.TMP1, B.TMP2...B.TMPn, the first risk feature 1, first risk feature 2, ..., first risk feature n under the new standard are calculated in parallel and denoted as newitem1, newitem2...newitemn, respectively, and then assigned to tables B.CAL1, B.CAL2...B.CALn. Stratified sampling is performed on each table in B.TMP1, B.TMP2...B.TMPn to verify the accuracy of the calculated risk features for each table. If discrepancies are found, the data is retrieved from tables B.TMP1, B.TMP2...B.TMPn, recalculated according to the new standard, and then supplemented in B.CAL1, B.CAL2...B.CALn.

[0127] Consolidate tables B.CAL1, B.CAL2...B.CALn into table B.CAL using primary keys, and export B.CAL to generate file B_BIN. Perform a primary key uniqueness check on table B.CAL, confirming no duplicate entries. Then perform an integration integrity check on table B.CAL, identifying discrepancies, filtering all records with the same characteristics as the discrepancies, and supplementing them according to the same rules. Output the discrepancies as a report for subsequent iterations and optimizations.

[0128] The exported report styles and examples are shown in Table 1 below. The report is used for statistical verification and data correction results, and provides details of the corrected data.

[0129] Table 1 Report Format

[0130]

[0131] S4, Oracle data migration (such as...) Figure 7 (As shown)

[0132] Export m second-risk data tables from the source database, each storing the same type of second-risk data. Migrate these m second-risk data tables to the target database. Verify the integrity of the data migration by performing an exact match on the primary keys; if differences exist, output the differences.

[0133] Based on the old standard, calculate the risk characteristics corresponding to each second risk data table under the old standard, denoted as olditem1, olditem2, ..., olditemm. Use the same integration method to integrate olditem1, olditem2, ..., olditemm to obtain the second risk characteristics, and generate table A1.TMP1. Perform uniqueness verification on the primary key, stratified sampling based on sample characteristics, and verify the accuracy of the calculation rules. If discrepancies exist, recalculate the risk characteristics corresponding to the discrepancies and add them to the results table; filter all records with the same characteristics as the discrepancies and add them according to the same rules; output the discrepancies as a report for subsequent iterative optimization.

[0134] Based on the mapping relationship between the risk characteristics of the new standard and the risk characteristics of the old standard, the second risk characteristic in Table A1.TMP1 is converted into the third risk characteristic under the new standard, and Table A2.TMP1 is generated. The accuracy of the mapping relationship under the new standard is then verified.

[0135] Data updates after S5 and Oracle data migration (e.g.) Figure 8 (As shown)

[0136] Load file B_BIN into the target database to generate table A2.TMP2, and link it to A2.TMP1 via the primary key. Calculate and integrate the key elements newitem1, newitem2...newitemN generated under the new standard, and record them in table A2.TMP3. Perform a uniqueness check on the primary key of table A2.TMP3; check the uniqueness of the same key element under all primary keys. If a key element is duplicated, remove the duplicates and update the corresponding field in table A2.TMP3. Output a report based on the check and update results.

[0137] S6, Oracle data export (e.g.) Figure 9 (As shown)

[0138] Export table A2.TMP3 as A2_BIN and transfer it to the HADOOP server. Verify the integrity of the exported file by checking the number of rows.

[0139] S7, Hadoop data import (e.g.) Figure 10 (As shown)

[0140] Load A2_BIN into the previously initialized corresponding table, and assign default values ​​to fields other than risk characteristics. Verify the integrity of the exported file and the accuracy of key elements using primary key statistics.

[0141] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0142] Based on the same inventive concept, this application also provides a risk data processing apparatus for implementing the risk data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more risk data processing apparatus embodiments provided below can be found in the limitations of the risk data processing method described above, and will not be repeated here.

[0143] In one embodiment, such as Figure 11 As shown, a risk data processing device 400 is provided, including: an acquisition module 401, a first determination module 402, a second determination module 403, a conversion module 404, and an integration module 405, wherein:

[0144] The acquisition module 401 is used to obtain the first risk data and the second risk data of the target object from the source database.

[0145] The first determination module 402 is used to migrate the first risk data to an intermediate database. In the intermediate database, the first risk characteristics of the target object are determined based on the first risk data and the first standard, and the first risk characteristics are migrated to the target database.

[0146] The second determination module 403 is used to migrate the second risk data to the target database, and determine the second risk characteristics of the target object in the target database based on the second risk data and the second standard.

[0147] The conversion module 404 is used to convert the second risk characteristic of the target object into the third risk characteristic of the target object according to the mapping relationship between the risk characteristics under the first standard and the risk characteristics under the second standard; the third risk characteristic is the risk characteristic under the first standard.

[0148] The integration module 405 is used to integrate the first risk feature and the third risk feature according to the first standard to obtain the integrated risk feature; the integrated risk feature is used to assess the risk level of the target object.

[0149] In one embodiment, the integration module 405 is specifically used to delete either the first risk feature or the third risk feature of the target object if the first risk feature and the third risk feature are the same, thereby obtaining an integrated risk feature; and to combine the first risk feature and the third risk feature of the target object if the first risk feature and the third risk feature are different, thereby obtaining an integrated risk feature.

[0150] In one embodiment, the first risk data is multiple types of risk data; the first determining module 402 is specifically used to determine the risk characteristics corresponding to each type of first risk data according to the first standard and each type of first risk data; to integrate the risk characteristics corresponding to each type of first risk data according to the first standard to obtain multiple first risk characteristics; wherein, the number of features of the multiple first risk characteristics does not exceed the number of categories of multiple risk data; and to determine the multiple first risk characteristics as the first risk characteristics of the target object.

[0151] In one embodiment, the target object includes multiple objects, and the risk data processing device 400 further includes a first verification module. The first verification module is used to determine the object to be verified from the target object; for each object to be verified, the integration accuracy of the first risk feature of the target object is verified according to the number of categories of the first risk data of the object to be verified and the number of features of the first risk feature of the object to be verified; if the number of categories is equal to the number of features, or if the number of categories is not equal to the number of features but the integration result of the risk feature corresponding to each category of the first risk data of the object to be verified meets the first standard, then the integration accuracy of the first risk feature of the object to be verified is determined to be accurate; if the first risk features of all the objects to be verified are integrated accurately, then the integration accuracy verification is determined to be successful, and if the integration accuracy verification is successful, the first risk feature is migrated to the target database.

[0152] In one embodiment, the first verification module is further configured to, if the number of categories is not equal to the number of features and the integration result of the risk features corresponding to each category of first risk data of the object to be verified does not conform to the first standard, analyze the difference features between the integration result and the standard result corresponding to the object to be verified according to the first standard; and correct the first risk features with difference features in the target object according to the difference features.

[0153] In one embodiment, the risk data processing device 400 further includes a second verification module, which is used to verify the accuracy of the calculation of the risk features corresponding to each type of first risk data according to the first risk data and the first standard; if the accuracy of the calculation of the risk features corresponding to all types of first risk data is successfully verified, the risk features corresponding to each type of first risk data are integrated according to the first standard to obtain multiple first risk features.

[0154] In one embodiment, the risk data processing device 400 further includes a third verification module, which is used to perform integrity and consistency verification on the first risk data of the target object in the intermediate database based on the first risk data of the target object in the source database; if the integrity and consistency verification is successful, the first risk characteristic of the target object is determined based on the first risk data and the first standard.

[0155] Each module in the aforementioned risk data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0156] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores risk-related data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a risk data processing method.

[0157] Those skilled in the art will understand that Figure 12The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0158] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0159] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0160] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0161] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0162] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A risk data processing method, characterized in that, The method includes: Retrieve the first and second risk data of the target object from the source database; The first risk data is migrated to an intermediate database. In the intermediate database, a first risk characteristic of the target object is determined based on the first risk data and a first standard, and the first risk characteristic is migrated to the target database. The intermediate database is a database based on the Hadoop framework. Both the source database and the target database are Oracle databases. The second risk data is migrated to the target database, and the second risk characteristic of the target object is determined in the target database based on the second risk data and the second standard. Based on the mapping relationship between the risk characteristics under the first standard and the risk characteristics under the second standard, the second risk characteristics of the target object are converted into the third risk characteristics of the target object; the third risk characteristics are the risk characteristics under the first standard. The first risk feature and the third risk feature are integrated according to the first standard to obtain an integrated risk feature; the integrated risk feature is used to assess the risk level of the target object. The first risk data includes multiple types of risk data; determining the first risk characteristic of the target object based on the first risk data and the first standard includes: Based on the first standard and each type of first risk data, determine the risk characteristics corresponding to each type of first risk data; Based on the first standard, the risk characteristics corresponding to each type of first risk data are integrated to obtain multiple first risk characteristics; wherein, the number of features of the multiple first risk characteristics does not exceed the number of categories of the multiple types of risk data; The plurality of first risk characteristics are determined as the first risk characteristics of the target object.

2. The method according to claim 1, characterized in that, The process of integrating the first risk feature and the third risk feature according to the first standard to obtain the integrated risk feature includes: If the first risk characteristic and the third risk characteristic of the target object are the same, then either the first risk characteristic or the third risk characteristic is deleted to obtain the integrated risk characteristic; If the first risk characteristic and the third risk characteristic of the target object are different, then the first risk characteristic and the third risk characteristic are combined to obtain the integrated risk characteristic.

3. The method according to claim 1, characterized in that, The target object includes multiple objects, and before migrating the first risk feature to the target database, it also includes: The object to be verified is determined from the target object; For each object to be verified, the accuracy of the integration of the first risk features of the target object is verified based on the number of categories of the first risk data of the object to be verified and the number of features of the first risk features of the object to be verified. If the number of categories is equal to the number of features, or if the number of categories is not equal to the number of features but the integration result of the risk features corresponding to each category of the first risk data of the object to be verified meets the first standard, then the integration of the first risk features of the object to be verified is determined to be accurate. If the first risk characteristics of all objects to be verified are integrated accurately, then the verification of integration accuracy is successful. If the accuracy verification is successful, the first risk feature will be migrated to the target database.

4. The method according to claim 3, characterized in that, After performing accuracy verification on the first risk features of the target object based on the number of categories of the first risk data of the object to be verified and the number of features of the first risk features of the object to be verified, the method further includes: If the number of categories is not equal to the number of features and the integration result of the risk features corresponding to each category of the first risk data of the object to be verified does not meet the first standard, then the difference features between the integration result and the standard result corresponding to the object to be verified are analyzed according to the first standard. Correct the first risk feature of the target object that has the difference feature based on the difference feature.

5. The method according to claim 1, characterized in that, Before integrating the risk characteristics corresponding to each type of first risk data according to the first standard to obtain multiple first risk characteristics, the method further includes: The accuracy of the calculation of risk characteristics corresponding to each type of first risk data is verified based on the first risk data and the first standard. If the accuracy of the calculation of the risk features corresponding to the first risk data in all categories is successfully verified, the risk features corresponding to each category of first risk data are integrated according to the first standard to obtain multiple first risk features.

6. The method according to any one of claims 1-5, characterized in that, Before determining the first risk characteristic of the target object based on the first risk data and the first standard, the method further includes: The integrity and consistency of the first risk data of the target object in the intermediate database are verified based on the first risk data of the target object in the source database. If the integrity and consistency verification is successful, the first risk characteristic of the target object is determined based on the first risk data and the first standard.

7. A risk data processing device, characterized in that, The device includes: The acquisition module is used to retrieve the first risk data and the second risk data of the target object from the source database; A first determining module is used to migrate the first risk data to an intermediate database. In the intermediate database, based on the first risk data and a first standard, a first risk characteristic of the target object is determined, and the first risk characteristic is migrated to the target database. The intermediate database is a database of the Hadoop framework. Both the source database and the target database are Oracle databases. The second determining module is used to migrate the second risk data to the target database, and determine the second risk characteristics of the target object in the target database based on the second risk data and the second standard. The conversion module is used to convert the second risk feature of the target object into a third risk feature of the target object based on the mapping relationship between the risk features under the first standard and the risk features under the second standard; the third risk feature is the risk feature under the first standard. An integration module is used to integrate the first risk feature and the third risk feature according to the first standard to obtain an integrated risk feature; the integrated risk feature is used to assess the risk level of the target object; The first risk data includes multiple types of risk data; the first determining module is specifically used to determine the risk characteristics corresponding to each type of first risk data based on the first standard and each type of first risk data. Based on the first standard, the risk characteristics corresponding to each type of first risk data are integrated to obtain multiple first risk characteristics; wherein, the number of features of the multiple first risk characteristics does not exceed the number of categories of the multiple types of risk data; The plurality of first risk characteristics are determined as the first risk characteristics of the target object.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images