Credit investigation variable derivation method and device based on double paths

Abstract

The invention discloses a credit investigation variable derivation method and device based on double paths, and belongs to the field of credit investigation variable derivation, and the method comprises the steps: obtaining each to-be-derived credit investigation variable and the corresponding JAVA structured data and derivative variable configuration table, and classifying the to-be-derived variables into JAVA derivative variables and SQL derivative variables according to the configuration table; and processing the JAVA derivative variable through a corresponding JAVA code to obtain a result, obtaining an SQL derivative variable result from an Oracle transaction level temporary table of the optimal database subjected to health degree screening according to the corresponding SQL code, and finally combining the two types of results to output a final derivative result. Therefore, by implementing the method and the device, the problem that it is difficult to ensure that the credit investigation variable derivation result is generated in time when the credit investigation data volume reaches the ten million level in the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of credit variable derivation, and in particular to a method and apparatus for credit variable derivation based on a dual-path approach. Background Technology

[0002] In the field of credit reporting variable derivation, credit reporting derived variables refer to feature variables generated by calculation and processing based on the original fields of the second-generation credit reporting report. Their value has broken through the limitations of the simple "have / have not" record or basic overdue statistics in the first-generation credit reporting period, and has become the core input for pre-loan / post-loan risk control decisions in personal consumer finance.

[0003] The commonly used technical solution in the current field is to obtain the second-generation credit report, convert it into a JavaBean object using XStream technology, store the structured data in a database, and finally execute the variable derivation logic based on SQL statements. Because this solution heavily relies on the database to perform complex SQL calculations and requires full storage of the structured credit report data, the following problems exist: ① When the amount of credit data reaches tens of millions and the number of variables exceeds thousands, the SQL calculation time increases significantly, leading to delays in risk control approval; ② When concurrent requests exceed the database processing threshold, resource contention causes the derivation time to rise non-linearly; ③ Due to limitations in SQL function capabilities and optimization bottlenecks, the execution time for some complex logical variables can reach seconds. Therefore, there is an urgent need for a technical solution that ensures the timely generation of credit variable derivation results when the amount of credit data reaches tens of millions. Summary of the Invention

[0004] This invention provides a dual-path-based method and apparatus for credit variable derivation, which can solve the problem in the prior art of ensuring timely generation of credit variable derivation results when the amount of credit data reaches tens of millions.

[0005] In a first aspect, embodiments of the present invention provide a method for deriving credit variables based on a dual-path approach, comprising:

[0006] Obtain each credit information variable to be derived and the corresponding JAVA structured data for each of the credit information variables to be derived, and obtain the derived variable configuration table;

[0007] Based on the derived variable configuration table, the various credit information variables to be derived are classified to obtain various JAVA derived variables and various SQL derived variables;

[0008] According to the derived variable configuration table, obtain the JAVA code corresponding to each JAVA derived variable, and process the JAVA structured data corresponding to each JAVA derived variable through each JAVA code to obtain the derived result corresponding to each JAVA derived variable.

[0009] The SQL code corresponding to each SQL derived variable is obtained according to the derived variable configuration table, and the derived result corresponding to each SQL derived variable is obtained from the Oracle transaction-level temporary table of the optimal database according to the SQL code corresponding to each SQL derived variable; wherein, the optimal database is selected from a preset executable database according to the health screening method, and the Oracle transaction-level temporary table is obtained according to each SQL derived variable;

[0010] The derivation results corresponding to each of the JAVA derivation variables and the derivation results corresponding to each of the SQL derivation variables are merged to output the final derivation result.

[0011] This application's embodiments categorize variables to be derived into JAVA processing paths and SQL processing paths through a derived variable configuration table. This allows complex logical variables to escape database computational bottlenecks, while simple variables retain the advantages of batch SQL processing. This enables targeted optimization of the derived paths, releasing the derived capabilities of complex variables through the JAVA path and preserving the batch processing efficiency of the SQL path. Together, they cover a wider range of derived scenarios. Secondly, for JAVA path variables, code logic is executed directly based on in-memory structured objects, avoiding database interaction overhead and SQL function limitations, ensuring the efficient and stable execution of complex derived tasks. Simultaneously, for SQL path variables, the optimal database node is dynamically selected, and a minimal data processing environment is constructed on demand using Oracle transaction-level temporary tables, significantly reducing the scale of data operations and isolating concurrent transaction interference. Therefore, this application's embodiments can solve the problem in the prior art of ensuring timely generation of credit variable derived results when the amount of credit data reaches tens of millions.

[0012] As a preferred example of the first aspect, the step of classifying the various credit scoring variables to be derived according to the derived variable configuration table to obtain various JAVA derived variables and various SQL derived variables specifically involves:

[0013] Iterate through each credit information variable to be derived, and query the derived variable configuration table to find the derived method flag corresponding to the currently iterated credit information variable to be derived;

[0014] If the derivation method corresponding to the credit variable to be derived in the current iteration is marked as JAVA, then the credit variable to be derived in the current iteration will be classified as a JAVA derived variable.

[0015] If the derivation method corresponding to the credit variable to be derived currently being iterated is marked as SQL, then the credit variable to be derived currently being iterated will be classified as an SQL derived variable.

[0016] In this preferred example, by traversing the credit scoring variables to be derived and classifying them according to the derivation method, variables with different characteristics can be matched with the most suitable derivation method: variables suitable for processing with JAVA code can be efficiently derived using JAVA, and variables suitable for processing with SQL can be executed in an orderly manner using SQL. This avoids efficiency loss caused by mismatch between the derivation method and the characteristics of the variables, giving full play to the advantages of JAVA in processing complex logic, and making reasonable use of the applicable scenarios of SQL, effectively optimizing the overall process of variable derivation, and improving the efficiency and flexibility of real-time credit scoring variable derivation.

[0017] As a preferred example of the first aspect, the optimal database is selected from a preset executable database according to a health screening method, specifically:

[0018] Select available databases from the preset executable databases and use all available databases as the first database set;

[0019] Based on the preset validity time and the update time of each database in the first database set, the first database set is filtered by time to obtain the second database set;

[0020] A weighted comprehensive scoring method is used to calculate the health score of each database in the second database set, and the database with the highest health score is selected as the optimal database.

[0021] In this preferred example, by filtering available databases from a preset executable database and obtaining an effective set through time filtering, and then selecting the optimal database with the highest health through a weighted comprehensive scoring method, this process ensures that the databases involved in variable derivation have basic availability and timely status. At the same time, it allows variable derivation tasks to be assigned to databases with better performance and status, effectively avoiding processing bottlenecks caused by insufficient database resources or poor status, reducing derivation time, and improving the overall efficiency and stability of credit variable derivation. Especially in concurrent scenarios, it can more reasonably allocate database resources and improve the efficiency of credit variable derivation.

[0022] As a preferred example of the first aspect, the Oracle transaction-level temporary table is obtained based on the JAVA structured data corresponding to each of the SQL derived variables, specifically:

[0023] Based on the SQL code corresponding to each of the SQL derived variables, the Apache Calcite technology is used to parse and obtain the set of related tables;

[0024] Locate the JAVA structured data corresponding to the set of related tables from the JAVA structured data corresponding to each of the SQL derived variables, and insert all the JAVA structured data into the temporary table of the optimal database to obtain the Oracle transaction-level temporary table.

[0025] In this preferred example, the set of related tables is obtained by parsing the SQL code of the SQL derived variables using Apache Calcite technology. This allows for precise location of the tables corresponding to the required data, avoiding processing of irrelevant tables. Then, the data corresponding to the related tables is extracted from the Java structured data and inserted into the Oracle transaction-level temporary table of the optimal database. This reduces unnecessary data operations and the amount of data processing. Furthermore, leveraging the characteristics of transaction-level temporary tables reduces the consumption of physical table resources. Combined with the optimal database's high performance, this effectively improves the efficiency of data insertion and subsequent variable derivation, optimizes the overall processing flow, and enhances the smoothness of real-time credit variable derivation.

[0026] As a preferred example of the first aspect, the acquisition of each credit variable to be derived and the corresponding JAVA structured data of each credit variable to be derived specifically includes:

[0027] Obtain the credit report and convert it into a JavaBean structured object using XStream technology;

[0028] Extract data associated with each of the proposed credit variables from the JavaBean structured object, and generate JAVA structured data corresponding to each of the proposed credit variables.

[0029] In this preferred example, after obtaining the credit report, it is converted into a JavaBean structured object using XStream technology. This transforms unstructured credit report data into a structured form that is easy for programs to manipulate, providing a standardized and clear data foundation for subsequent variable derivation. Then, the data associated with the credit variables to be derived is accurately extracted from this JavaBean structured object, and the corresponding JAVA structured data is generated. This avoids processing irrelevant data, reduces data redundancy, and allows subsequent variable derivation, whether using JAVA code or SQL, to be directly based on the required accurate data. This simplifies the data processing flow and improves the efficiency of pre-derivation data preparation.

[0030] Secondly, the present invention provides a credit variable derivation device based on dual paths, comprising: a data acquisition module, a data classification module, a first derivation module, a second derivation module, and a result output module;

[0031] The data acquisition module is used to acquire each credit variable to be derived and the corresponding JAVA structured data of each credit variable to be derived, and to acquire the derived variable configuration table;

[0032] The data classification module is used to classify each credit information variable to be derived according to the derived variable configuration table to obtain each JAVA derived variable and each SQL derived variable;

[0033] The first derivation module is used to obtain the JAVA code corresponding to each JAVA derivation variable according to the derivation variable configuration table, and process the JAVA structured data corresponding to each JAVA derivation variable through each JAVA code to obtain the derivation result corresponding to each JAVA derivation variable;

[0034] The second derivation module is used to obtain the SQL code corresponding to each of the SQL derivation variables according to the derivation variable configuration table, and to obtain the derivation result corresponding to each of the SQL derivation variables from the Oracle transaction-level temporary table of the optimal database according to the SQL code corresponding to each of the SQL derivation variables; wherein, the optimal database is selected from a preset executable database according to the health screening method, and the Oracle transaction-level temporary table is obtained according to each of the SQL derivation variables;

[0035] The result output module is used to merge the derived results corresponding to each of the JAVA derived variables and the derived results corresponding to each of the SQL derived variables, and output the final derived result.

[0036] As a preferred example of the second aspect, the data acquisition module includes a first acquisition unit, a second acquisition unit, and a third acquisition unit;

[0037] The first acquisition unit is used to traverse each credit information variable to be derived and query the derivation method tag corresponding to the currently traversed credit information variable to be derived from the derivation variable configuration table;

[0038] The second acquisition unit is used to classify the currently traversed credit information variable to be derived into a JAVA derived variable if the derivation method corresponding to the currently traversed credit information variable is marked as JAVA.

[0039] The third acquisition unit is used to classify the currently traversed credit information variable into an SQL derived variable if the derivation method corresponding to the currently traversed credit information variable is marked as SQL.

[0040] As a preferred example of the second aspect, the second derivative module includes a first derivative unit, a second derivative unit, and a third derivative unit;

[0041] The first derivative unit is used to filter out available databases from a preset executable database and to set all available databases as a first database set;

[0042] The second derivative unit is used to perform time filtering on the first database set according to the preset valid time and the update time of each database in the first database set to obtain a second database set;

[0043] The third derivative unit is used to calculate the health score of each database in the second database set using a weighted comprehensive scoring method, and to select the database with the highest health score as the optimal database.

[0044] As a preferred example of the second aspect, the second derivative module further includes a fourth derivative unit and a fifth derivative unit;

[0045] The fourth derivative unit is used to parse the SQL code corresponding to each SQL derivative variable using Apache Calcite technology to obtain a set of related tables.

[0046] The fifth derivative unit is used to locate the JAVA structured data corresponding to the set of associated tables from the JAVA structured data corresponding to each of the SQL derived variables, and insert all the JAVA structured data into the temporary table of the optimal database to obtain the Oracle transaction-level temporary table.

[0047] As a preferred example of the second aspect, the data acquisition module further includes a fourth acquisition unit and a fifth acquisition unit;

[0048] The fourth acquisition unit is used to acquire the credit report and convert the credit report into a JavaBean structured object based on XStream technology;

[0049] The fifth acquisition unit is used to extract data associated with each of the credit variables to be derived from the JavaBean structured object, and generate JAVA structured data corresponding to each of the credit variables to be derived.

[0050] In summary, this application's embodiments categorize variables to be derived into JAVA processing paths and SQL processing paths through a derived variable configuration table. This allows complex logical variables to escape database computation bottlenecks, while simple variables retain the advantages of batch SQL processing. This enables targeted optimization of the derived paths, releasing the derived capabilities of complex variables through the JAVA path and preserving the batch processing efficiency of the SQL path. Together, they cover a wider range of derived scenarios. Secondly, for JAVA path variables, code logic is executed directly based on in-memory structured objects, avoiding database interaction overhead and SQL function limitations, ensuring the efficient and stable execution of complex derived tasks. Simultaneously, for SQL path variables, the optimal database node is dynamically selected, and a minimal data processing environment is constructed on demand using Oracle transaction-level temporary tables, significantly reducing the scale of data operations and isolating concurrent transaction interference. Therefore, this application's embodiments can solve the problem in the prior art of ensuring timely generation of credit variable derived results when the amount of credit data reaches tens of millions.

[0051] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the dual-path credit variable derivation method of the present invention.

[0052] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the dual-path-based credit variable derivation method of the present invention. Attached Figure Description

[0053] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating an embodiment of a credit variable derivation method based on a dual-path approach provided by the present invention.

[0055] Figure 2 This is a schematic diagram of a structured object representing an embodiment of a dual-path credit variable derivation method provided by the present invention.

[0056] Figure 3 The computational logic diagram of the optimal database for an embodiment of a dual-path credit variable derivation method provided by the present invention;

[0057] Figure 4The flowchart shows an embodiment of a credit variable derivation method based on dual paths provided by the present invention.

[0058] Figure 5 This is a module structure diagram of one embodiment of a credit variable derivation device based on dual paths provided by the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0061] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0062] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0063] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0064] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0065] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0066] Example 1

[0067] See Figure 1 To address the problem in existing technologies of ensuring timely generation of credit variable derivation results when the amount of credit data reaches tens of millions, an embodiment of the present invention provides a credit variable derivation method based on a dual-path approach, comprising:

[0068] S1. Obtain each credit variable to be derived and the corresponding JAVA structured data for each credit variable to be derived, and obtain the derived variable configuration table.

[0069] In some embodiments of this application, obtaining each credit variable to be derived and the corresponding JAVA structured data specifically involves:

[0070] Obtain the credit report and convert it into a JavaBean structured object using XStream technology;

[0071] Extract data associated with each of the proposed credit variables from the JavaBean structured object, and generate JAVA structured data corresponding to each of the proposed credit variables.

[0072] In some optional embodiments of this application, the preferred format of the credit report is XML format.

[0073] For example, the exemplary format of the JAVA structured data may be as follows: Figure 2 As shown, the information includes basic identity information, mobile phone number information, and spouse information.

[0074] S2. Classify the credit information variables to be derived according to the derived variable configuration table to obtain each JAVA derived variable and each SQL derived variable.

[0075] In some embodiments of this application, the step of classifying the various credit information variables to be derived according to the derived variable configuration table to obtain various JAVA derived variables and various SQL derived variables specifically involves:

[0076] Iterate through each credit information variable to be derived, and query the derived variable configuration table to find the derived method flag corresponding to the currently iterated credit information variable to be derived;

[0077] If the derivation method corresponding to the credit variable to be derived in the current iteration is marked as JAVA, then the credit variable to be derived in the current iteration will be classified as a JAVA derived variable.

[0078] If the derivation method corresponding to the credit variable to be derived currently being iterated is marked as SQL, then the credit variable to be derived currently being iterated will be classified as an SQL derived variable.

[0079] Specifically, the variable configuration information in the derived variable configuration table can be obtained through the following preferred methods:

[0080] ①Strategy risk control analysts use structured credit data from offline data warehouses to obtain offline credit derived variables using methods such as hierarchical derivation and multi-dimensional combination derivation;

[0081] ② The strategy risk control personnel evaluate the effect of the variables derived from step ① using real data. If the effect is appropriate and suitable for real-time risk control approval, then the variable is used as a real-time credit scoring derived variable.

[0082] ③ Use SQL to implement the derivation logic of the real-time derived variable obtained in step ②, and confirm the derivation efficiency of the SQL. If the efficiency is too low (e.g., the SQL execution time is >100ms) or the derivation logic of the variable is too complex to write a suitable SQL, then provide the variable as a JAVA-based derived variable for development requirements. Otherwise, configure the variable information of the derived variable and the SQL of the derivation logic in the variable configuration.

[0083] ④ After receiving the requirements, the developers will develop the code using JavaBean structured objects based on the logic of the derived variables. For example, the derived variable "maximum number of queries per day in the past 3 months (max_querycnt_1day_3m)" will be implemented using a List. <queryrecord>The data in queryRecords is filtered to extract detailed query records for the past 90 days. Then, the number of queries for each query date is counted, and the maximum number of queries is taken. After the variables are developed, they are configured in the variable configuration table.

[0084] S3. Obtain the JAVA code corresponding to each JAVA derived variable according to the derived variable configuration table, and process the JAVA structured data corresponding to each JAVA derived variable through each JAVA code to obtain the derived result corresponding to each JAVA derived variable.

[0085] For example, the JAVA derived variable is "maximum number of queries per day in the past 3 months (max_querycnt_1day_3m)", the input is a JavaBean YhzxRpt object that has been structured based on the second-generation credit report (xml format), and the output is {"max_querycnt_1day_3m":4}.

[0086] S4. Obtain the SQL code corresponding to each of the SQL derived variables according to the derived variable configuration table, and obtain the derived result corresponding to each of the SQL derived variables from the Oracle transaction-level temporary table of the optimal database according to the SQL code corresponding to each of the SQL derived variables; wherein, the optimal database is selected from a preset executable database according to the health screening method, and the Oracle transaction-level temporary table is obtained according to each of the SQL derived variables.

[0087] In some embodiments of this application, the optimal database is selected from a preset executable database according to a health screening method, specifically:

[0088] Select available databases from the preset executable databases and use all available databases as the first database set;

[0089] Based on the preset validity time and the update time of each database in the first database set, the first database set is filtered by time to obtain the second database set;

[0090] A weighted comprehensive scoring method is used to calculate the health score of each database in the second database set, and the database with the highest health score is selected as the optimal database.

[0091] Specifically, the optimal database is selected from a preset executable database based on a health screening method, and can be implemented in the following ways:

[0092] like Figure 3 As shown, firstly, all executable databases are extracted from the preset executable database information table to generate an initial list ListA. If ListA is empty, an exception "No available executable database" is thrown, terminating the filtering process; if ListA is not empty, proceed to the next step. Based on the "valid time" (e.g., time interval, in minutes) of the configuration item and the current time, a filtering timeline is calculated (e.g., if the valid time is 5 minutes and the current time is 18:10, then the filtering timeline is 18:05); databases whose update time is later than this timeline are filtered from ListA to form a list ListB. If ListB is empty, a random allocation algorithm is used to randomly select one database from ListA as a candidate executable database; if ListB is not empty, proceed to the next step. Databases with a health value higher than the "health threshold" of the configuration item are filtered from ListB to form a list ListC. If ListC is empty, a random allocation algorithm is used to randomly select one database from ListB as a candidate executable database; if ListC is not empty, the database with the highest health value in ListC is selected as the optimal database. Among them, the health status is based on real-time collected information such as database CPU utilization, memory utilization, and I / O wait time. It uses Flink real-time computing technology and a weighted comprehensive scoring method to calculate the database health status and update it to the database.

[0093] In some embodiments of this application, the Oracle transaction-level temporary table is obtained based on the JAVA structured data corresponding to each of the SQL derived variables, specifically:

[0094] Based on the SQL code corresponding to each of the SQL derived variables, the Apache Calcite technology is used to parse and obtain the set of related tables;

[0095] Locate the JAVA structured data corresponding to the set of related tables from the JAVA structured data corresponding to each of the SQL derived variables, and insert all the JAVA structured data into the temporary table of the optimal database to obtain the Oracle transaction-level temporary table.

[0096] It's important to note that an Oracle Transaction-Level Temporary Table (TPTable) is a special table type in Oracle databases used to temporarily store data that is only valid within the lifecycle of the current transaction. Its core mechanism is that data is automatically cleared when the transaction is committed or rolled back. It is suitable for scenarios requiring isolation of intermediate transaction results, and its core features include automatic data cleanup, performance advantages, and support for database objects.

[0097] For example, the SQL codes corresponding to each of the SQL derived variables include SQL1 and SQL2. SQL1 involves tables T1 and T2, and SQL2 involves tables T2 and T3. The required tables are T1, T2, and T3. In order to reduce the data entry time, the batch insert technology is used in the selected database to execute the Insert statement to insert the structured data in the JavaBean into the temporary table, thus obtaining the Oracle transaction-level temporary table.

[0098] For example, when deriving the variable "number of credit inquiries within 3 months: qqr_90d_cc_cnt", an SQL statement is used for the derivation. This SQL statement uses the table query_record_info, whose corresponding JavaBean structured object is a List. <queryrecord>Therefore, this step involves inserting the data from queryRecords into the value table query_record_info. Compared to the original approach (inserting data from all tables), this approach reduces the number of table operations.

[0099] S5. Merge the derivative results corresponding to each JAVA derivative variable and the derivative results corresponding to each SQL derivative variable, and output the final derivative result.

[0100] Specifically, to fully explain the above process, the following scheme will be used as an example:

[0101] like Figure 4 As shown, after the business system initiates a credit variable derivation request, the real-time credit variable derivation system first obtains the credit report and performs structured parsing; then, it distributes the data according to the type of derived variable: if it is a JAVA derivation method, the variable is directly derived based on the JAVA code; if it is an SQL derivation method, it first selects the variable derivation database (during this process, the second-generation variable derivation database collects core performance data and sends it to the real-time data warehouse, which calculates the database health and stores the results in a table), then incorporates the structured credit report into a temporary table, and then executes the SQL derivation variable based on the temporary table; finally, the system integrates the results, returns the processed credit derivation variables, and achieves data interaction with the central bank's credit reference center, completing the business loop.

[0102] In summary, this application's embodiments categorize variables to be derived into JAVA processing paths and SQL processing paths through a derived variable configuration table. This allows complex logical variables to escape database computation bottlenecks, while simple variables retain the advantages of batch SQL processing. This enables targeted optimization of the derived paths, releasing the derived capabilities of complex variables through the JAVA path and preserving the batch processing efficiency of the SQL path. Together, they cover a wider range of derived scenarios. Secondly, for JAVA path variables, code logic is executed directly based on in-memory structured objects, avoiding database interaction overhead and SQL function limitations, ensuring the efficient and stable execution of complex derived tasks. Simultaneously, for SQL path variables, the optimal database node is dynamically selected, and a minimal data processing environment is constructed on demand using Oracle transaction-level temporary tables, significantly reducing the scale of data operations and isolating concurrent transaction interference. Therefore, this application's embodiments can solve the problem in the prior art of ensuring timely generation of credit variable derived results when the amount of credit data reaches tens of millions.

[0103] Example 2

[0104] like Figure 5 As shown, based on the above method embodiments, corresponding device embodiments are provided;

[0105] An embodiment of the present invention provides a credit variable derivation device based on dual paths, including: a data acquisition module 51, a data classification module 52, a first derivation module 53, a second derivation module 54, and a result output module 55;

[0106] The data acquisition module 51 is used to acquire each credit variable to be derived and the corresponding JAVA structured data of each credit variable to be derived, and to acquire the derived variable configuration table;

[0107] Data classification module 52 is used to classify each credit information variable to be derived according to the derived variable configuration table to obtain each JAVA derived variable and each SQL derived variable;

[0108] The first derivative module 53 is used to obtain the JAVA code corresponding to each JAVA derivative variable according to the derivative variable configuration table, and process the JAVA structured data corresponding to each JAVA derivative variable through each JAVA code to obtain the derivative result corresponding to each JAVA derivative variable.

[0109] The second derivative module 54 is used to obtain the SQL code corresponding to each SQL derivative variable according to the derivative variable configuration table, and to obtain the derivative result corresponding to each SQL derivative variable from the Oracle transaction-level temporary table of the optimal database according to the SQL code corresponding to each SQL derivative variable; wherein, the optimal database is selected from a preset executable database according to a health screening method.

[0110] Oracle transaction-level temporary tables are obtained based on the SQL derived variables described above;

[0111] The result output module 55 is used to merge the derived results corresponding to each of the JAVA derived variables and the derived results corresponding to each of the SQL derived variables, and output the final derived result.

[0112] In some embodiments of this application, the data acquisition module 51 includes a first acquisition unit, a second acquisition unit, and a third acquisition unit;

[0113] The first acquisition unit is used to traverse each credit information variable to be derived and query the derivation method tag corresponding to the currently traversed credit information variable to be derived from the derivation variable configuration table;

[0114] The second acquisition unit is used to classify the currently traversed credit information variable to be derived into a JAVA derived variable if the derivation method corresponding to the currently traversed credit information variable is marked as JAVA.

[0115] The third acquisition unit is used to classify the currently traversed credit information variable into an SQL derived variable if the derivation method corresponding to the currently traversed credit information variable is marked as SQL.

[0116] In some embodiments of this application, the second derivative module 54 includes a first derivative unit, a second derivative unit, and a third derivative unit;

[0117] The first derivative unit is used to filter out available databases from a preset executable database and to set all available databases as a first database set;

[0118] The second derivative unit is used to perform time filtering on the first database set according to the preset valid time and the update time of each database in the first database set to obtain a second database set;

[0119] The third derivative unit is used to calculate the health score of each database in the second database set using a weighted comprehensive scoring method, and to select the database with the highest health score as the optimal database.

[0120] In some embodiments of this application, the second derivative module 54 further includes a fourth derivative unit and a fifth derivative unit;

[0121] The fourth derivative unit is used to parse the SQL code corresponding to each SQL derivative variable using Apache Calcite technology to obtain a set of related tables.

[0122] The fifth derivative unit is used to locate the JAVA structured data corresponding to the set of associated tables from the JAVA structured data corresponding to each of the SQL derived variables, and insert all the JAVA structured data into the temporary table of the optimal database to obtain the Oracle transaction-level temporary table.

[0123] In some embodiments of this application, the data acquisition module 51 further includes a fourth acquisition unit and a fifth acquisition unit;

[0124] The fourth acquisition unit is used to acquire the credit report and convert the credit report into a JavaBean structured object based on XStream technology;

[0125] The fifth acquisition unit is used to extract data associated with each of the credit variables to be derived from the JavaBean structured object, and generate JAVA structured data corresponding to each of the credit variables to be derived.

[0126] For more detailed steps and working principles of this embodiment, please refer to the relevant description in Embodiment 1, but not limited to these descriptions.

[0127] In summary, this application's embodiments categorize variables to be derived into JAVA processing paths and SQL processing paths through a derived variable configuration table. This allows complex logical variables to escape database computation bottlenecks, while simple variables retain the advantages of batch SQL processing. This enables targeted optimization of the derived paths, releasing the derived capabilities of complex variables through the JAVA path and preserving the batch processing efficiency of the SQL path. Together, they cover a wider range of derived scenarios. Secondly, for JAVA path variables, code logic is executed directly based on in-memory structured objects, avoiding database interaction overhead and SQL function limitations, ensuring the efficient and stable execution of complex derived tasks. Simultaneously, for SQL path variables, the optimal database node is dynamically selected, and a minimal data processing environment is constructed on demand using Oracle transaction-level temporary tables, significantly reducing the scale of data operations and isolating concurrent transaction interference. Therefore, this application's embodiments can solve the problem in the prior art of ensuring timely generation of credit variable derived results when the amount of credit data reaches tens of millions.

[0128] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the dual-path credit variable derivation method provided by any of the above-described method embodiments of the present invention.

[0129] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0130] Based on the above embodiments of the dual-path credit variable derivation method, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the dual-path credit variable derivation method of any embodiment of the present invention.

[0131] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0132] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0133] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0134] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the dual-path-based credit variable derivation method described in any of the above-described method embodiments of the present invention.

[0135] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0136] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.< / queryrecord> < / queryrecord>

Claims

1. A method for deriving credit variables based on a dual-path approach, characterized in that, include: Obtain each credit information variable to be derived and the corresponding JAVA structured data for each of the credit information variables to be derived, and obtain the derived variable configuration table; Based on the derived variable configuration table, the various credit information variables to be derived are classified to obtain various JAVA derived variables and various SQL derived variables; According to the derived variable configuration table, obtain the JAVA code corresponding to each JAVA derived variable, and process the JAVA structured data corresponding to each JAVA derived variable through each JAVA code to obtain the derived result corresponding to each JAVA derived variable. The SQL code corresponding to each SQL derived variable is obtained according to the derived variable configuration table, and the derived result corresponding to each SQL derived variable is obtained from the Oracle transaction-level temporary table of the optimal database according to the SQL code corresponding to each SQL derived variable; wherein, the optimal database is selected from a preset executable database according to the health screening method, and the Oracle transaction-level temporary table is obtained according to each SQL derived variable; The derivation results corresponding to each of the JAVA derivation variables and the derivation results corresponding to each of the SQL derivation variables are merged to output the final derivation result.

2. The credit variable derivation method based on a dual-path approach as described in claim 1, characterized in that, The step of classifying the credit information variables to be derived according to the derived variable configuration table to obtain each JAVA derived variable and each SQL derived variable is as follows: Iterate through each credit information variable to be derived, and query the derived variable configuration table to find the derived method flag corresponding to the currently iterated credit information variable to be derived; If the derivation method corresponding to the credit variable to be derived in the current iteration is marked as JAVA, then the credit variable to be derived in the current iteration will be classified as a JAVA derived variable. If the derivation method corresponding to the credit variable to be derived currently being iterated is marked as SQL, then the credit variable to be derived currently being iterated will be classified as an SQL derived variable.

3. The credit variable derivation method based on a dual-path approach as described in claim 1, characterized in that, The optimal database is selected from a preset executable database based on a health-based screening method, specifically as follows: Select available databases from the preset executable databases and use all available databases as the first database set; Based on the preset validity time and the update time of each database in the first database set, the first database set is filtered by time to obtain the second database set; A weighted comprehensive scoring method is used to calculate the health score of each database in the second database set, and the database with the highest health score is selected as the optimal database.

4. The credit variable derivation method based on a dual-path approach as described in claim 1, characterized in that, The Oracle transaction-level temporary table is obtained based on the Java structured data corresponding to each of the SQL derived variables, specifically: Based on the SQL code corresponding to each of the SQL derived variables, the Apache Calcite technology is used to parse and obtain the set of related tables; Locate the JAVA structured data corresponding to the set of related tables from the JAVA structured data corresponding to each of the SQL derived variables, and insert all the JAVA structured data into the temporary table of the optimal database to obtain the Oracle transaction-level temporary table.

5. The credit variable derivation method based on a dual-path approach as described in claim 1, characterized in that, The specific steps for obtaining each credit variable to be derived and the corresponding JAVA structured data are as follows: Obtain the credit report and convert it into a JavaBean structured object using XStream technology; Extract data associated with each of the proposed credit variables from the JavaBean structured object, and generate JAVA structured data corresponding to each of the proposed credit variables.

6. A credit variable derivation device based on dual paths, characterized in that, include: The system includes a data acquisition module, a data classification module, a first derivative module, a second derivative module, and a result output module. The data acquisition module is used to acquire each credit variable to be derived and the corresponding JAVA structured data of each credit variable to be derived, and to acquire the derived variable configuration table; The data classification module is used to classify each credit information variable to be derived according to the derived variable configuration table to obtain each JAVA derived variable and each SQL derived variable; The first derivation module is used to obtain the JAVA code corresponding to each JAVA derivation variable according to the derivation variable configuration table, and process the JAVA structured data corresponding to each JAVA derivation variable through each JAVA code to obtain the derivation result corresponding to each JAVA derivation variable. The second derivation module is used to obtain the SQL code corresponding to each of the SQL derivation variables according to the derivation variable configuration table, and to obtain the derivation result corresponding to each of the SQL derivation variables from the Oracle transaction-level temporary table of the optimal database according to the SQL code corresponding to each of the SQL derivation variables; wherein, the optimal database is selected from a preset executable database according to the health screening method, and the Oracle transaction-level temporary table is obtained according to each of the SQL derivation variables; The result output module is used to merge the derived results corresponding to each of the JAVA derived variables and the derived results corresponding to each of the SQL derived variables, and output the final derived result.

7. The credit variable derivation device based on dual paths as described in claim 6, characterized in that, The data acquisition module includes a first acquisition unit, a second acquisition unit, and a third acquisition unit; The first acquisition unit is used to traverse each credit information variable to be derived and query the derivation method tag corresponding to the currently traversed credit information variable to be derived from the derivation variable configuration table; The second acquisition unit is used to classify the currently traversed credit information variable to be derived into a JAVA derived variable if the derivation method corresponding to the currently traversed credit information variable is marked as JAVA. The third acquisition unit is used to classify the currently traversed credit information variable into an SQL derived variable if the derivation method corresponding to the currently traversed credit information variable is marked as SQL.

8. The credit variable derivation device based on dual paths as described in claim 6, characterized in that, The second derivative module includes a first derivative unit, a second derivative unit, and a third derivative unit; The first derivative unit is used to filter out available databases from a preset executable database and to set all available databases as a first database set; The second derivative unit is used to perform time filtering on the first database set according to the preset valid time and the update time of each database in the first database set to obtain a second database set; The third derivative unit is used to calculate the health score of each database in the second database set using a weighted comprehensive scoring method, and to select the database with the highest health score as the optimal database.

9. A credit variable derivation device based on dual paths as described in claim 6, characterized in that, The second derivative module also includes a fourth derivative unit and a fifth derivative unit; The fourth derivative unit is used to parse the SQL code corresponding to each SQL derivative variable using Apache Calcite technology to obtain a set of related tables. The fifth derivative unit is used to locate the JAVA structured data corresponding to the set of associated tables from the JAVA structured data corresponding to each of the SQL derived variables, and insert all the JAVA structured data into the temporary table of the optimal database to obtain the Oracle transaction-level temporary table.

10. A credit variable derivation device based on dual paths as described in claim 6, characterized in that, The data acquisition module further includes a fourth acquisition unit and a fifth acquisition unit; The fourth acquisition unit is used to acquire the credit report and convert the credit report into a JavaBean structured object based on XStream technology; The fifth acquisition unit is used to extract data associated with each of the credit variables to be derived from the JavaBean structured object, and generate JAVA structured data corresponding to each of the credit variables to be derived.

Images