Credit data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of data processing, and discloses a credit data processing method, device and equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed operator generated in a credit data processing process, at least one first intermediate operator which the to-be-processed operator directly depends on, and a first intermediate parameter corresponding to each first intermediate operator; the operator to be processed comprises data, computational logic and a first operator dependency relationship; determining a second operator dependency relationship corresponding to each first intermediate operator, and determining a first operator result based on the second operator dependency relationship, the first intermediate operators and the corresponding first intermediate parameters; the second operator dependency relationship indicates at least one second intermediate operator depended by the first intermediate operator to obtain the corresponding operator result and the execution sequence of the second intermediate operators; and determining a loan prediction result based on the at least one first operator result, the data and the computational logic. Computing logic is packaged in an operator, and the correctness of a debit and credit prediction result is ensured through ordered calculation.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a credit data processing method, apparatus, device and storage medium. Background Art

[0002] In the field of data processing technology, especially in credit scenarios, when providing predictions about loan information provided by loan-assisting institutions to users, repayment information provided by loan-assisting institutions to investors, and loan fund plans generated by loan-assisting institutions with the assistance of users and investors, it is often necessary to calculate large amounts of data, such as the loan-assisting institutions' historical loan information, repayment information, customer attribution, and order-related data. In existing technologies, computing and processing large amounts of data mostly relies on distributed big data processing tools for parallel computing and uses distributed data storage to efficiently process large amounts of data. However, for credit data where the computational logic has a strict sequential relationship, parallel processing can lead to confusion in the order of the data, resulting in erroneous calculation results. Summary of the Invention

[0003] The purpose of the present invention is to at least provide a method, device, equipment and storage medium for processing credit data, which can at least solve the technical problem that when parallel processing data with a strict sequential relationship in calculation logic occurs, it will lead to erroneous calculation results, and at least can achieve the accurate acquisition of correct calculation results based on data with a strict sequential relationship in calculation logic.

[0004] To solve the above technical problems, at least one embodiment of the present application provides a method for processing credit data, including: obtaining an operator to be processed generated during the credit data processing process, at least one first intermediate operator on which the operator to be processed directly depends, and a first intermediate parameter corresponding to each first intermediate operator, wherein the operator to be processed includes data, calculation logic, and a first operator dependency relationship; for each first intermediate operator in at least one first intermediate operator, determining a second operator dependency corresponding to the first intermediate operator based on the first operator dependency relationship, and determining a first operator result based on the second operator dependency relationship, the first intermediate operator, and the corresponding first intermediate parameter, thereby obtaining at least one first operator result; wherein the second operator dependency relationship indicates that the first intermediate operator obtains at least one second intermediate operator on which the corresponding operator result depends, and the execution order of each second intermediate operator in the at least one second intermediate operator; determining a lending prediction result based on the at least one first operator result, the data, and the calculation logic.

[0005] At least one embodiment of the present application also provides a credit data processing device, including: an acquisition module, used to acquire the operator to be processed generated in the credit data processing process, at least one first intermediate operator on which the operator to be processed directly depends, and the first intermediate parameters corresponding to each first intermediate operator, wherein the operator to be processed includes data, calculation logic and first operator dependency; a determination module, used to determine, for each first intermediate operator in at least one first intermediate operator, the second operator dependency corresponding to the first intermediate operator based on the first operator dependency, and determine the first operator result based on the second operator dependency, the first intermediate operator and the corresponding first intermediate parameter, and then obtain at least one first operator result; wherein the second operator dependency indicates the at least one second intermediate operator on which the first intermediate operator depends to obtain the corresponding operator result and the execution order of each second intermediate operator in the at least one second intermediate operator; the determination module is also used to determine the lending prediction result based on the at least one first operator result, the data and the calculation logic.

[0006] At least one embodiment of the present application also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned credit data processing method.

[0007] At least one embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned credit data processing method when executed by a processor.

[0008] Embodiments of the present application provide a credit data processing method that obtains a pending operator generated during credit data processing, at least one first intermediate operator directly dependent on the pending operator, and first intermediate parameters corresponding to each first intermediate operator. The pending operator includes data, computational logic, and a first operator dependency relationship. For each first intermediate operator in at least one first intermediate operator, the method determines a second operator dependency corresponding to the first intermediate operator based on the first operator dependency relationship, and determines a first operator result based on the second operator dependency relationship, the first intermediate operator, and the corresponding first intermediate parameter, thereby obtaining at least one first operator result. The second operator dependency relationship indicates the execution order of at least one second intermediate operator on which the first intermediate operator depends in obtaining the corresponding operator result, as well as each second intermediate operator in the at least one second intermediate operator. The method determines a loan prediction result based on the at least one first operator result, the data, and the computational logic. By encapsulating computational logic within operators, corresponding logic can be obtained based on different operator combinations. This allows for dynamic adjustment of operator logical combinations while maintaining a strict logical order of the data, while also enabling operator reuse. This ensures accurate loan prediction results.

[0009] In some optional embodiments, obtaining the operator to be processed, at least one first intermediate operator directly dependent on the operator to be processed, and the first intermediate parameters corresponding to each first intermediate operator generated during the credit data processing process includes: obtaining a target computing task and a target parameter, and determining the operator to be processed based on the target computing task; determining at least one first intermediate operator directly dependent on the operator to be processed based on the first operator dependency relationship of the operator to be processed; and determining the first intermediate parameter corresponding to each first intermediate operator in the at least one first intermediate operator based on the target parameter. By determining the first intermediate parameter corresponding to each first intermediate operator and passing the parameters required to obtain the result corresponding to each first intermediate operator to the operator, the accuracy of the obtained loan prediction results is improved.

[0010] In some optional embodiments, determining the first operator result based on the second operator dependency, the first intermediate operator, and the corresponding first intermediate parameter includes: determining whether the first intermediate operator depends on a database to obtain the first operator result based on the second operator dependency; if not, using the first intermediate operator as the operator to be processed, the second operator dependency as the first operator dependency, and the first intermediate parameter as the target parameter; and returning to the step of determining at least one first intermediate operator on which the operator to be processed directly depends based on the first operator dependency of the operator to be processed, until the first intermediate operator directly depends on the database to obtain the first operator result, and then determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter. The operator that depends on the database is determined through the operator dependency relationship, and the operator that depends on the database is executed first, and then other operators that depend on the operator (the database-dependent operator) are executed, so as to execute each operator based on the dependency order between the operators, so that the logical order between the data corresponding to each operator is not disrupted, thereby ensuring the correctness of the loan prediction result.

[0011] In some optional embodiments, before determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: if the first intermediate operator directly relies on data in the database to obtain the first operator result, then using the operator to be processed and the second operator dependency relationship to determine the operator identifier of the first intermediate operator; if the first operator result cannot be obtained using the operator identifier, then executing the step of determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter; after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: using the operator identifier as the unique identifier of the first operator result, and persisting the first operator result with the unique identifier. The operator result of the operator that depends on the database is persisted and identified based on the unique identifier, so that the operator that depends on the database only needs to be calculated once, avoiding multiple calculations and wasting computing resources, and also saving access resources for accessing the database. The utilization rate of the operator result is improved.

[0012] In some optional embodiments, after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: if the first intermediate operator only relies on one operator that directly depends on the database to obtain the first operator result, and the first intermediate operator is calculated for the first time, determining the operator identifier of the first intermediate operator using the operator to be processed and the dependency relationship of the second operator; using the operator identifier as the unique identifier of the first operator result, and storing the first operator result with the unique identifier. For a first intermediate operator that only relies on one operator that directly depends on the database, the first operator result of the first intermediate operator is fixed, so by storing the first operator result corresponding to the first intermediate operator, when the first intermediate operator is obtained again, it can be directly obtained from the cache area, avoiding repeated calculations, saving computing resources and computing efficiency.

[0013] In some optional embodiments, after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: if the first intermediate operator relies on multiple operators to obtain the first operator result, not storing the first operator result. For a first intermediate operator that relies on multiple operators, the first operator result of the first intermediate operator is not stored due to uncertainty, thereby avoiding wasting storage resources.

[0014] In some optional embodiments, the method further includes: when verifying whether the first operator result corresponding to any first intermediate operator in the at least one first intermediate operator is correct, passing the first intermediate parameter corresponding to the first intermediate operator to the first intermediate operator to obtain a verification operator result; if the verification operator result is the same as the preset operator result, then determining that the first operator result is correct. Through the dependency and independence between operators, the correctness of any operator can be verified, including the correctness of operators that are dependent on other operators, without the need to manually construct data to overwrite a section of logic for evaluation verification, thereby improving verification efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are exemplarily described by the figures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments.

[0016] Figure 1 This is a flowchart of a method for processing credit data provided by an embodiment of the present application;

[0017] Figure 2 This is a schematic diagram of a structure for obtaining operator results provided by an embodiment of the present application;

[0018] Figure 3This is a schematic diagram of a structure for persistently storing operator results provided by an embodiment of the present application;

[0019] Figure 4 This is a schematic diagram of a structure for storing operator results provided by an embodiment of the present application;

[0020] Figure 5 This is a schematic diagram of a structure that does not store operator results, provided by another embodiment of the present application;

[0021] Figure 6 This is a schematic diagram of a structure for obtaining a target operator result provided by an embodiment of the present application;

[0022] Figure 7 is a schematic diagram of a credit data processing device provided by another embodiment of the present application;

[0023] Figure 8 It is a structural diagram of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0025] It should be noted that the acquisition or use of data in the embodiments of this application requires the user's consent. The relevant data can only be obtained after the user's authorization and permission, and the acquisition or use of the data complies with the provisions of relevant laws and regulations.

[0026] Loan facilitation institutions: Borrowers obtain credit product information through internet platforms. The mutual finance platform conducts a preliminary assessment of the user and provides feedback to the borrower. Credit users are then distributed to different joint loan funding parties through a funds routing mechanism. The two funding parties jointly determine customer screening criteria, and through joint risk control, target customers are screened for joint credit. The borrower signs a loan agreement with all funding parties, pools the loan funds to the funding merchant, and issues a loan disbursement instruction. Financial institutions or mutual finance platforms issue loans to borrowers. Financial institutions and loan facilitation institutions conduct daily reconciliation and repay funds to all funding merchants according to specific settlement rules. This credit model is called a loan facilitation model, and the funding partner institution is the loan facilitation institution.

[0027] Intelligent Distribution: During the completion of an assisted loan, the user's final loan distribution process is dynamically determined by the target institution, completing the credit distribution process. Taking a new credit user as an example, the process involves screening (screening those that meet the institution's requirements), scoring (strategy-based scoring of institutions), sorting (institutions prioritized distribution), and filtering to ensure that users are matched with institutions that meet their requirements as much as possible.

[0028] Customer / Historical Affiliation: During a loan process, a user ultimately goes to a loan facilitating institution for credit approval, loan utilization, and loan disbursement. The user is considered to have been affiliated with this loan facilitating institution. If a user has multiple loans distributed across different loan facilitating institutions, the user's historical affiliation may include multiple loan facilitating institutions.

[0029] CAL operator: The simplest component of a calculation logic, including calculation logic, caching, persistence, and validation components. For example: The success rate of the institution's no-strategy credit card application in a historical month = the number of successful no-strategy credit card applications for the institution's old customers in the historical month / the number of no-strategy credit card customers on the observation day.

[0030] Lineage dependency: The loan repayment prediction model relies on more than 70 operators, and the dependencies between operators are complex. The dependency relationship between operators is called lineage dependency. Through this dependency, the calculation process of any operator can be pulled up with a single click.

[0031] In order to facilitate understanding of the embodiments of the present application, relevant content about the credit data processing method is first introduced here.

[0032] In the field of data processing technology, especially in credit scenarios, when providing predictions about loan information provided by loan-assisting institutions to users, repayment information provided by loan-assisting institutions to investors, and loan fund plans generated by loan-assisting institutions with the assistance of users and investors, it is often necessary to calculate large amounts of data, such as the loan-assisting institutions' historical loan information, repayment information, customer attribution, and order-related data. In existing technologies, computing and processing large amounts of data mostly relies on distributed big data processing tools for parallel computing and uses distributed data storage to efficiently process large amounts of data. However, for credit data where the computational logic has a strict sequential relationship, parallel processing can lead to confusion in the order of the data, resulting in erroneous calculation results.

[0033] Specifically, the existing technology often uses big data tools Spark / Flink as distributed big data processing tools. Although Spark is a powerful data processing framework, it cannot achieve strict serial calculations in scenarios where a large amount of computing logic is required: if the computing logic has a strict sequential relationship, and the calculation result of each step depends on the previous step, Spark parallel computing will cause errors. For example: in certain recursive algorithms or scenarios where data needs to be processed row by row in a specific order, Spark's distributed architecture will increase the complexity of the calculation, and the disorder of the order caused by parallel processing will lead to erroneous results. In addition, the Spark task is an overall task. If you want to verify whether the calculation logic of a certain link in the middle is correct, you can only do it by logging. To verify multiple intermediate results, you need to frequently modify the code and frequently start the code for verification. The verification cost is high and the steps are complicated.

[0034] In order to solve the technical problem that the above-mentioned parallel processing calculation logic may cause erroneous calculation results when processing data with a strict sequential relationship, the present invention proposes a method for processing credit data. The implementation details of the credit data processing method of this embodiment are specifically described below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0035] Example 1:

[0036] The credit data processing method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. The specific process can be as follows: Figure 1 As shown, it includes steps 101 to 103:

[0037] Step 101: Obtain the operator to be processed generated during the credit data processing process, at least one first intermediate operator on which the operator to be processed directly depends, and the first intermediate parameters corresponding to each first intermediate operator, wherein the operator to be processed includes data, calculation logic, and first operator dependency.

[0038] Specifically, credit data includes the historical loan issuance and repayment history of loan assistance institutions, as well as customer attribution and order-related data in the intelligent distribution decision-making process.

[0039] Specifically, the credit data processing process includes predicting future loan information and repayment information of loan assistance institutions based on the historical loan information, historical repayment information and customer attribution and order-related data in the intelligent distribution decision-making process of the loan assistance institutions, as well as the processing process of loan funds plans generated by the loan assistance institutions to help users. Among them, the loan funds plan includes the investor who lends money to the user and the loan amount.

[0040] Specifically, a pending operator refers to an executable operation or computing unit that encapsulates data, computing logic, and dependencies between operators.

[0041] Specifically, the operator to be processed, also known as a CAL operator, includes a reproducible atomic caliber (i.e., computational logic), data for obtaining computational results based on the atomic caliber, and dependencies between operators. Operator implementation encapsulates different computational logics and dependencies between different operators, clarifying the data order between the operator results corresponding to each operator.

[0042] Specifically, the first intermediate operator has the same meaning as the operator to be processed, and the first intermediate operator is an operator that the operator to be processed needs to directly rely on to obtain the corresponding calculation result.

[0043] Specifically, the first intermediate parameter refers to a parameter required to obtain the first intermediate operator or can be understood as a conditional value for obtaining an operator result corresponding to the first intermediate operator.

[0044] In some examples, in the aforementioned step 101, the obtaining of the operator to be processed generated in the credit data processing process, at least one first intermediate operator on which the operator to be processed directly depends, and the first intermediate parameters corresponding to each first intermediate operator include: obtaining a target calculation task and a target parameter, and determining the operator to be processed based on the target calculation task; determining at least one first intermediate operator on which the operator to be processed directly depends based on the first operator dependency relationship of the operator to be processed; and determining the first intermediate parameters corresponding to each first intermediate operator in the at least one first intermediate operator based on the target parameter.

[0045] Specifically, the target parameter is a parameter required to obtain the operator result corresponding to the operator to be processed, or can be understood as a conditional value for obtaining the operator result corresponding to the operator to be processed generated during the credit data processing process.

[0046] Specifically, the first operator dependency relationship describes the operator results corresponding to the operator to be processed generated during the credit data processing process, and the operators that are dependent in order. The operator directly adjacent to the operator to be processed is considered as the first intermediate operator that the operator to be processed directly depends on.

[0047] In some examples, the pending operator and the first intermediate operator can be implemented using JavaBeans or SpringBeans. When calculating the pending operator, the pending operator and each first intermediate operator are stored in a context to obtain the operator results corresponding to the pending operator generated during the credit data processing process. Once the calculation of the pending operator is completed, the context is destroyed.

[0048] JavaBean operators generally perform only logical calculations and data extraction and do not require property injection. SpringBean operators can leverage various Spring container capabilities, such as dependency injection, inversion of control, and data extraction. Both types of operators include capabilities such as parameter validation, calculation, persistence, and caching.

[0049] For example, see Figure 2 When the service for calculating the pending operator is started, a context is first created, and the operator factory, persistence factory, and checker factory are registered in the context so that the context can obtain the pending operator generated during the credit data processing from the operator factory. If the operator factory has not instantiated the pending operator, the data, calculation logic, and first operator dependency corresponding to the pending operator are encapsulated in the operator factory through the operator implementation layer of JavaBean or SpringBean to create or register the pending operator for use as a singleton. With the pending operator as the entry point, the dependency building module is used to build the first operator dependency of the pending operator, and based on the first operator dependency of the pending operator, the corresponding dependent operators are obtained in sequence. The check module is used to obtain the singleton checker corresponding to the operator (the pending operator and the dependent operator) from the checker factory to check whether the operator is correct and initialize the operator for the correct operator. The operator result of the pending operator is obtained based on automatic downward calculation. Among them, the automatic downward calculation method is that if the operator to be processed is operator A, and the first operator dependency is that operator A depends on operator B, and operator B depends on operator C, then the call chain automatically calculates downward, first executing operator C, obtaining the calculation result corresponding to operator C, calculating operator B based on the calculation result of operator C, obtaining the calculation result of operator B, and finally calculating operator A based on the calculation result of operator B, and obtaining the operator result of operator A; the entire process automatically completes dependency management and calculation, and each operator is logically independent and can be assembled arbitrarily. After the calculation is completed, the context will be destroyed, which can be done by triggering the destruction of the operator instance or clearing the program through the local thread. Among them, the operator factory is used to produce operators, the persistence factory is used to persist operators, operator results or other data, and the checker factory is used to generate a singleton checker to check whether the operator is correct.

[0050] In some examples, in terms of dimensions, operators that depend on more operators have higher dimensions, and operators that depend on fewer operators have lower dimensions. The transfer between CAL operators generally includes parameter transfer and result transfer. Parameter transfer is generally the parameters required by high-dimensional operators transferred to low-dimensional operators, and result transfer is generally the operator results required by low-dimensional operators transferred to high-dimensional operators.

[0051] Step 102: For each first intermediate operator in at least one first intermediate operator, determine the second operator dependency corresponding to the first intermediate operator based on the first operator dependency, and determine the first operator result based on the second operator dependency, the first intermediate operator and the corresponding first intermediate parameter, and then obtain at least one first operator result; wherein, the second operator dependency indicates the at least one second intermediate operator on which the first intermediate operator depends to obtain the corresponding operator result and the execution order of each second intermediate operator in the at least one second intermediate operator.

[0052] For example, if the first operator dependency is that operator A depends on operator B, operator B depends on operator C, and operator C depends on operator D, then the at least one first intermediate operator is operator B, and the second operator dependency is that operator B depends on operator C, and operator C depends on operator D. The second operator dependency indicates the execution order of computing operator D first, operator C second, and operator B last.

[0053] In some examples, determining the first operator result based on the second operator dependency, the first intermediate operator and the corresponding first intermediate parameter includes: determining whether the first intermediate operator depends on a database to obtain the first operator result based on the second operator dependency; if not, using the first intermediate operator as the operator to be processed, the second operator dependency as the first operator dependency, and the first intermediate parameter as the target parameter, returning to the step of determining at least one first intermediate operator on which the operator to be processed directly depends based on the first operator dependency of the operator to be processed, until the first intermediate operator directly depends on the database to obtain the first operator result, and then determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter.

[0054] Specifically, relying on a database means directly accessing the database to obtain data.

[0055] In some examples, before determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: if the first intermediate operator directly depends on data in a database to obtain the first operator result, then determining the operator identifier of the first intermediate operator using the operator to be processed and the second operator dependency relationship; if the first operator result cannot be obtained using the operator identifier, executing the step of determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter; after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: using the operator identifier as the unique identifier of the first operator result, and performing persistence processing on the first operator result with the unique identifier.

[0056] Specifically, for operators that rely on a database, it is only necessary to calculate each dimension of data in sequence, and each dimension of data is used to represent a different operator that directly calls the operator (the operator that relies on a database).

[0057] For example, see Figure 3 For operators B and C that rely on the database, if both operators B and C are used as entry points for calculation for the first time, they will encapsulate various dimensional data into memory and complete the initialization of each dimensional data, such as Figure 3 As shown, for the dimension where operator A (the dependent operator) calls operator B, if operator A is calculated, operator A is used as the entry point to obtain the operators B and C on which operator A depends. Operator B and operator C are finitely calculated, and the A storage corresponding to operator A is obtained to store the data corresponding to operator A. If operator B receives parameter B passed in by operator A for the first time and obtains the data required to calculate the first operator result corresponding to operator B from the database, when the first operator result B corresponding to operator B is obtained, the first operator result B is identified with "A-[B#database]" or other unique identifier, and the first operator result B marked with the unique identifier (the dimension data of operator A calling operator B) is persistently cached. For operator A calling operator C, if operator C receives parameter C passed in by operator A for the first time and obtains the data required to calculate the first operator result corresponding to operator C from the database, when the first operator result C corresponding to operator C is obtained, the first operator result C is identified with "A-[C#database]" or other unique identifier, and the first operator result C marked with the unique identifier is persistently cached.

[0058] Furthermore, when operator A calls operator B and operator C again, operators B and C do not need to be recalculated. Instead, the corresponding operator results can be retrieved from persistent memory based on the unique identifier. This eliminates the need for recalculation and database access, saving computational and database access overhead.

[0059] In some examples, after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: if the first intermediate operator only relies on one operator that directly depends on the database to obtain the first operator result, and the first intermediate operator is calculated for the first time, using the operator to be processed and the second operator dependency relationship to determine the operator identifier of the first intermediate operator; using the operator identifier as the unique identifier of the first operator result, and storing the first operator result with the unique identifier.

[0060] For example, see Figure 4During the same context calculation process, obtain the A storage area corresponding to operator A to store the data corresponding to operator A. If operator A depends on operators B and C, and operators B and C both depend on operator D, which in turn depends on the database, then to obtain the calculation result corresponding to operator A, you need to first use operator A as the entry point and pass the parameters required by operators B and C to operators B and C. When calculating operator B, use operator B as the entry point and pass parameter 1 to operator D to call operator D to trigger the calculation of operator D once, obtain result 3 corresponding to operator D, persist result 3, and then calculate operator B based on the parameters and result 3 to obtain result 1 corresponding to operator B. Then, obtain the unique identifier A-[B#D#database] to identify result 1 and store result 1. When calculating operator C, operator C is used as the entry point, and parameter 2 is passed to operator D. Operator D is called to trigger the calculation of operator D once, obtaining result 4 corresponding to operator D. Result 4 is persisted, and operator C is calculated again based on the parameters and result 4, obtaining result 2 corresponding to operator C. The unique identifier A-[C#D#Database] is used to identify result 2, and result 2 is stored. When operator A is used as the entry point to call operators B and C again, operator D is not executed repeatedly, nor are operators B and C calculated repeatedly. Instead, the corresponding results are obtained based on the unique identifier, saving memory, reducing time consumption, and improving data processing efficiency.

[0061] In some examples, after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: if the first intermediate operator depends on multiple operators to obtain the first operator result, then the first operator result is not stored.

[0062] For example, see Figure 5 If operator F directly depends on operator E and operator G, operator E depends on operator Q, operator Q depends on other operators, operator G depends on operator 3, and operator 3 depends on other operators, then operators F, E, and G need to be calculated each time they are called and executed.

[0063] In some examples, if the first intermediate operator only has calculation logic, the execution result is different each time, and the calculation speed is very fast, there is no need to set up a cache, and calculation processing is required each time the first intermediate operator is called.

[0064] For example, see Figure 2In this solution, a cache manager is set up, which includes three cache modes. One is no cache, that is, each time the corresponding operator is called, the operator needs to be calculated and processed. One is Map cache, which uses Map cache to cache the operator results that do not need to be cached but need to be cached. The other is BigDecimal cache, which uses BigDecimal cache to cache the operator results that need to be persisted. When the context obtains the operator from the operator factory, that is, after the operator is initialized, the operator can instantiate the cache manager to obtain the cache mode corresponding to the operator from the cache manager to build the cache and obtain the corresponding storage area. When calculating the operator to be processed, the storage resources and database resources on which the operator depends are obtained from the storage area, that is, the dependent operators and the parameters required by each dependent operator are obtained, and the dependent operator is calculated. If the dependent operator also depends on other dependent operators, then the other dependent operators and the parameters required by each dependent operator are obtained again, that is, through iterative processing of dependent calculations, the dependent operator that depends on the database is obtained, the dependent operator that depends on the database is calculated, the intermediate calculation results are obtained, and the intermediate calculation results are stored. Then, based on the intermediate calculation results, the operators that depend on the dependent operator that depends on the database are calculated in turn, and other intermediate calculation results are obtained and stored. And so on, the calculation results of the operator to be processed are obtained based on other intermediate calculation results, and the calculation results are stored. In addition, a Bloom filter is set up in the storage area. Based on the Bloom filter, it verifies whether the operator results that need to be persisted already exist. If not, the operator results are persisted. If they exist, they are not persisted. When the calculation results of the operator to be processed (loan prediction results) are obtained, the context is destroyed. The destruction module is used to clear the singleton data (data corresponding to each operator) in the storage area. The destruction module is also used to destroy the memory in the cache manager (destroy the storage area).

[0065] Step 103: Determine a lending prediction result based on the at least one first operator result, the data, and the calculation logic.

[0066] Specifically, the loan prediction result is used to describe the predicted loan information of the loan assistance agency, or the predicted repayment information of the loan assistance agency, or the predicted loan fund plan generated by the loan assistance agency to help the user, wherein the loan fund plan includes the investor who lends money to the user and the loan amount.

[0067] In summary, the design using the CAL operator, hierarchical caching, and persistence supports the dynamic combination and adjustment of any computing logic, and can efficiently determine loan prediction results based on the minute level, thereby improving the accuracy of the predicted loan prediction results and the efficiency of iterative verification in the process of determining the loan prediction results.

[0068] Exemplarily, if the calculation logic is operator r = data S × (operator w + operator S), the first operator result corresponding to operator W and the first operator result corresponding to operator S are added to obtain a first sum, and the product of data S and the first sum is used as the lending prediction result.

[0069] In some examples, the method further includes: when verifying whether the first operator result corresponding to any first intermediate operator among the at least one first intermediate operator is correct, passing the first intermediate parameter corresponding to the first intermediate operator to the first intermediate operator to obtain a verification operator result; if the verification operator result is the same as the preset operator result, determining that the first operator result is correct.

[0070] For example, it should be understood that in the existing technology, during the development process or the online verification stage, in order to determine whether the logic of each intermediate calculation process meets expectations, it is usually necessary to see the intermediate data of all calculations, so as to check which step of the calculation caused the problem. This method uses a configuration center and a persistence component to support the rapid location of a problem in a certain calculation logic.

[0071] Specifically, the persistence component can be used to customize the storage logic of operator results. The storage logic takes effect when the operator results are persisted after each operator calculation is completed. The configuration center can be used to dynamically configure which operators to persist. Each operator has a unique identifier, and the configuration center configures a list of operator unique identifiers. If the configuration center has a configuration during the dynamic calculation process, the persistence operation will be performed after the operator calculation is completed. For example, when verifying whether the logic of A=B+C is correct, the configuration center needs to configure the A, B, and C lists and store the three calculation data of A, B, and C in the specified location. Through these three calculation data and calculation rules, it can be determined which operator has a problem and quickly repair it.

[0072] For example, see Figure 2 If you need to verify an operator, you can set the corresponding check items in the checker factory, such as partition check, value check, or logic check. Based on the check items, the corresponding context is generated and the corresponding operator to be verified is initialized to obtain the corresponding operator result. The method for obtaining the operator result can be found in the previous content and will not be elaborated here.

[0073] In some examples, it should be understood that in complex computations, such as those with the logic of A = B + C or D = B + X, when calculating A, operator B will be executed and persisted. When calculating operator D, operator B will also be executed. This raises the question of whether operators D and B should continue to be persisted. Conventional methods use the database to achieve idempotency. However, in scenarios where complex shared operators are short, this approach places significant pressure on the database and exponentially increases the time required due to network interactions. To address this, this solution also sets a unique key value (unique identifier) for the operator result. Using a Bloom filter, the Bloom filter is used to check whether the unique key value exists. If it does not exist, the operator result is persisted in the database. If it does exist, persistence is terminated immediately. This significantly improves computational efficiency while preventing data contamination caused by multiple storage of results. Furthermore, the Bloom filter has a very low false positive rate and currently has no duplicates. Even if duplicate keys are present, this solution does not affect computational accuracy.

[0074] In some examples, it should be understood that in conventional calculations, if you want to get the intermediate result of a calculation, you need to run the entire process from the beginning, or manually construct data to overwrite a section of logic for evaluation. This device constructs lineage dependencies through the calculation rules of operators and operator dependencies. When any operator needs to be executed, it only needs to pass the required parameters, and the evaluation of the entire call chain will be automatically completed starting from the operator. The difference from Spark lineage is that the calculation direction is forward. It can quickly obtain the intermediate result of any calculation (that is, obtain the operator corresponding to the intermediate, pass the parameters corresponding to the operator, and then obtain the intermediate result). Specifically, please refer to Figure 6 When calculating operator 12, you only need to pass the parameters required by operator 12. Operator 12 automatically calls the dependent operator 13, operator 14 and other dependent operators to obtain the operator result corresponding to operator 12.

[0075] Specifically, this solution can be used to abstract any computing process into a unified model, namely: input → output high-level classes, create Spring and Javabean (or Spark, Hive) operators as needed, and realize the process of obtaining calculation results by combining operators. It also adopts a design that moves data to calculation, which improves data computing capabilities.

[0076] In summary, the credit data processing method provided by the embodiments of the present application obtains a pending operator generated during the credit data processing process, at least one first intermediate operator directly dependent on the pending operator, and first intermediate parameters corresponding to each first intermediate operator, wherein the pending operator includes data, computational logic, and a first operator dependency relationship. For each first intermediate operator in at least one first intermediate operator, the second operator dependency corresponding to the first intermediate operator is determined based on the first operator dependency relationship, and a first operator result is determined based on the second operator dependency relationship, the first intermediate operator, and the corresponding first intermediate parameter, thereby obtaining at least one first operator result. The second operator dependency relationship indicates the execution order of at least one second intermediate operator on which the first intermediate operator depends in obtaining the corresponding operator result, as well as each second intermediate operator in the at least one second intermediate operator. A loan prediction result is determined based on the at least one first operator result, the data, and the computational logic. By encapsulating computational logic within operators, corresponding logic can be obtained based on different operator combinations. This allows for dynamic adjustment of operator logical combinations while maintaining a strict logical order of the data, while also enabling operator reuse. This ensures the accuracy of loan prediction results.

[0077] Specifically, the design of operators, hierarchical caching, and persistence supports the dynamic combination and adjustment of any computing logic, achieving efficient predictions at the minute level, improving accuracy, and enhancing the efficiency of iterative verification. For the intermediate data generated during mathematical calculations on large amounts of data, a multi-level cache design is adopted to support caching of different strategies for various types of logic, avoiding repeated execution of the logic and its downstream call chain (call chains consisting of calls to low-dimensional operators), reducing call time and improving efficiency. The introduction of Bloom filter functionality for idempotence solves the problem of repeated persistence, reduces database interactions, improves computing performance, and reduces the time spent in the entire calculation process. By breaking down complex computational logic into individual operators and implementing them through operator combination, the computational logic can be orchestrated more flexibly to implement calculations. During iterations, specific operators can be located and modified without having to worry about other operators. This solves the problems of traditional hard-coded code, including high comprehension and iteration costs, and poor readability. When performing logic verification, the combined design of the configuration center and persistent components allows for dynamic access to the results of each intermediate computation process, verifying whether any computational logic meets expectations. This solves the problem of computational process logic being unverifiable, provides a method for verification, and improves verification efficiency. This solution can reduce iteration costs and improve verification efficiency while providing efficient computing for scenarios involving large quantities of complex business logic.

[0078] Example 2:

[0079] Another embodiment of the present application relates to a credit data processing device. The implementation details of the credit data processing device of this embodiment are described in detail below. The following content is only for the convenience of understanding the implementation details and is not necessary for the implementation of this solution. The schematic diagram of the credit data processing device of this embodiment can be as follows: Figure 7 As shown, it includes an acquisition module 701 and a determination module 702.

[0080] Acquisition module 701 is configured to acquire an operator to be processed generated during credit data processing, at least one first intermediate operator directly dependent on the operator to be processed, and first intermediate parameters corresponding to each first intermediate operator, wherein the operator to be processed includes data, calculation logic, and first operator dependency relationships;

[0081] Determining module 702, configured to determine, for each first intermediate operator in at least one first intermediate operator, a second operator dependency corresponding to the first intermediate operator based on the first operator dependency, and determine a first operator result based on the second operator dependency, the first intermediate operator, and the corresponding first intermediate parameter, thereby obtaining at least one first operator result;

[0082] The second operator dependency relationship indicates at least one second intermediate operator on which the first intermediate operator depends to obtain the corresponding operator result and the execution order of each second intermediate operator in the at least one second intermediate operator;

[0083] The determination module 702 is further configured to determine a lending prediction result based on the at least one first operator result, the data, and the calculation logic.

[0084] In some examples, when the device is used to obtain the operator to be processed generated in the credit data processing process, at least one first intermediate operator on which the operator to be processed directly depends, and the first intermediate parameters corresponding to each first intermediate operator, it is specifically used to: obtain the target calculation task and the target parameter, and determine the operator to be processed based on the target calculation task; determine at least one first intermediate operator on which the operator to be processed directly depends based on the first operator dependency relationship of the operator to be processed; and determine the first intermediate parameter corresponding to each first intermediate operator in the at least one first intermediate operator based on the target parameter.

[0085] In some examples, when the device is used to determine the first operator result based on the second operator dependency, the first intermediate operator and the corresponding first intermediate parameter, it is specifically used to: determine whether the first intermediate operator depends on the database to obtain the first operator result based on the second operator dependency; if not, use the first intermediate operator as the operator to be processed, the second operator dependency as the first operator dependency, and the first intermediate parameter as the target parameter, and return to the step of determining at least one first intermediate operator on which the operator to be processed directly depends based on the first operator dependency of the operator to be processed, until the first intermediate operator directly depends on the database to obtain the first operator result, and then execute the step of determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter.

[0086] In some examples, the device is used to, before determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, also be used to: if the first intermediate operator directly depends on data in a database to obtain the first operator result, then determine the operator identifier of the first intermediate operator using the operator to be processed and the second operator dependency relationship; if the first operator result cannot be obtained using the operator identifier, then execute the step of determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter; after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method also includes: using the operator identifier as the unique identifier of the first operator result, and persisting the first operator result with the unique identifier.

[0087] In some examples, after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the device is also used to: if the first intermediate operator only depends on one operator that directly depends on the database to obtain the first operator result, and the first intermediate operator is calculated for the first time, determine the operator identifier of the first intermediate operator using the operator to be processed and the second operator dependency relationship; use the operator identifier as the unique identifier of the first operator result, and store the first operator result with the unique identifier.

[0088] In some examples, the device is used to, after determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, not store the first operator result if the first intermediate operator depends on multiple operators to obtain the first operator result.

[0089] In some examples, the device is also used to: when verifying whether the first operator result corresponding to any first intermediate operator among the at least one first intermediate operator is correct, pass the first intermediate parameter corresponding to the first intermediate operator to the first intermediate operator to obtain a verification operator result; if the verification operator result is the same as the preset operator result, determine that the first operator result is correct.

[0090] It is worth mentioning that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovation of this application, this embodiment does not include units that are not closely related to solving the technical problem proposed by this application. However, this does not mean that other units do not exist in this embodiment.

[0091] Example 3:

[0092] Another embodiment of the present application relates to an electronic device, such as Figure 8 As shown, it includes: at least one processor 901; and a memory 902 that is communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions that can be executed by the at least one processor 901, and the instructions are executed by the at least one processor 901 to enable the at least one processor 901 to execute the credit data processing method in the above-mentioned embodiments.

[0093] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0094] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0095] Example 4:

[0096] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0097] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0098] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for processing credit data, characterized in that: include: Obtaining an operator to be processed generated during credit data processing, at least one first intermediate operator directly dependent on the operator to be processed, and first intermediate parameters corresponding to each first intermediate operator, wherein the operator to be processed includes data, calculation logic, and first operator dependency relationships; For each first intermediate operator in at least one first intermediate operator, determining a second operator dependency corresponding to the first intermediate operator based on the first operator dependency, and determining a first operator result based on the second operator dependency, the first intermediate operator, and the corresponding first intermediate parameter, thereby obtaining at least one first operator result; The second operator dependency relationship indicates at least one second intermediate operator on which the first intermediate operator depends to obtain the corresponding operator result and the execution order of each second intermediate operator in the at least one second intermediate operator; A lending prediction result is determined based on the at least one first operator result, the data, and the calculation logic.

2. The credit data processing method according to claim 1, characterized in that: The method of obtaining the operator to be processed generated during the credit data processing, at least one first intermediate operator on which the operator to be processed directly depends, and first intermediate parameters corresponding to each first intermediate operator include: Obtaining a target computing task and target parameters, and determining an operator to be processed based on the target computing task; Determining, based on the first operator dependency relationship of the operator to be processed, at least one first intermediate operator on which the operator to be processed directly depends; A first intermediate parameter corresponding to each first intermediate operator in the at least one first intermediate operator is determined based on the target parameter.

3. The credit data processing method according to claim 2, characterized in that: The determining a first operator result based on the second operator dependency, the first intermediate operator, and the corresponding first intermediate parameter includes: Based on the second operator dependency, determine whether the first intermediate operator depends on the database to obtain the first operator result. If not, use the first intermediate operator as the operator to be processed, the second operator dependency as the first operator dependency, and the first intermediate parameter as the target parameter. Return to the step of determining at least one first intermediate operator on which the operator to be processed directly depends based on the first operator dependency of the operator to be processed, until the first intermediate operator directly depends on the database to obtain the first operator result, and then determine the first operator result based on the first intermediate operator and the corresponding first intermediate parameter.

4. The credit data processing method according to claim 3, characterized in that: Before determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: If the first intermediate operator directly relies on data in the database to obtain the first operator result, the operator identifier of the first intermediate operator is determined by using the operator to be processed and the dependency relationship of the second operator; If the first operator result cannot be obtained by using the operator identifier, performing the step of determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter; After determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: using the operator identifier as a unique identifier of the first operator result, and performing persistence processing on the first operator result with the unique identifier.

5. The credit data processing method according to claim 3, characterized in that: After determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: If the first intermediate operator only depends on one operator that directly depends on the database to obtain the first operator result, and the first intermediate operator is calculated for the first time, the operator identifier of the first intermediate operator is determined by using the operator to be processed and the dependency relationship of the second operator; The operator identifier is used as a unique identifier of the first operator result, and the first operator result with the unique identifier is stored.

6. The credit data processing method according to claim 3, characterized in that: After determining the first operator result based on the first intermediate operator and the corresponding first intermediate parameter, the method further includes: If the first intermediate operator depends on multiple operators to obtain the first operator result, the first operator result is not stored.

7. The credit data processing method according to any one of claims 1 to 6, characterized in that: The method further comprises: When verifying whether a first operator result corresponding to any first intermediate operator among the at least one first intermediate operator is correct, passing a first intermediate parameter corresponding to the first intermediate operator to the first intermediate operator to obtain a verification operator result; If the verification operator result is the same as the preset operator result, it is determined that the first operator result is correct.

8. A credit data processing device, characterized in that: include: An acquisition module, configured to acquire an operator to be processed generated during credit data processing, at least one first intermediate operator on which the operator to be processed directly depends, and first intermediate parameters corresponding to each first intermediate operator, wherein the operator to be processed includes data, calculation logic, and first operator dependency relationships; a determination module, configured to determine, for each first intermediate operator in at least one first intermediate operator, a second operator dependency corresponding to the first intermediate operator based on the first operator dependency, and determine a first operator result based on the second operator dependency, the first intermediate operator, and the corresponding first intermediate parameter, thereby obtaining at least one first operator result; The second operator dependency relationship indicates at least one second intermediate operator on which the first intermediate operator depends to obtain the corresponding operator result and the execution order of each second intermediate operator in the at least one second intermediate operator; The determination module is further configured to determine a lending prediction result based on the at least one first operator result, the data, and the calculation logic.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the credit data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the credit data processing method according to any one of claims 1 to 7 is implemented.

Images