A data scoring method and apparatus

Abstract

This invention discloses a data scoring method and apparatus, relating to the field of credit scoring technology. The main technical solution of this invention is as follows: A target sample received from a client is input into a first model; the output of the first model is determined to be abnormal; if abnormal, the target sample is input into a second model to obtain a second scoring result corresponding to the second model, wherein the second model is a disaster recovery model with different input feature variables and the same overall feature missing rate as the first model; the second scoring result is converted into a first scoring result corresponding to the first model based on a quantile mapping relationship, wherein the quantile mapping relationship is a mapping relationship determined based on the first scoring result and the second scoring result for the same sample; the first scoring result is sent to the client. This invention is used for data scoring.

Description

Technical Field

[0001] This invention relates to the field of credit scoring technology, and in particular to a data scoring method and apparatus. Background Technology

[0002] In the field of personal credit risk control and marketing, a common method for assessing the qualifications of applicants or marketing targets is to build a data scoring model. This type of model requires initial construction based on specific data and algorithms. After deployment, the model can be used by the client. Generally, the features (input variables) involved in building the model can come from multiple data sources. However, since these data sources may originate from multiple organizations and are affected by hardware or network conditions, there is a possibility that the data sources may become unavailable or ineffective within a certain period. Data scoring models require all input variables to be evaluated together. If some or all variables are missing (no response value or a null response value), the model will fail (no output value) or its effectiveness will diminish (causing significant fluctuations in the generated quantitative assessment score), leading to disruptions and obstacles in the client's related business processes and harming actual benefits.

[0003] Existing technologies employ a dual-model deployment as a disaster recovery solution for the aforementioned situations. This involves building an auxiliary model using a different data source (to prevent simultaneous failure) while the primary model is being constructed, and deploying both simultaneously to maintain business continuity. However, because the auxiliary model's discriminative power and score distribution differ from the primary model, a separate set of business thresholds needs to be set for it. Since the score distributions corresponding to these thresholds differ between the two models, relevant business monitoring metrics can fluctuate significantly, leading to substantial discrepancies between the scores generated by the auxiliary and primary models. This, in turn, affects the ease, stability, and accuracy of business operations. Summary of the Invention

[0004] In view of the above problems, the present invention provides a data scoring method and apparatus, the main purpose of which is to reduce the difference between the scores generated by the auxiliary model and the scores generated by the main model, so as to ensure that the relevant business is carried out in a simple, stable and accurate manner.

[0005] To solve the above-mentioned technical problems, the present invention proposes the following solution:

[0006] In a first aspect, the present invention provides a data scoring method, the method comprising:

[0007] The received target sample sent by the client is input into the first model;

[0008] Determine if the output of the first model is abnormal;

[0009] If an anomaly is detected, the target sample is input into the second model to obtain the second score result corresponding to the second model. The second model is a disaster recovery model with different input feature variables from the first model and the same overall feature missing rate.

[0010] The second rating result is converted into the first rating result corresponding to the first model based on the quantile mapping relationship. The quantile mapping relationship is determined based on the first rating result and the second rating result of the same sample.

[0011] Send the first rating result to the client.

[0012] Secondly, the present invention provides a data scoring device, the device comprising:

[0013] The first input unit is used to input the target sample received from the client into the first model;

[0014] The judgment unit is used to determine whether the output result of the first model obtained by the first input unit is abnormal;

[0015] The second input unit is used to input the target sample into the second model if the judgment unit determines that the output result of the first model is abnormal, so as to obtain the second score result corresponding to the second model. The second model is a disaster recovery model with different input feature variables and the same overall feature missing rate as the first model.

[0016] The conversion unit is used to convert the second score result obtained by the second input unit into the first score result corresponding to the first model based on the quantile mapping relationship. The quantile mapping relationship is a mapping relationship determined based on the first score result and the second score result of the same sample.

[0017] The sending unit is used to send the first scoring result obtained by the conversion unit to the client.

[0018] To achieve the above objectives, according to a third aspect of the present invention, a storage medium is provided, the storage medium including a stored program, wherein, when the program is executed, the device on which the storage medium is located executes the data scoring method of the first aspect described above.

[0019] To achieve the above objectives, according to a fourth aspect of the present invention, a processor is provided for running a program, wherein the program executes the data scoring method of the first aspect described above.

[0020] By means of the above technical solution, the present invention provides a data scoring method and apparatus. When scoring is required based on data, the target sample received from the client is first input into a first model to obtain the output result of the first model. Then, it is determined whether the output result of the first model is abnormal to determine whether the first model is usable. If it is abnormal, it means that the first model is unusable. Therefore, the target sample can be input into a second model to obtain a second scoring result corresponding to the second model. The second model is a disaster recovery model with different input feature variables and the same overall feature missing rate as the first model. Then, the second scoring result is converted into a first scoring result corresponding to the first model based on the quantile mapping relationship. The quantile mapping relationship is a mapping relationship determined based on the first scoring result and the second scoring result of the same sample. The first scoring result corresponding to the second scoring result can be determined through the quantile mapping relationship. Finally, the first scoring result is sent to the client so that the client can perform relevant subsequent business processes based on the first scoring result received by the client. The data scoring scheme provided by this invention allows for the calculation of a second score for the target sample using a second model when the output of the first model is abnormal. Then, by using quantile mapping, the first score corresponding to the second score under the first model is found. Finally, the first score is sent to the client. This ensures that both the first score obtained from the first model and the first score derived from the second score obtained from the second model can be processed using the business threshold corresponding to the first model, eliminating the need to set additional business thresholds for the second model. This avoids discrepancies in score distribution due to differences in business thresholds between the two models, reduces fluctuations in relevant business monitoring indicators, and minimizes the difference between the scores generated by the auxiliary model and the main model, thereby ensuring the simple, stable, and accurate execution of related business processes.

[0021] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0023] Figure 1 A flowchart of a data scoring method provided by an embodiment of the present invention is shown;

[0024] Figure 2 A flowchart of another data scoring method provided by an embodiment of the present invention is shown;

[0025] Figure 3 This diagram illustrates a data scoring device according to an embodiment of the present invention.

[0026] Figure 4 A block diagram of another data scoring device provided by an embodiment of the present invention is shown. Detailed Implementation

[0027] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0028] In the field of personal credit risk control and marketing, a common method for assessing the qualifications of applicants (or marketing targets) is to build a data scoring model. This type of model requires initial construction based on specific data and algorithms (modeling process). After deployment, the model can be used by the client (application process). The standard model building process is: extracting labeled samples, data cleaning, feature matching, model building, model tuning, and model deployment. Furthermore, since a model can involve multiple features, and the acquisition costs of different features vary (e.g., educational verification information needs to be purchased from the Ministry of Education), the constructed model typically needs to consider the cost-benefit ratio, evaluating whether the gain brought by a certain feature (the improvement in the model's discriminative ability) can cover the cost of purchasing that feature, thus deciding whether to include that feature in the model. During use (application process), by inputting the multi-dimensional features of the applicant samples into the model and performing specific logical calculations, quantitative values ​​of relevant indicators for the applicant's qualifications can be obtained. Based on these quantitative values, relevant thresholds can be set according to business needs, and different business processes, such as approval or rejection, can be set for different score ranges. The aforementioned data model is typically deployed on an online platform, accepting client requests and test samples via API interfaces. It accesses sample features (input variables) matched to the data source through the API interface, performs specified logical operations, and returns an evaluation score to the requester's server via the API interface. The features (input variables) involved in the model can come from multiple data sources, and these different data sources can originate from multiple organizations.

[0029] Because data sources may originate from multiple organizations, they are susceptible to disruptions due to hardware or network conditions, potentially leading to temporary data loss or failure. Generally, data scoring models require all input variables for evaluation. If some or all variables are missing (no response value or null response value), the model will fail (no output value) or its effectiveness will diminish (causing significant fluctuations in the generated quantitative evaluation score), disrupting the client's business processes and harming actual benefits. Current technologies employ a dual-model deployment as a disaster recovery solution. This involves building an auxiliary model using a different data source (to prevent simultaneous failure) while the primary model is being constructed, and deploying both simultaneously to maintain business continuity. However, because the auxiliary model's discriminative power and score distribution differ from the primary model, a separate set of business thresholds needs to be set for the auxiliary model. Since the business thresholds for the two models correspond to different score distributions, related business monitoring indicators will fluctuate significantly, resulting in substantial differences between the scores generated by the auxiliary and primary models. This impacts the ease, stability, and accuracy of business operations. To address this, this invention provides a data scoring method that reduces the discrepancy between the scores generated by the auxiliary model and those generated by the main model, ensuring the simplicity, stability, and accuracy of related business operations. The specific execution steps are as follows: Figure 1 As shown, it includes:

[0030] 101. Input the target sample received from the client into the first model.

[0031] It should be noted that in this embodiment, the first model is a main scoring model built based on given samples with good and bad labels and their feature data, according to a specific operation process and construction algorithm. The model performance is optimized by adjusting model parameters, and finally deployed on the application platform for use. The sample features of the model are obtained from one or more specified data sources, specifically referring to information of a certain dimension of the sample, such as the age, education, income status, etc. of a natural person. The target sample specifically refers to the object to be scored, which is issued by the client. The client refers to the caller, usually a credit department such as a bank, insurance company, or internet finance institution. This embodiment does not make specific limitations on this.

[0032] 102. Determine whether the output of the first model is abnormal.

[0033] In this step, since the first model is the main scoring model for the target sample in this embodiment, after receiving the target sample, the target sample will be input into the first model as soon as possible. The first model will obtain the corresponding input feature variables based on the target sample, and then send an input feature matching request to the specified data source through the input feature variables. If the matched input features are complete (response value is obtained), the output result of the first model is directly used as the scoring result of the target sample. If the matched input features are partially or completely missing (no response value is obtained or the response value is empty), the output result of the first model will be abnormal. It is generally manifested as no output value (model failure) or the output value is a preset specified value, such as 0 or 50 (model effect decay). At this time, the subsequent step 103 is executed.

[0034] 103. Input the target sample into the second model to obtain the second score result corresponding to the second model.

[0035] The second model is a disaster recovery model that differs from the first model in terms of input feature variables but has the same overall feature missing rate. In this embodiment, the second model is an auxiliary scoring model built based on given samples with good and bad labels and their feature data, according to a specific operation process and construction algorithm. The model performance is optimized by adjusting model parameters, and finally deployed on the application platform for use when the first model fails or the output results are abnormal. The modeling samples of the second model are the same as those of the first model. The difference is that the modeling sample features of the second model are obtained from a data source other than the specified data source. The specified data source and other data sources may have different feature data based on the same modeling samples. That is to say, although the modeling samples of the first model and the second model are the same, the feature sources are different, which leads to differences in the feature data of the modeling samples in the specified data source and the feature data in other data sources. Therefore, when the output results of the first model are abnormal, it means that the input features obtained by the first model from the specified data source are missing, and it is impossible to obtain an accurate first score result corresponding to the target sample. Therefore, the score result of the target sample can be calculated by the second model. The input feature variables of the second model and the input feature variables of the first model can be partially different or completely different. This embodiment does not limit this. The overall feature missing rate is based on the overall missing rate of the M modeling samples in the selected D-dimensional input features. Since the first model and the second model are created using all the features of the same modeling samples, by setting the overall feature missing rate of the second model to be the same as that of the first model, the stability of the caller's related business can be guaranteed, and the distinction between the two scoring results obtained after inputting the same target sample into the first model and the second model can be improved. However, it should be noted that in this embodiment, the overall feature missing rate of the first model and the overall feature missing rate of the second model being the same is not completely equivalent. Due to the large number and types of features, it is difficult to make them completely identical during calculation. Therefore, a preset range can be set. As long as the overall feature missing rate of the second model is within 2%-5% lower than that of the first model, the overall feature missing rates of the first model and the second model can be considered to be the same. For example, assuming the scoring range is 0-100, with 50 as the standard value for judging good or bad, if the score is in the range of 51-100, the scoring result is excellent; if it is in the range of 1-49, the scoring result is poor. Specifically, if the target sample is known to be a good sample and its scoring result under the first model is excellent, or if the target sample is known to be a bad sample and its scoring result under the first model is poor, then it proves that the first model has high discrimination; otherwise, it proves that the first model has low discrimination. The same applies to the second model. Therefore, by making the overall feature loss rate of the two models the same, it can be guaranteed that the scoring result of the target sample under the first model is within the same range of good or bad, that is, the discrimination is consistent.

[0036] 104. Based on the quantile mapping relationship, convert the second score result into the first score result corresponding to the first model.

[0037] The quantile mapping relationship is a mapping relationship determined based on the first and second rating results of the same sample. In this embodiment, since the second rating result of the target sample under the second model was obtained in step 103, the first rating result under the first model corresponding to the second rating result can be obtained through the quantile mapping relationship in this step. The quantile mapping relationship is established between the first rating result array and the second rating result array obtained after inputting multiple identical historical samples into the first and second models respectively. Specifically, the quantile refers to the position of a number in a sequence of ordered numbers, represented by a decimal array with equal intervals between 0 and 1, such as the median having a quantile of 0.5.

[0038] 105. Send the first rating result to the client.

[0039] In this step, since the target sample is sent by the caller based on the client, the scoring result based on the target sample (the first scoring result) needs to be sent back to the client so that the caller can process the first scoring result through the business threshold corresponding to the first model. There is no need to set another set of business thresholds, thereby ensuring that the relevant business is carried out accurately, stably and easily.

[0040] Based on the above Figure 1As can be seen from the implementation method, the data scoring method proposed in this embodiment of the invention first inputs the target sample sent by the client into the first model to obtain the output result of the first model when scoring is required based on data. Then, it is judged whether the output result of the first model is abnormal to determine whether the first model is usable. If it is abnormal, it means that the first model is unusable. Therefore, the target sample can be input into the second model to obtain the second scoring result corresponding to the second model. The second model is a disaster recovery model with different input feature variables and the same overall feature missing rate as the first model. Then, the second scoring result is converted into the first scoring result corresponding to the first model based on the quantile mapping relationship. The quantile mapping relationship is a mapping relationship determined based on the first scoring result and the second scoring result of the same sample. The first scoring result corresponding to the second scoring result can be determined through the quantile mapping relationship. Finally, the first scoring result is sent to the client so that the client can perform relevant subsequent business processes based on the first scoring result received by the client. The data scoring scheme provided by this invention allows for the calculation of a second score for the target sample using a second model when the output of the first model is abnormal. Then, by using quantile mapping, the first score corresponding to the second score under the first model is found. Finally, the first score is sent to the client. This ensures that both the first score obtained from the first model and the first score derived from the second score obtained from the second model can be processed using the business threshold corresponding to the first model, eliminating the need to set additional business thresholds for the second model. This avoids discrepancies in score distribution due to differences in business thresholds between the two models, reduces fluctuations in relevant business monitoring indicators, and minimizes the difference between the scores generated by the auxiliary model and the main model, thereby ensuring the simple, stable, and accurate execution of related business processes.

[0041] Furthermore, as a response to Figure 1 Further refinement and extension of the illustrated embodiment, this invention also provides another data scoring method, such as... Figure 2 As shown, the specific steps are as follows:

[0042] 201. Obtain the modeling samples with missing first scoring results from the modeling sample set of the first model, and form an analysis sample subset.

[0043] The analysis sample subset consists of modeling samples whose features are missing relative to the inherent features of the specified data source corresponding to the first model. In this embodiment, it should be noted that the features inherent to both the specified data source and other data sources are inherent. Since the first model is pre-built and trained based on the modeling samples and their feature data, and the corresponding features are obtained from the specified data source during construction, the modeling samples with missing first score results output during the construction and training of the first model are first acquired and constituted as the analysis sample subset. The purpose is to obtain as many second model modeling feature sets as possible that can be used to create the second model by comparing the modeling samples with missing first score results and their corresponding feature sets, so that the missing samples of the first model and the second model overlap, thereby helping to make the overall feature missing rate of the two models the same.

[0044] 202. Extract and analyze the sample features in the sample subset that are completely missing from the inherent features of other data sources, and form a basic feature set.

[0045] Other data sources refer to data sources other than the specified data source. In this step, since the subset of analysis samples has been determined in step 201 above, it is possible to identify the missing features of each modeling sample in the subset of analysis samples relative to the inherent features of other data sources, i.e., features that do not exist in other data sources. These features are then extracted to form a basic feature set, which can be used as a candidate modeling feature set for creating the second model.

[0046] 203. Sort the remaining candidate features in descending order according to their respective feature missing rates in the subset of the analysis samples.

[0047] The remaining candidate features are those features in the sample subset that are partially missing relative to the inherent features of other data sources. It should be noted that in this step, since all features in the sample subset that are completely missing relative to other data sources have already been extracted, to ensure that the overall feature loss rate of the basic feature set is the same as the overall feature loss rate of the first model, the remaining candidate features need to be added to the basic feature set sequentially. To improve the ease of addition, specifically, based on the calculation of the overall feature loss rate of the first model and the feature loss rate of the remaining candidate features in their respective sample subsets, the missing status of each dimension feature on each sample can be stored as a Boolean value (true or false), with True for missing and False for not missing. For D-dimensional features and M modeling samples, this constitutes an M-row, D-column Boolean matrix. For each row in the matrix, determine whether it is all true (or calculate its bitwise AND value), which is the missing status of the modeling sample in the D-dimensional feature corresponding to each row. Calculate the true value ratio (true value count / M) of the resulting M Boolean values, which is the overall missing rate of the M samples in the selected D-dimensional feature. After the overall feature missing rate of the first model and the remaining candidate features have been calculated according to their respective feature missing rates in the sample subsets, they can be sorted in descending order based on the feature missing rate in order to execute the subsequent step 204.

[0048] 204. According to the overall feature missing rate corresponding to the first model, the remaining candidate features with the highest feature missing rate in the sample subset of analysis are added to the basic feature set in turn, so that the overall feature missing rate corresponding to the first model is the same as the overall feature missing rate of the basic feature set.

[0049] In this step, since the overall feature missing rate of the first model and the remaining candidate features are both known according to their respective feature missing rates in the analysis sample subset, in order to make the overall feature missing rate of the second model the same as that of the first model, the features with the highest feature missing rate in the analysis sample subset can be added to the basic feature set in sequence until the overall feature missing rate of the basic feature set is the same as that of the first model. Specifically, when adding a one-dimensional input feature variable, it is necessary to calculate the missing rate of its corresponding input feature. This can be done by performing a bitwise AND operation between the M Boolean values ​​formed by the missing status of the feature in each modeling sample and the M Boolean values ​​formed by the missing status of the selected features in each modeling sample. This will give the overall missing status after adding the one-dimensional feature. However, "same" does not mean completely identical. Due to the large number and types of features, it is difficult to make them completely identical during the calculation. Therefore, a preset range can be set. As long as the overall feature missing rate of the second model is within 2%-5% lower than that of the first model, the overall feature missing rate of the first model and the basic feature set used to construct the second model can be considered to be the same. Based on the above method, it can be ensured that the second scoring result obtained after the target sample is input into the second model has the same degree of discrimination as the first scoring result.

[0050] 205. Create a second model based on the basic feature set and the modeling sample set.

[0051] In this embodiment, since the second model and the first model are created using the same modeling samples and feature data, and since the overall feature missing rate of the basic feature set has been determined as described in step 204 above, the second model can be created based on the basic feature set and the modeling sample set as a disaster recovery model when the output result of the first model is abnormal.

[0052] 206. Obtain historical samples of the first model.

[0053] In this step, the historical samples can be historical samples within a specified period or all historical samples; this embodiment does not specifically limit this.

[0054] 207. Input the historical samples into the first model and the second model respectively to obtain the first scoring result array and the second scoring result array.

[0055] In this step, since there may be multiple historical samples, after they are input into the first model and the second model respectively, there are also multiple first and second score results. Therefore, multiple first score results and multiple second score results can be processed in the form of an array in order to execute the subsequent step 208.

[0056] 208. Construct a quantile mapping relationship based on the first and second score result arrays.

[0057] In this step, since quantiles specifically refer to the position of a number within a sequence of ordered numbers, in order to construct the mapping relationship between the first and second score result arrays, the score results in the first and second score result arrays are sorted in ascending order. It is then determined whether the number of score results in the first and second score result arrays is the same. If they are different, the first and second score result arrays are scaled to ensure that the overall field length of the score results in the first and second score result arrays is the same. A quantile mapping relationship is then constructed for the score results in the first and second score result arrays that are in the same position.

[0058] For example, suppose array A (the first rating result array) has M non-missing scores, and array B (the second rating result array) has N non-missing scores. If M and N are different, they need to be scaled to the same length to ensure that every score (rating result) in array B has a unique corresponding score (rating result) in array A, and these two corresponding scores (rating results) have the same quantile distribution in their respective arrays. Without loss of generality, assume M is greater than N.

[0059] Array A: [1,3,4,5,5,6,7,8,9,9];

[0060] Array B: [2,4,4,5,8,9,9,9].

[0061] Expand array B into an array of the same length as array A based on the following logic.

[0062] (1) Count the frequency of each unique fraction in array B, divide the frequency by N to obtain its relative frequency, and then multiply by M to convert it into the corresponding frequency in an array of length M, as shown in Table 1:

[0063] Table 1

[0064] Here, array C is a modified array set up by multiplying the frequency * M of each fraction in array B.

[0065] (2) Iterate through the frequency of each fraction in the scaled array B from smallest to largest, and sum them to obtain the quantile of each fraction, as shown in Table 2:

[0066] element 2 4 5 8 9 Quantiles of each number 1.25 3.75 5 6.25 10

[0067] Table 2

[0068] Then, iterate through each integer from 0 to M, selecting the element corresponding to the smallest quantile greater than that integer, and add it to array C: {0:2,1:2,2:4,3:4,4:5,5:5,6:8,7:9,8:9,9:9}. For example, for the integer 6, the smallest quantile greater than that integer is 6.25, and its corresponding number is 8, as shown in Table 3.

[0069]

[0070] Table 3

[0071] (3) Sort array A in ascending order and array C in ascending order. Construct one-to-one data pairs from array A to array C between numbers with the same sort index in both arrays, as shown in Table 4:

[0072] Array A 1 3 4 5 5 6 7 8 9 9 Array C 2 2 4 4 5 5 8 9 9 9

[0073] Table 4

[0074] (4) a. For the cases where the scores in the same array A correspond to the scores in the same array C, construct a quantile mapping relationship between these two scores (each element in array A uniquely corresponds to an element in array C). For example, element 1 in array A corresponds only to element 2 in array C, therefore there is a mapping relationship of 1 to 2. Conversely, element 5 in array A corresponds to elements 4 and 5 in array C, therefore, no mapping relationship can be formed in this step. The correspondence relationship after removing element 5 is shown in Table 5:

[0075] Elements in array A 1 3 4 6 7 8 9 Elements in array C 2 2 4 5 8 9 9

[0076] Table 5

[0077] b. When a score in the same array A corresponds to a score in multiple arrays C, calculate the proportion of scores in array C corresponding to the scores in array A, and use this as the quantile mapping relationship. For example, if the element 5 in array A corresponds to either 4 or 5 in array C, then it needs to be divided according to its frequency, i.e., 50% likely to correspond to 4 and 50% likely to correspond to 5.

[0078] In summary, the relationship from array A to array B can be represented as follows:

[0079]

[0080] The following mapping relationship can be obtained:

[0081]

[0082] Based on the above method, the first score result corresponding to the second score result can be quickly found based on the quantile mapping relationship. Then, subsequent processing can be carried out based on the business threshold of the first model corresponding to the first score result, which greatly improves the ease of related business operations, eliminates the need to set an additional set of business thresholds, and reduces the complexity of use.

[0083] 209. Input the target sample received from the client into the first model.

[0084] This step combines the description of step 101 in the above method, and the same content will not be repeated here.

[0085] 210. Determine whether the output of the first model is abnormal.

[0086] This step combines the description of step 102 in the above method, and the same content will not be repeated here. Specifically, based on the target sample, the input feature variables corresponding to the first model are obtained. The input feature variables are used to match the corresponding input features from the specified data source. The input features are then input into the first model to obtain the output result of the first model. It is then determined whether the output result of the first model conforms to the preset rules. If it does, the output result of the first model is determined to be abnormal.

[0087] The input feature variables are pre-set based on the first model. When the first model receives a target sample, it obtains the corresponding input feature variables based on the target sample and then sends an input feature matching request to the specified data source through the input feature variables. That is, it uses the input feature variables to match the corresponding input features from the specified data source. If the matched input feature variables are complete (response value is obtained), the output result of the first model is directly used as the score result of the target sample. If some or all of the matched input feature variables are missing (no response value is obtained or the response value is empty), the preset rules are set to determine the abnormal output result of the first model. For example, the first model output result has no output value (model failure) or the output value is a preset specified value, etc. If the output result of the first model meets the preset rules, the output result of the first model is determined to be abnormal, that is, the first model output result has no output value (model failure) or the output value is a preset specified value, such as 0 or 50 (model effect decay), thereby ensuring that the transition of the target sample from the first model to the second model is faster and more accurate.

[0088] Furthermore, it can also detect whether the input feature matching request sent to the specified data source has timed out. If it has timed out, the input feature matching request is resent to ensure the efficiency of input feature acquisition. It should be noted that this step applies to both the first model and the second model.

[0089] 211. Input the target sample into the second model to obtain the second scoring result corresponding to the second model.

[0090] This step combines the description of step 103 in the above method, and the same content will not be repeated here.

[0091] 212. Based on the quantile mapping relationship, convert the second score result into the first score result corresponding to the first model.

[0092] This step combines the description of step 104 in the above method, and the same content will not be repeated here.

[0093] 213. Send the first rating result to the client.

[0094] This step combines the description of step 105 in the above method, and the same content will not be repeated here.

[0095] 214. Obtain the first and second score result arrays corresponding to the historical samples according to the preset period.

[0096] In this step, the quantile mapping relationship is determined based on the first and second score result arrays obtained after inputting the same historical samples into the first and second models respectively. However, as the target samples are processed subsequently, the period between the same historical samples used to create the quantile mapping relationship and the target samples becomes increasingly longer. Furthermore, due to the diversity of the target samples, the corresponding first and second score results may differ, leading to a decrease in the accuracy of the quantile mapping relationship. Therefore, to ensure the accuracy of the quantile mapping relationship, the first and second score result arrays corresponding to the historical samples can be obtained within a preset period. The preset period can be set in units such as weeks or months. The first and second score results of the historical samples obtained each time can be score result arrays from a recent time period or score result arrays from a specified time period. If the data volume is too large, extraction can also be selected. This embodiment does not specifically limit this method.

[0097] 215. Update the quantile mapping relationship based on the first and second score result arrays.

[0098] In this step, since the first and second score result arrays corresponding to the historical samples have already been obtained in step 214, step 208 can be executed again to update the quantile mapping relationship, thereby ensuring the accuracy of the first score result corresponding to the target sample and ensuring that the relevant business can be carried out accurately.

[0099] Based on the above Figure 2As can be seen from the implementation method, the data scoring method proposed in this embodiment of the invention, when scoring is required based on data, firstly obtains modeling samples with missing first scoring results from the modeling sample set of the first model, and forms an analysis sample subset. Then, extracts the sample features in the analysis sample subset that are completely missing relative to the inherent features of other data sources, and forms a basic feature set. Next, the remaining candidate features are sorted in descending order according to their respective feature missing rates in the analysis sample subset. Then, according to the overall feature missing rate corresponding to the first model, the remaining candidate features with the highest feature missing rate in the analysis sample subset are added to the basic feature set, so that the overall feature missing rate corresponding to the first model is the same as the overall feature missing rate of the basic feature set. Finally, a second model is created based on the basic feature set and the modeling sample set. Based on the above method, it can be ensured that the overall feature missing rate of the created second model is the same as the overall feature missing rate of the first model, thereby ensuring the stability of the client's related business. It can also improve the discriminatory power of the two scoring results obtained after inputting the same target sample into the first model and the second model respectively, making the scoring results more accurate. Accurately, by acquiring historical samples from the first model and inputting them into the first and second models respectively, a first score result array and a second score result array are obtained. A quantile mapping relationship is constructed based on the first and second score result arrays. The target sample received from the client is input into the first model and then into the second model to obtain the second score result corresponding to the second model. The output result of the first model is checked for abnormality. Based on the quantile mapping relationship, the second score result is converted into the first score result corresponding to the first model and sent to the client. The first score result array and the second score result array corresponding to the historical samples are acquired within a preset period. The quantile mapping relationship is updated based on the first and second score result arrays. Based on the above method, the first score result corresponding to the second score result can be quickly found based on the quantile mapping relationship. Subsequent processing can then be performed based on the business threshold of the first model corresponding to the first score result, greatly improving the simplicity of related business operations. There is no need to set an additional set of business thresholds, reducing the complexity of use and ensuring the processing effect of subsequent steps.

[0100] Furthermore, as a response to the above Figure 1 In addition to the implementation of the method shown, this embodiment of the invention also provides a data scoring device for the above-described method. Figure 1 The method shown is implemented accordingly. This device embodiment corresponds to the foregoing method embodiment. For ease of reading, this device embodiment will not repeat the details of the foregoing method embodiment, but it should be clear that the device in this embodiment can implement all the contents of the foregoing method embodiment. Figure 3 As shown, the device includes:

[0101] The first input unit 301 is used to input the target sample received from the client into the first model;

[0102] The judgment unit 302 is used to determine whether the output result of the first model obtained by the first input unit 301 is abnormal;

[0103] The second input unit 303 is used to input the target sample into the second model if the judgment unit 302 determines that the output result of the first model is abnormal, so as to obtain the second score result corresponding to the second model. The second model is a disaster recovery model that has different input feature variables from the first model and the same overall feature missing rate.

[0104] The conversion unit 304 is used to convert the second rating result obtained by the second input unit 303 into the first rating result corresponding to the first model based on the quantile mapping relationship, wherein the quantile mapping relationship is a mapping relationship determined based on the first rating result and the second rating result of the same sample;

[0105] The sending unit 305 is used to send the first scoring result obtained by the conversion unit 304 to the client.

[0106] Furthermore, such as Figure 4 As shown, the device further includes: a first acquisition unit 306, configured to acquire modeling samples whose first scoring result is missing from the modeling sample set of the first model before the second input unit 302, and form an analysis sample subset;

[0107] Extraction unit 307 extracts the sample features from the subset of analysis samples obtained by the first acquisition unit 306 that are completely missing from the inherent features of other data sources, and forms a basic feature set.

[0108] Arrangement unit 308 is used to sort the remaining candidate features in descending order according to their respective feature missing rate in the analysis sample subset after extraction unit 307.

[0109] Adding unit 309 is used to add the remaining candidate features with the highest feature missing rate obtained by the arrangement unit 308 in the analysis sample subset to the basic feature set according to the overall feature missing rate corresponding to the first model, so that the overall feature missing rate corresponding to the first model is the same as the overall feature missing rate of the basic feature set.

[0110] The adjustment unit 310 is used to create the second model based on the basic feature set obtained by the addition unit 309 and the modeling sample set.

[0111] Furthermore, such as Figure 4 As shown, the device further includes:

[0112] The second acquisition unit 311 is used to acquire historical samples of the first model before the conversion unit 304;

[0113] The third input unit 312 is used to input the historical samples obtained by the second acquisition unit 311 into the first model and the second model respectively, so as to obtain the first scoring result array and the second scoring result array;

[0114] The construction unit 313 is used to construct the quantile mapping relationship based on the first scoring result array and the second scoring result array obtained by the third input unit 312.

[0115] Furthermore, such as Figure 4 As shown, the building unit 313 includes:

[0116] The sorting module 3131 is used to sort the scoring results in the first scoring result array and the second scoring result array in ascending order, respectively.

[0117] The first judgment module 3132 is used to determine whether the number of score results in the first score array and the second score array obtained by the arrangement module 3131 are the same;

[0118] The processing module 3133 is used to scale and transform the first rating result array and the second rating result array if the first judgment module 3132 determines that the number of rating results in the first rating array and the second rating array are different, so as to ensure that the overall field length of the rating results in the first rating array and the second rating array is the same.

[0119] The construction module 3134 is used to construct the quantile mapping relationship from the score results in the first score result array and the second score result array obtained by the processing module 3133 at the same position.

[0120] Furthermore, such as Figure 4 As shown, the device further includes:

[0121] The third acquisition unit 314 is used to acquire the first scoring result array and the second scoring result array corresponding to the historical sample within a preset period;

[0122] The update unit 315 is used to update the quantile mapping relationship based on the first score result array and the second score result array obtained by the third acquisition unit.

[0123] Furthermore, such as Figure 4 As shown, the judgment unit 302 includes:

[0124] The acquisition module 3021 is used to acquire the input feature variables corresponding to the first model based on the target sample;

[0125] The matching module 3022 is used to match the corresponding input features from the specified data source using the input feature variables obtained by the acquisition module 3021;

[0126] Input module 3023 is used to input the input features obtained by matching module 3022 into the first model to obtain the output result of the first model;

[0127] The second judgment module 3024 is used to judge whether the output result of the first model obtained by the input module 3023 conforms to the preset rules.

[0128] The determination module 3025 is used to determine that the output result of the first model is abnormal if the second judgment module 3024 determines that the output result of the first model conforms to the preset rules.

[0129] Furthermore, embodiments of the present invention also provide a storage medium for storing a computer program, wherein the computer program, when running, controls the device where the storage medium is located to execute the above-described... Figure 1-2 The data scoring method described in [the document].

[0130] Furthermore, embodiments of the present invention also provide a processor for running a program, wherein the program executes the above-described... Figure 1-2 The data scoring method described in [the document].

[0131] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0132] It is understood that the relevant features in the above methods and apparatus can be referenced interchangeably. Furthermore, the terms "first," "second," etc., in the above embodiments are used to distinguish between embodiments and do not represent the superiority or inferiority of any particular embodiment.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0134] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0135] In addition, the memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0136] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0138] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0139] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0140] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0141] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0142] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0143] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data scoring method applied in the field of credit scoring, characterized in that, include: The target sample received from the client is input into the first model. The first model is a main scoring model built based on the given samples with good and bad labels and their feature data. The sample features used to build the model are obtained from one or more specified data sources. The target sample is the object to be scored. To determine whether the output of the first model is abnormal, specifically, based on the target sample, the input feature variable corresponding to the first model is obtained, and the input feature variable of the first model is used to match the corresponding input feature from the specified data source. The matched input feature is input into the first model to obtain the output of the first model. If the output of the first model is no output value or a preset specified value, then the output of the first model is determined to be abnormal. If an anomaly is detected, the target sample is input into the second model to obtain the second score result corresponding to the second model. The second model is a disaster recovery model with different input feature variables from the first model and the same overall feature missing rate. The modeling sample features of the second model are obtained from other data sources besides the specified data source. The second rating result is converted into the first rating result corresponding to the first model based on the quantile mapping relationship, wherein the quantile mapping relationship is determined based on the mapping relationship between the first rating result array and the second rating result array obtained by inputting the same sample into the first model and the second model respectively; The first rating result is sent to the client.

2. The method according to claim 1, characterized in that, Before inputting the target sample into the second model, the method further includes: The modeling samples for which the first scoring result is missing are obtained from the modeling sample set of the first model and constitute an analysis sample subset. The analysis sample subset is a set of modeling samples whose sample features are missing relative to the inherent features of the specified data source corresponding to the first model. The sample features in the subset of the analysis samples are completely missing relative to the inherent features of other data sources, and these features are used to form a basic feature set. The other data sources are data sources other than the specified data source. The remaining candidate features are sorted in descending order of their respective feature missing rates in the subset of the analysis samples. The remaining candidate features are those features in the subset of the analysis samples that are partially missing relative to the inherent features of other data sources. According to the overall feature missing rate corresponding to the first model, the remaining candidate features with the highest feature missing rate in the subset of analysis samples are added to the basic feature set in sequence, so that the overall feature missing rate corresponding to the first model is the same as the overall feature missing rate of the basic feature set. The second model is created based on the basic feature set and the modeling sample set.

3. The method according to claim 1, characterized in that, Before converting the second rating result into the first rating result corresponding to the first model based on the quantile mapping relationship, the method further includes: Obtain historical samples from the first model; The historical samples are input into the first model and the second model respectively to obtain a first scoring result array and a second scoring result array; The quantile mapping relationship is constructed based on the first scoring result array and the second scoring result array.

4. The method according to claim 3, characterized in that, Constructing the quantile mapping relationship based on the first scoring result array and the second scoring result array includes: Sort the scores in the first and second score result arrays in ascending order, respectively. Determine whether the number of rating results in the first rating result array and the second rating result array are the same; If they are different, the first rating result array and the second rating result array are scaled and transformed to ensure that the overall field length of the rating results in the first rating result array and the second rating result array is the same; The quantile mapping relationship is constructed for the rating results in the first and second rating result arrays that are in the same position.

5. The method according to claim 3, characterized in that, The method further includes: The first scoring result array and the second scoring result array corresponding to the historical samples are obtained according to a preset period. The quantile mapping relationship is updated based on the first scoring result array and the second scoring result array.

6. The method according to claim 1, characterized in that, Determining whether the output of the first model is abnormal includes: Based on the target sample, obtain the input feature variables corresponding to the first model; The input model feature variables are used to match the corresponding input model features from the specified data source; The input features are input into the first model to obtain the output result of the first model; Determine whether the output of the first model conforms to the preset rules; If the conditions are met, then the output of the first model is determined to be abnormal.

7. A data scoring device, characterized in that, The device, used in the field of credit scoring, includes: The first input unit is used to input the target sample received from the client into the first model. The first model is a main scoring model built based on the given samples with good and bad labels and their feature data. The sample features used to build the model are obtained from one or more specified data sources. The target sample is the object to be scored. The judgment unit is used to judge whether the output result of the first model obtained by the first input unit is abnormal. Specifically, based on the target sample, the input feature variable corresponding to the first model is obtained, the input feature variable of the first model is used to match the corresponding input feature from the specified data source, and the matched input feature is input into the first model to obtain the output result of the first model. If the output result of the first model is no output value or a preset specified value, then the output result of the first model is determined to be abnormal. The second input unit is used to input the target sample into the second model if the judgment unit determines that the output result of the first model is abnormal, so as to obtain the second score result corresponding to the second model. The second model is a disaster recovery model with different input feature variables from the first model and the same overall feature missing rate. The modeling sample features of the second model are obtained from other data sources outside the specified data source. A conversion unit is used to convert the second score result obtained by the second input unit into the first score result corresponding to the first model based on the quantile mapping relationship. The quantile mapping relationship is a mapping relationship determined based on the first score result array and the second score result array obtained by inputting the same sample into the first model and the second model respectively. A sending unit is used to send the first scoring result obtained by the conversion unit to the client.

8. The apparatus according to claim 7, characterized in that, Before inputting the target sample into the second model, the device further includes: The first acquisition unit is used to acquire modeling samples whose first scoring result is missing from the modeling sample set of the first model, and form an analysis sample subset, wherein the analysis sample subset is a set of modeling samples whose sample features are missing relative to the inherent features of the specified data source corresponding to the first model. The extraction unit extracts the sample features from the subset of analysis samples obtained by the first acquisition unit that are completely missing from the inherent features of other data sources, and forms a basic feature set. The sorting unit is used to sort the remaining candidate features in descending order according to their respective feature missing rates in the subset of analysis samples extracted by the extraction unit. An adding unit is used to add the remaining candidate features with the highest feature missing rate obtained by the arrangement unit in the subset of analysis samples to the basic feature set according to the overall feature missing rate corresponding to the first model, so that the overall feature missing rate corresponding to the first model is the same as the overall feature missing rate of the basic feature set. An adjustment unit is used to create the second model based on the basic feature set obtained by the addition unit and the modeling sample set.

9. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the data scoring method as described in any one of claims 1 to 6.

10. A processor, characterized in that, The processor is used to run a program, wherein the program executes the data scoring method as described in any one of claims 1 to 6.

Images