Data processing method, data processing device and electronic equipment

By transforming variables in binary classification statistical models, using the log-probability ratio linear regression of sample proportions, the problem that variable transformation methods in the existing technology is difficult to improve generalization ability and stability, and the retention of variable order information is achieved. and effective reflection of grouped sample distribution.

CN112115955BActive Publication Date: 2024-11-15ZHONGKE JUXIN INFORMATION TECH BEIJING CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910535040.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-06-20
Publication Date
2024-11-15
Estimated Expiration
2039-06-20

AI Technical Summary

Technical Problem

In the development of binary classification statistical models, existing variable transformation methods are difficult to effectively improve the generalization ability and stability of variables, especially when dealing with missing samples and smooth noise. Common methods will lose variable order information or be unable to react Distribution of grouped samples.

Method used

By dividing the test samples into multiple groups, the proportion of samples of specific categories of each group and their logarithmic probability ratio is calculated, and the logarithmic probability ratio is used as the dependent variable and the group sequence number is the independent variable to perform univariate linear regression, and the variable transformation is obtained by obtaining the logarithmic probability ratio. Model.

Benefits of technology

The generalization ability and stability of variables are improved, the order information of the original variables is retained, and missing values ​​and outliers are effectively processed, reflecting the distribution of grouped samples.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112115955B_ABST
    Figure CN112115955B_ABST
Patent Text Reader

Abstract

The present application relates to a data processing method, a data processing device and an electronic device. The data processing method is used for variable transformation of a binary classification statistical model, including: dividing a test sample into a plurality of groups in sequence, each of the plurality of groups having a group number; calculating the sample proportion of a specific category sample of each group; calculating the logarithmic probability ratio of the sample proportion; and performing a univariate linear regression with the logarithmic probability ratio as the dependent variable and the group number as the independent variable to obtain a variable transformation model. In this way, the variable transformation of the binary classification statistical model can be performed by linear regression based on the logarithmic probability ratio of the sample proportion, thereby improving the generalization ability and stability of the variable.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application generally relates to the field of data processing, and more specifically, to a data processing method, a data processing device and an electronic device for variable transformation of a binary classification statistical model. Background Art

[0002] Variable transformation is a commonly used method in statistics. It is commonly used in the model development process and is also a very necessary data preprocessing process. The purpose of variable transformation may vary slightly depending on the choice of specific model. The usual purposes are to reduce variance, limit the range of data values, smooth data noise, etc. For example, commonly used variable transformation methods include range transformation and regularization transformation.

[0003] Binary classification statistical models are widely used in credit risk models, fraud risk models, and marketing models. For example, the most widely used credit scoring model - binary logistic regression model is a binary classification model.

[0004] In the process of developing binary classification statistical models, variables also need to be transformed to improve model performance.

[0005] Therefore, there is a need for improved variable transformation mechanisms for binary classification statistical models. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a data processing method, a data processing device and an electronic device, which can perform variable transformation of a binary classification statistical model through linear regression of logarithmic probability ratio based on sample proportions, thereby improving the generalization ability and stability of the variables.

[0007] According to one aspect of the present application, there is provided a data processing method for variable transformation of a binary classification statistical model, the method comprising: dividing a test sample into a plurality of groups in sequence, each of the plurality of groups having a group number; calculating the sample proportion of samples of a specific category of each group; calculating the logarithmic probability ratio of the sample proportion; and performing univariate linear regression with the logarithmic probability ratio as the dependent variable and the group number as the independent variable to obtain a variable transformation model.

[0008] In the above data processing method, dividing the test sample into a plurality of groups in sequence includes: determining the type of the test sample; and, in response to the test sample being a categorical predictor variable, setting the test sample of each classification as a group.

[0009] In the above data processing method, dividing the test samples into a plurality of groups in sequence further comprises: sorting the plurality of groups based on the sample ratio of the group corresponding to each classification.

[0010] In the above data processing method, it further includes: in response to the test sample being an ordinal predictor variable, sorting the test sample in order; and setting the test sample corresponding to each ordinal value as a group.

[0011] In the above data processing method, it further includes: in response to the test sample being a continuous predictive variable, sorting the test sample according to the predictive variable; dividing the test sample into N groups in sequence, N is an integer greater than one; and setting each of the N groups as a group.

[0012] In the above data processing method, it further includes: obtaining an input sample; determining whether the input sample is a missing sample or a non-missing sample; and, in response to the input sample being a non-missing sample, calculating the transformed variable value based on the variable transformation model.

[0013] In the above data processing method, it further includes: in response to the input sample being a missing sample, analyzing the distribution of the target variable in the input sample to calculate the sample proportion of samples of a specific category in the input sample; and calculating the logarithmic probability ratio of the sample proportion as the transformed variable value.

[0014] According to another aspect of the present application, there is provided a data processing device for variable transformation of a binary classification statistical model, the device comprising: a group division unit for dividing a test sample into a plurality of groups in sequence, each of the plurality of groups having a group number; a proportion calculation unit for calculating the sample proportion of samples of a specific category in each group; a logarithm calculation unit for calculating the logarithmic probability ratio of the sample proportion; and a model building unit for performing univariate linear regression with the logarithmic probability ratio as the dependent variable and the group number as the independent variable to obtain a variable transformation model.

[0015] In the above data processing device, the group division unit is used to: determine the type of the test sample; and, in response to the test sample being a categorical predictor variable, set the test sample of each classification as a group.

[0016] In the above data processing device, the group division unit is further used to: sort the multiple groups based on the sample ratio of the group corresponding to each category.

[0017] In the above data processing device, the group division unit is further used to: in response to the test samples being ordinal predictor variables, sort the test samples in order; and set the test samples corresponding to each ordinal value as a group.

[0018] In the above-mentioned data processing device, the grouping unit is further used to: in response to the test sample being a continuous predictive variable, sort the test sample according to the predictive variable; divide the test sample into N groups in sequence, N is an integer greater than one; and set each of the N groups as a group.

[0019] In the above-mentioned data processing device, it further includes: a data input unit for obtaining input samples; a sample determination unit for determining whether the input sample is a missing sample or a non-missing sample; and a variable transformation unit for calculating the transformed variable value based on the variable transformation model in response to the input sample being a non-missing sample.

[0020] In the above-mentioned data processing device, the variable transformation unit is further used to: in response to the input sample being a missing sample, analyze the distribution of the target variable in the input sample to calculate the sample proportion of samples of a specific category in the input sample; and calculate the logarithmic probability ratio of the sample proportion as the transformed variable value.

[0021] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the data processing method as described above.

[0022] The data processing method, data processing device and electronic device provided in the present application can perform variable transformation of a binary classification statistical model through log-probability ratio linear regression based on sample proportions, thereby improving the generalization ability and stability of the variables. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the detailed description of the preferred specific embodiments below, various other advantages and benefits of the present application will become clear to those of ordinary skill in the art. The drawings in the specification are only used for the purpose of illustrating the preferred embodiments and are not considered to be limitations of the present application. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work. Moreover, the same reference numerals are used to represent the same components throughout the drawings.

[0024] Figure 1 A flowchart of a data processing method according to an embodiment of the present application is illustrated;

[0025] Figure 2 A schematic diagram illustrating the fitting results of the data in Table 1 is shown;

[0026] Figure 3 A schematic diagram illustrating the fitting results of the data in Table 2 is shown;

[0027] Figure 4 A schematic diagram illustrating the fitting results of the data in Table 3 is shown;

[0028] Figure 5 A block diagram of a data processing device according to an embodiment of the present application is illustrated;

[0029] Figure 6 A block diagram of an electronic device according to an embodiment of the present application is illustrated. DETAILED DESCRIPTION

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0031] Application Overview

[0032] As mentioned above, there are various variable transformation methods for different models. For example, the original variable can be directly transformed using a continuous function, such as squaring, square root, logarithm, etc. This method has the advantage of being simple and easy to implement, but it requires separate processing of missing samples and does not smooth noise. Alternatively, another method is to segment the variable and then process each segment into a 0 / 1 variable. This method can smooth noise and facilitate the processing of missing values, but it weakens the variable order information and cannot reflect the distribution of grouped samples.

[0033] In response to the above technical problems, the present application proposes a data processing method, a data processing device, and an electronic device, which divide the samples into multiple groups in sequence, calculate the sample proportion and logarithmic probability ratio of the specific category samples of each group, and use the logarithmic probability ratio as the dependent variable and the group number as the independent variable to perform univariate linear regression to obtain a variable transformation model. In this way, the generalization ability and stability of the transformed variables can be improved. In addition, since the variables are segmented, and then a linear model is established based on the order of each segment and the logarithmic probability ratio of the segmented samples, it reflects the distribution of the grouped samples and retains the order information of the original variables.

[0034] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0035] Exemplary Methods

[0036] Figure 1 The figure illustrates a flow chart of a data processing method according to an embodiment of the present application.

[0037] like Figure 1As shown, the data processing method according to the embodiment of the present application is used for variable transformation of a binary classification statistical model, including: S110, dividing the test sample into a plurality of groups in sequence, each of the plurality of groups having a group number; S120, calculating the sample proportion of samples of a specific category of each group; S130, calculating the logarithmic probability ratio of the sample proportion; and S140, performing univariate linear regression with the logarithmic probability ratio as the dependent variable and the group number as the independent variable to obtain a variable transformation model.

[0038] In step S110, the test sample is divided into a plurality of groups in sequence. Here, in the data processing method according to the embodiment of the present application, the test sample is a non-missing sample, and a non-missing sample refers to a sample whose all features are not missing. Specifically, assuming that the salary income of a customer group is to be collected, then for the variable "salary income", the customer group that cannot collect the variable is a missing sample, and the customer group that can be collected is a non-missing sample. In other words, the missingness of the sample is specific to a certain feature, that is, a sample with a missing feature. Because if all the features of a sample are missing, it is meaningless for analysis and is usually deleted as an invalid sample. In addition, the way of dividing the groups is different for different types of test samples, which will be explained in detail below.

[0039] First, for a categorical predictor variable, a group can be set for each category. That is, in the data processing method according to an embodiment of the present application, sequentially dividing the test sample into a plurality of groups includes: determining the type of the test sample; and, in response to the test sample being a categorical predictor variable, setting the test sample of each category as a group.

[0040] Furthermore, in the data processing method according to the embodiment of the present application, by calculating the sample ratio of the specific category samples of each group, the above groups can be further sorted according to the order of the sample ratio, for example, in the order of the sample ratio from small to large. That is, in the above data processing method, dividing the test sample into a plurality of groups in sequence further includes: sorting the plurality of groups based on the sample ratio of the group corresponding to each classification.

[0041] Here, because the categorical predictor variables themselves have no concept of order, for the convenience of processing, we use the specific category samples of each grouped sample, such as the proportion of "bad" samples, to sort them, so that all types of variables can be unified under the same framework for processing.

[0042] In addition, for ordinal predictor variables, the groups can be directly divided and sorted according to the order. That is, in the data processing method according to the embodiment of the present application, dividing the test samples into a plurality of groups in order further includes: in response to the test samples being ordinal predictor variables, sorting the test samples in order; and setting the test samples corresponding to each order value as a group.

[0043] In addition, for continuous predictive variables, the test samples may be first sorted according to the predictive variables, and then the test samples may be divided into a plurality of groups according to the order. That is, in the data processing method according to the embodiment, the test samples are divided into a plurality of groups in sequence further comprising: in response to the test samples being continuous predictive variables, the test samples are sorted according to the predictive variables; the test samples are divided into N groups in sequence, where N is an integer greater than one; and each of the N groups is set as a group. Here, the test samples are usually divided equally into N groups, and N is usually 10.

[0044] Next, in step S120, the sample proportion of the special category samples of each group is calculated. Here, the special category samples refer to samples of one category in the binary classification in the binary classification statistical model, and they vary depending on the statistical model. In the credit risk model and the fraud risk model, the samples are divided into samples with credit risk or fraud risk (also called "bad" samples) and samples without credit risk or fraud risk (also called "good" samples). Or, in the model for the hospital to determine whether the parturient is a high-risk parturient, the samples are divided into samples of high-risk parturients (also called "bad" samples) and samples of not high-risk parturients (also called "good" samples). Of course, those skilled in the art can understand that for different classifications in the binary classification statistical model, the samples are not necessarily divided into "good" samples and "bad" samples. For example, in the binary classification statistical model involving gender, they are simply divided into male samples and female samples.

[0045] In addition, in many cases, binary classification statistical models are used to predict relatively rare events, such as the above-mentioned credit risk model, fraud risk model, or model for judging high-risk pregnant women. Therefore, in the data processing method according to the embodiment of the present application, the specific category sample can also be a sample with a lower probability in the binary classification statistical model, such as the above-mentioned "bad" sample.

[0046] In step S130, the logarithmic probability ratio of the sample proportion is calculated.

[0047] Next, the above-mentioned sample grouping, sample ratio calculation and log probability ratio calculation processes will be described in detail in combination with the above-mentioned classification of sample types.

[0048] For example, if it is a categorical predictor variable, calculate the sample ratio of a specific type of sample for each category, such as the above-mentioned "bad" sample, and sort the categorical values ​​according to the sample ratio of "bad" samples. The ordinal number of each categorical value is represented by i, and the sample ratio corresponding to the categorical value is represented by p. i Denotes, where i = 1, ..., m, and m is the number of classification values. Next, calculate the log probability ratio of each sample proportion:

[0049]

[0050] For example, the example is shown in Table 1 below:

[0051]

Table 1

[0052] Buy Products Total sample "Bad" Sample Proportion of "bad" samples No.(i) <![CDATA[y i ]]> television 3171 190 6.00% 3 -2.75 cell phone 4382 329 7.50% 4 -2.51 refrigerator 3111 157 5.05% 2 -2.93 washing machine 1243 59 4.75% 1 -3.00

[0053] If it is an ordinal predictor variable, sort it according to the order, calculate the specific type of samples for each order value, such as the "bad" sample ratio mentioned above, and sort it according to the predictor variable. The ordinal number of each order value is represented by i, and the sample ratio corresponding to the order value is represented by p. i Denotes, where i = 1, ..., m, and m is the number of order values. Next, calculate the log probability ratio of each sample proportion:

[0054]

[0055] For example, the example is shown in Table 2 below:

[0056]

Table 2

[0057] Membership Level Total sample "Bad" Sample Proportion of "bad" samples No.(i) <![CDATA[y i ]]> Level 1 User 3171 190 6.00% 1 -2.75 Secondary User 4382 329 7.50% 2 -2.51 Level 3 User 3111 157 5.05% 3 -2.93 Level 4 User 1243 59 4.75% 4 -3.00

[0058] If it is a continuous predictor variable, the test samples are divided into N groups according to the order of the predictor variable, for example, 10 groups, and the specific type of samples in each group, such as the proportion of "bad" samples mentioned above, are calculated. The serial number of each group is represented by i, and the sample proportion corresponding to the serial number value is represented by p i Denotes, where i = 1, ..., m, and m is the number of groups. Next, calculate the logarithmic probability ratio of each sample proportion:

[0059]

[0060] For example, the example is shown in Table 3 below:

[0061]

Table 3

[0062]

[0063]

[0064] Next, in step S140, a univariate linear regression is performed with the log probability ratio as the dependent variable and the group number as the independent variable to obtain a variable transformation model. That is, according to the point pair (i, y i ) is fitted linearly, with the logarithmic probability ratio y i As the dependent variable and i as the independent variable, we can perform univariate linear regression and obtain For the data in Tables 1 to 3 above, linear fitting is performed respectively, and the fitting results are as follows: Figures 2 to 4 As shown. Among them, Figure 2 A schematic diagram showing the fitting results of the data in Table 1 is shown. Figure 3 A schematic diagram showing the fitting results of the data in Table 2 is shown, and Figure 4 A schematic diagram showing the fitting results of the data in Table 3 is shown.

[0065] After obtaining the variable transformation model, when obtaining a new variable, the variable transformation can be performed based on the variable transformation model to obtain the transformed variable value. Here, since the variable transformation model is obtained based on non-missing samples, when obtaining new samples, it is also necessary to determine whether the samples are non-missing samples.

[0066] That is, in the data processing method according to the embodiment of the present application, it further includes: obtaining an input sample; determining whether the input sample is a missing sample or a non-missing sample; and, in response to the input sample being a non-missing sample, calculating the transformed variable value based on the variable transformation model.

[0067] In addition, for missing samples, we can analyze the distribution of the target variable in the samples where the variable to be transformed is missing, calculate the specific category samples in the missing samples, such as the proportion of samples with the above-mentioned "bad" root, for example, expressed as p0, and then calculate the logarithmic probability ratio of the sample proportion p0, that is, In this way, the data processing method according to the embodiment of the present application includes a complete missing value processing mechanism.

[0068] That is, in the data processing method according to the embodiment of the present application, it further includes: in response to the input sample being a missing sample, analyzing the distribution of the target variable in the input sample to calculate the sample proportion of samples of a specific category in the input sample; and calculating the logarithmic probability ratio of the sample proportion as the transformed variable value.

[0069] Regarding the processing of outliers, specifically, since the outliers themselves are observable values, this value may be quite extreme. If the original values ​​are used directly for linear regression, it may have a relatively large impact on the regression results. The processing of outliers refers to eliminating the impact of such extreme values. Because in the data processing method according to the embodiment of the present application, the samples are first segmented, so that the extreme values ​​are divided into a certain section, and the sequence number of the segment is used to do linear regression, so as to focus only on the main trend and eliminate the impact of extreme values. Therefore, the data processing method according to the embodiment of the present application also realizes a good outlier processing mechanism.

[0070] Exemplary Devices

[0071] Figure 5 The figure shows a schematic block diagram of a data processing device according to an embodiment of the present application.

[0072] like Figure 5 As shown, the data processing device 200 according to the embodiment of the present application is used for variable transformation of a binary classification statistical model, and the device 200 includes: a group division unit 210, used to divide the test sample into a plurality of groups in sequence, each of the plurality of groups having a group number; a proportion calculation unit 220, used to calculate the sample proportion of samples of a specific category of each group divided by the group division unit 210; a logarithm calculation unit 230, used to calculate the logarithm probability ratio of the sample proportion calculated by the proportion calculation unit 220; and a model building unit 240, used to perform univariate linear regression with the logarithm probability ratio calculated by the logarithm calculation unit 230 as the dependent variable and the group number obtained by the group division unit 210 as the independent variable to obtain a variable transformation model.

[0073] In one example, in the above data processing device 200, the grouping unit 210 is used to: determine the type of the test sample; and, in response to the test sample being a categorical predictor variable, set the test sample of each classification as a group.

[0074] In one example, in the above data processing device 200, the group division unit 210 is further used to: sort the multiple groups based on the sample ratio of the group corresponding to each category.

[0075] In one example, in the above-mentioned data processing device 200, the grouping unit 210 is further used to: in response to the test sample being an ordinal predictor variable, sort the test sample in order; and set the test sample corresponding to each ordinal value as a group.

[0076] In one example, in the above-mentioned data processing device 200, the grouping unit 210 is further used to: in response to the test sample being a continuous predictive variable, sort the test sample according to the predictive variable; divide the test sample into N groups in sequence, N is an integer greater than one; and set each of the N groups as a group.

[0077] In one example, the above-mentioned data processing device 200 further includes: a data input unit for acquiring an input sample; a sample determination unit for determining whether the input sample is a missing sample or a non-missing sample; and a variable transformation unit for calculating a transformed variable value based on the variable transformation model in response to the input sample being a non-missing sample.

[0078] In one example, in the above-mentioned data processing device 200, the variable transformation unit is further used to: in response to the input sample being a missing sample, analyze the distribution of the target variable in the input sample to calculate the sample proportion of samples of a specific category in the input sample; and calculate the logarithmic probability ratio of the sample proportion as the transformed variable value.

[0079] Here, those skilled in the art will appreciate that the specific functions and operations of the various units and modules in the above-mentioned data processing device 200 have been described in detail above. Figures 1 to 4 The data processing method described is described in detail, and therefore, its repeated description will be omitted.

[0080] As described above, the data processing device 200 according to the embodiment of the present application can be implemented in various terminal devices, such as a server for a binary classification statistical model. In one example, the data processing device 200 according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the data processing device 200 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the data processing device 200 can also be one of the many hardware modules of the terminal device.

[0081] Alternatively, in another example, the data processing apparatus 200 and the terminal device may also be separate devices, and the data processing apparatus 200 may be connected to the terminal device via a wired and / or wireless network, and transmit the interactive information in accordance with an agreed data format.

[0082] Exemplary Electronic Devices

[0083] Below, reference Figure 6 To describe an electronic device according to an embodiment of the present application.

[0084] Figure 6 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0085] like Figure 6 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0086] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0087] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the data processing methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as sample data, group sequence number, sample ratio, logarithmic probability ratio, etc. may also be stored in the computer-readable storage medium.

[0088] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0089] For example, the input device 13 may be a keyboard, a mouse, etc.

[0090] The output device 14 can output various information to the outside, including transformed variables, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0091] Of course, to simplify, Figure 6 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0092] Exemplary computer program products and computer-readable storage media

[0093] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the data processing method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0094] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0095] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the data processing method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0096] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0097] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0098] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0099] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0100] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0101] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A data processing method for a credit evaluation model, used for variable transformation of a binary classification statistical model, the method include: Dividing the test sample into a plurality of groups in sequence, each of the plurality of groups having a group sequence number; Calculate the sample proportion of a specific category of samples for each group; calculating the log odds ratio of the sample proportion; as well as Performing univariate linear regression with the log probability ratio as the dependent variable and the group number as the independent variable to obtain a variable transformation model; The test samples are test samples used by the credit evaluation model, and the specific category samples are samples with credit risk; The test samples are divided into multiple groups in sequence, including: determining the type of the test sample; and In response to the test sample being a categorical predictor variable, the test sample of each classification is set as a group; the proportion of specific type samples of the group corresponding to each classification is calculated, and the multiple groups are sorted based on the proportion of specific type samples of the group corresponding to each classification.

2. The data processing method according to claim 1, further comprising: include: In response to the test sample being an ordinal predictor variable, sorting the test sample into order; as well as The test samples corresponding to each order value are set as a group.

3. The data processing method according to claim 1, further comprising: include: In response to the test sample being a continuous predictor variable, sorting the test sample according to the predictor variable; Sequentially dividing the test samples into N groups, where N is an integer greater than one; and Each of the N groups is set as a subgroup.

4. The data processing method according to claim 1, further comprising: include: Get input samples; Determining whether the input sample is a missing sample or a non-missing sample; as well as In response to the input sample being a non-missing sample, a transformed variable value is calculated based on the variable transformation model.

5. The data processing method according to claim 4, further comprising: include: In response to the input sample being a missing sample, analyzing the distribution of the target variable in the input sample to calculate the sample proportion of samples of a specific category in the input sample; as well as The log odds ratio of the sample proportions was calculated as the transformed variable value.

6. A data processing device for a credit evaluation model, used for variable transformation of a binary classification statistical model, the device include: A group division unit, used to divide the test sample into a plurality of groups in sequence, each of the plurality of groups having a group sequence number; A proportion calculation unit, used to calculate the sample proportion of samples of a specific category in each group; a logarithmic calculation unit, used to calculate the logarithmic probability ratio of the sample proportion; as well as A model building unit, used for performing univariate linear regression with the log probability ratio as the dependent variable and the group number as the independent variable to obtain a variable transformation model; The test samples are test samples used by the credit evaluation model, and the specific category samples are samples with credit risk; The grouping unit is used to: determine the type of the test sample; and in response to the test sample being a categorized predictor variable, set the test sample of each classification as a group, calculate the proportion of specific type samples of the group corresponding to each classification, and sort the multiple groups based on the proportion of specific type samples of the group corresponding to each classification.

7. The data processing device according to claim 6, in, The grouping unit is further used for: In response to the test sample being an ordinal predictor variable, sorting the test sample into an ordinal order; and The test samples corresponding to each order value are set as a group.

8. The data processing device according to claim 6, in, The grouping unit is further used for: In response to the test sample being a continuous predictor variable, sorting the test sample according to the predictor variable; Sequentially dividing the test samples into N groups, where N is an integer greater than one; and Each of the N groups is set as a subgroup.

9. The data processing device according to claim 6, further comprising: include: A data input unit, used for obtaining input samples; A sample determination unit, used to determine whether the input sample is a missing sample or a non-missing sample; as well as The variable transformation unit is used to calculate the transformed variable value based on the variable transformation model in response to the input sample being a non-missing sample.

10. The data processing device according to claim 9, in, The variable conversion unit is further used for: In response to the input sample being a missing sample, analyzing the distribution of the target variable in the input sample to calculate the sample proportion of samples of a specific category in the input sample; and The log odds ratio of the sample proportions was calculated as the transformed variable value.

11. An electronic device, include: processor; as well as A memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the data processing method according to any one of claims 1 to 5.