Method, device and storage medium for analyzing validity of newly added service data

Abstract

The application discloses a kind of effectiveness analysis method, equipment and storage medium of new service data.The method comprises the following steps: when detecting the evaluation request of new service data, determine new candidate features from the new service data, calculate the correlation coefficient between the candidate feature vector corresponding to new candidate features and the label vector of service model, and obtain the coefficient vector of new candidate features;The distance between the coefficient vector of new candidate features and each type of cluster center vector is calculated, and the target feature class cluster to which the new candidate features belong is determined, to determine the effectiveness of each service model according to the inventory features in the target feature class cluster.The calculation complexity of the effectiveness analysis method of new service data of the application is lower, and the effectiveness analysis efficiency of new service data can be improved.

Description

Technical Field

[0001] This invention relates to the field of feature engineering technology, and in particular to a method, device and storage medium for validity analysis of newly added business data. Background Technology

[0002] With the development of internet finance and big data technologies, the ways to acquire data are becoming increasingly diverse. The feature dimensions of enterprise financial information databases can reach tens of thousands. Directly applying business data to train each business model can lead to various problems such as overfitting, high model complexity, and large computational load. Therefore, how to quickly analyze the effectiveness of business data for different business models is a problem currently faced by the entire fintech industry.

[0003] As enterprise databases are constantly updated, it is necessary to continuously explore the effectiveness of newly added business data on business models and apply this data to the training of those models to maximize its impact. However, existing analytical methods for evaluating the effectiveness of new business data—that is, whether it has a positive effect on the training of the business model—require calculating the correlation between features in the new business data and existing features pairwise. The effectiveness of the new business data is then assessed based on the effectiveness of the existing features. Due to the large number of dimensions in the existing features, this analytical method has high time complexity, resulting in low efficiency in analyzing the effectiveness of new business data. Summary of the Invention

[0004] The main objective of this invention is to provide a method, device, and storage medium for analyzing the validity of newly added business data, aiming to solve the technical problem that current methods for analyzing the validity of newly added business data involve calculating the correlation between features in the newly added business data and existing feature vectors, resulting in high time complexity.

[0005] To achieve the above objectives, the present invention provides a method for validity analysis of newly added business data, the method comprising the following steps:

[0006] When an evaluation request for new business data is detected, new candidate features are determined from the new business data, feature values ​​of each first sample under the new candidate features are obtained from the new business data to form a candidate feature vector corresponding to the new candidate features, and label values ​​of each first sample under the target label of the business model are obtained to form a first label vector corresponding to the business model.

[0007] Calculate the first correlation coefficient between the candidate feature vector corresponding to the newly added candidate feature and the first label vector corresponding to the multiple business models respectively, and combine the multiple first correlation coefficients to form the coefficient vector corresponding to the newly added candidate feature.

[0008] The distance between the coefficient vector corresponding to the newly added candidate feature and the center vector of each cluster is calculated. The target feature cluster to which the newly added candidate feature belongs is determined based on the distance calculation result. The effectiveness of the newly added business data for each business model is determined based on the effectiveness of the existing features in the target feature cluster for each business model. The center vector of each cluster is obtained by clustering the coefficient vectors corresponding to each existing feature in the existing business data.

[0009] Optionally, before the step of determining new candidate features from the new business data when an evaluation request for new business data is detected, the method further includes:

[0010] Acquire existing business data and determine existing characteristics from the existing business data;

[0011] The existing business data is used to obtain the feature values ​​of each second sample under the existing feature to form the existing feature vector corresponding to the existing feature, and the label values ​​of each second sample under the target label of the business model are obtained to form the second label vector corresponding to the business model.

[0012] Calculate the second correlation coefficient between the existing feature vector corresponding to the existing feature and the second label vector corresponding to the multiple business models respectively, and combine the multiple second correlation coefficients to form the coefficient vector corresponding to the existing feature;

[0013] Cluster the coefficient vectors corresponding to the multiple stock features, divide each stock feature into multiple feature clusters, and obtain the cluster center vector corresponding to each feature cluster.

[0014] Optionally, the step of calculating the second correlation coefficient between the existing feature vector corresponding to the existing feature and the second label vector corresponding to the business model includes:

[0015] The first distance matrix is ​​obtained by calculating the pairwise distance between each element in the existing feature vector, and the second distance matrix is ​​obtained by calculating the pairwise distance between each element in the second label vector.

[0016] The third distance matrix is ​​obtained by subtracting the mean of the same row and the mean of the same column from each element of the first distance matrix and then adding the mean of the elements of the first distance matrix; and the fourth distance matrix is ​​obtained by subtracting the mean of the same row and the mean of the same column from each element of the second distance matrix and then adding the mean of the elements of the second distance matrix.

[0017] The first value is obtained by summing the squares of each element in the third distance matrix, dividing by the number of columns, and then calculating the square root. The second value is obtained by summing the squares of each element in the fourth distance matrix, dividing by the number of columns, and then calculating the square root. The third value is obtained by multiplying the third distance matrix and the fourth distance matrix, summing the elements, dividing by the number of columns, and then calculating the square root.

[0018] The square root of the product of the first value and the second value is calculated to obtain the fourth value. The third value is divided by the fourth value to obtain the second correlation coefficient between the stock feature vector and the second label vector.

[0019] Optionally, the step of clustering the coefficient vectors corresponding to the multiple existing features, dividing each existing feature into multiple feature clusters, and obtaining the cluster center vector corresponding to each feature cluster includes:

[0020] Initialize a preset number of cluster center vectors;

[0021] Calculate the distance between the coefficient vector corresponding to each of the existing features and the center vector of each of the clusters, and assign each of the existing features to the feature cluster corresponding to the nearest cluster center vector;

[0022] The coefficient vectors corresponding to the existing features in the feature clusters are averaged to obtain new cluster center vectors, and it is then checked whether the preset clustering termination condition is met.

[0023] If the clustering termination condition is met, then the clustering process ends.

[0024] If it is determined that the clustering termination condition is not met, then based on the new cluster center vector, the step of calculating the distance between the coefficient vector corresponding to the existing feature and each cluster center vector is returned.

[0025] Optionally, the step of obtaining the feature values ​​of each second sample under the existing feature from the existing business data to form the existing feature vector corresponding to the existing feature includes:

[0026] Obtain the feature values ​​of each second sample under the existing features from the existing business data;

[0027] The empty feature values ​​among the obtained feature values ​​are replaced with preset values ​​to form the stock feature vector corresponding to the stock feature.

[0028] Optionally, the step of determining the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result includes:

[0029] When the distance between the coefficient vector of the newly added candidate feature and one of the cluster center vectors is less than a preset value, the feature cluster corresponding to the cluster center vector whose distance to the coefficient vector of the newly added candidate feature is less than the preset value is taken as the target feature cluster to which the newly added candidate feature belongs.

[0030] When the distance between the coefficient vector of the newly added candidate feature and the multiple cluster center vectors is less than the preset value, the feature cluster corresponding to the cluster center vector with the smallest distance to the coefficient vector of the newly added candidate feature is taken as the target feature cluster to which the newly added candidate feature belongs.

[0031] When the distance between the coefficient vector of the newly added candidate feature and the center vector of each of the clusters is not less than the preset value, a new feature cluster is added as the target feature cluster to which the newly added candidate feature belongs.

[0032] Optionally, after determining the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result, the method further includes:

[0033] The newly added candidate features are added to the target feature cluster to increase the number of features in the target feature cluster;

[0034] When a feature selection request for a target business model is detected, the effect information of each of the existing features on each business model is obtained, and the target existing features that have a positive effect on the training of the target business model are determined from the existing features based on the effect information.

[0035] New candidate features are obtained from the feature clusters in which the target existing features are located and output as new features of the target business model.

[0036] Optionally, the step of determining new candidate features from the new business data and obtaining the feature values ​​of each first sample under the new candidate features to form the candidate feature vector corresponding to the new candidate features when an evaluation request for new business data is detected includes:

[0037] When an evaluation request for new business data is detected, multiple raw features are determined from the new business data.

[0038] New candidate features are obtained by transforming multiple original features using a preset cross-transformation method;

[0039] The feature values ​​of each first sample under the original feature are obtained from the newly added business data. The feature values ​​of each first sample under the original feature are calculated according to the transformation formula corresponding to the preset cross-transformation method to obtain the feature values ​​of each first sample under the newly added candidate feature.

[0040] The feature values ​​of each of the first samples under the newly added candidate feature are used to form the candidate feature vector corresponding to the newly added candidate feature.

[0041] To achieve the above objectives, the present invention also provides a device for analyzing the validity of newly added business data, the device comprising:

[0042] The acquisition module is used to determine new candidate features from the new business data when an evaluation request for new business data is detected, to obtain the feature values ​​of each first sample under the new candidate features from the new business data to form a candidate feature vector corresponding to the new candidate features, and to obtain the label values ​​of each first sample under the target label of the business model to form a first label vector corresponding to the business model.

[0043] The calculation module is used to calculate the first correlation coefficient between the candidate feature vector corresponding to the newly added candidate feature and the first label vector corresponding to the multiple business models, and to form the coefficient vector corresponding to the newly added candidate feature by combining the multiple first correlation coefficients.

[0044] The analysis module is used to calculate the distance between the coefficient vector corresponding to the newly added candidate feature and the cluster center vectors of each type, and determine the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result, so as to determine the effectiveness of the newly added business data for each of the business models based on the effectiveness of the existing features in the target feature clusters for each of the business models. The cluster center vectors are obtained by clustering based on the coefficient vectors corresponding to each existing feature in the existing business data.

[0045] To achieve the above objectives, the present invention also provides a device for analyzing the validity of newly added business data. The device includes a memory, a processor, and a program for analyzing the validity of newly added business data stored in the memory and executable on the processor. When the program for analyzing the validity of newly added business data is executed by the processor, it implements the steps of the method for analyzing the validity of newly added business data as described above.

[0046] Furthermore, to achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a validity analysis program for newly added business data, wherein when the validity analysis program for newly added business data is executed by a processor, it implements the steps of the validity analysis method for newly added business data as described above.

[0047] In this invention, when an evaluation request for new business data is detected, new candidate features are determined from the new business data. Candidate feature vectors corresponding to the new candidate features and first label vectors corresponding to the business models are obtained. First correlation coefficients are calculated between the candidate feature vectors corresponding to the new candidate features and the first label vectors corresponding to multiple business models. These first correlation coefficients are then combined to form a coefficient vector corresponding to the new candidate features. The distance between the coefficient vectors corresponding to the new candidate features and the cluster center vectors of various types is calculated. These cluster center vectors are obtained by clustering the coefficient vectors corresponding to each existing feature in the existing business data. Based on the distance calculation results, the target feature cluster to which the new candidate features belong is determined. The effectiveness of the new business data for each business model is determined based on the effectiveness of the existing features in the target feature cluster for each business model. Finally, feature selection is performed based on the effectiveness of the new business data for the business models. Compared to existing analysis methods that evaluate the effectiveness of new candidate features by calculating the correlation between each pair of new candidate features and existing features in new business data, this invention pre-calculates coefficient vectors that characterize the performance of each existing feature across different business models. These coefficient vectors are then clustered to divide existing features into different classes. When evaluating the effectiveness of new candidate features, the method simply calculates the distance between the feature vector of the new candidate feature and the center vector of each cluster to assign the new candidate feature to its respective feature cluster. The effectiveness of the new business data for the business model can then be determined by evaluating the effectiveness of existing features within their respective clusters. Since the number of models is much smaller than the number of features, the computational cost of calculating the distance between the candidate feature vector and the cluster center vector is far less than the computational cost of calculating the correlation between each pair of new candidate features and existing features. Therefore, the method for analyzing the effectiveness of new business data in this invention has lower computational complexity and higher efficiency. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention;

[0049] Figure 2 This is a flowchart illustrating the first embodiment of the method for analyzing the validity of newly added business data according to the present invention.

[0050] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0051] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0052] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0053] It should be noted that the device for analyzing the validity of the new business data in this embodiment of the invention can be a smartphone, a personal computer, a server, or other such device, and no specific limitations are imposed here.

[0054] like Figure 1 As shown, the device for analyzing the validity of newly added business data may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or stable non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art will understand that Figure 1 The equipment structure shown does not constitute a limitation on the equipment for analyzing the validity of new business data. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0056] like Figure 1 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a program for validating new business data. The operating system is a program that manages and controls the hardware and software resources of the device, supporting the execution of the program for validating new business data and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the validity analysis program for newly added business data stored in the memory 1005 and perform the following operations:

[0057] When an evaluation request for new business data is detected, new candidate features are determined from the new business data, feature values ​​of each first sample under the new candidate features are obtained from the new business data to form a candidate feature vector corresponding to the new candidate features, and label values ​​of each first sample under the target label of the business model are obtained to form a first label vector corresponding to the business model.

[0058] Calculate the first correlation coefficient between the candidate feature vector corresponding to the newly added candidate feature and the first label vector corresponding to the multiple business models respectively, and combine the multiple first correlation coefficients to form the coefficient vector corresponding to the newly added candidate feature.

[0059] The distance between the coefficient vector corresponding to the newly added candidate feature and the center vector of each cluster is calculated. The target feature cluster to which the newly added candidate feature belongs is determined based on the distance calculation result. The effectiveness of the newly added business data for each business model is determined based on the effectiveness of the existing features in the target feature cluster for each business model. The center vector of each cluster is obtained by clustering the coefficient vectors corresponding to each existing feature in the existing business data.

[0060] Furthermore, before determining new candidate features from the new service data when an evaluation request for new service data is detected, the processor 1001 can also call the validity analysis program for the new service data stored in the memory 1005 to perform the following operations:

[0061] Acquire existing business data and determine existing characteristics from the existing business data;

[0062] The existing business data is used to obtain the feature values ​​of each second sample under the existing feature to form the existing feature vector corresponding to the existing feature, and the label values ​​of each second sample under the target label of the business model are obtained to form the second label vector corresponding to the business model.

[0063] Calculate the second correlation coefficient between the existing feature vector corresponding to the existing feature and the second label vector corresponding to the multiple business models respectively, and combine the multiple second correlation coefficients to form the coefficient vector corresponding to the existing feature;

[0064] Cluster the coefficient vectors corresponding to the multiple stock features, divide each stock feature into multiple feature clusters, and obtain the cluster center vector corresponding to each feature cluster.

[0065] Further, the operation of calculating the second correlation coefficient between the existing feature vector corresponding to the existing feature and the second label vector corresponding to the business model includes:

[0066] The first distance matrix is ​​obtained by calculating the pairwise distance between each element in the existing feature vector, and the second distance matrix is ​​obtained by calculating the pairwise distance between each element in the second label vector.

[0067] The third distance matrix is ​​obtained by subtracting the mean of the same row and the mean of the same column from each element of the first distance matrix and then adding the mean of the elements of the first distance matrix; and the fourth distance matrix is ​​obtained by subtracting the mean of the same row and the mean of the same column from each element of the second distance matrix and then adding the mean of the elements of the second distance matrix.

[0068] The first value is obtained by summing the squares of each element in the third distance matrix, dividing by the number of columns, and then calculating the square root. The second value is obtained by summing the squares of each element in the fourth distance matrix, dividing by the number of columns, and then calculating the square root. The third value is obtained by multiplying the third distance matrix and the fourth distance matrix, summing the elements, dividing by the number of columns, and then calculating the square root.

[0069] The square root of the product of the first value and the second value is calculated to obtain the fourth value. The third value is divided by the fourth value to obtain the second correlation coefficient between the stock feature vector and the second label vector.

[0070] Furthermore, the operation of clustering the coefficient vectors corresponding to the multiple existing features, dividing each existing feature into multiple feature clusters, and obtaining the cluster center vector corresponding to each feature cluster includes:

[0071] Initialize a preset number of cluster center vectors;

[0072] Calculate the distance between the coefficient vector corresponding to each of the existing features and the center vector of each of the clusters, and assign each of the existing features to the feature cluster corresponding to the nearest cluster center vector;

[0073] The coefficient vectors corresponding to the existing features in the feature clusters are averaged to obtain new cluster center vectors, and it is then checked whether the preset clustering termination condition is met.

[0074] If the clustering termination condition is met, then the clustering process ends.

[0075] If it is determined that the clustering termination condition is not met, then based on the new cluster center vector, the operation of calculating the distance between the coefficient vector corresponding to the existing feature and each cluster center vector is returned.

[0076] Furthermore, the operation of obtaining the feature values ​​of each second sample under the existing feature from the existing business data to form the existing feature vector corresponding to the existing feature includes:

[0077] Obtain the feature values ​​of each second sample under the existing features from the existing business data;

[0078] The empty feature values ​​among the obtained feature values ​​are replaced with preset values ​​to form the stock feature vector corresponding to the stock feature.

[0079] Furthermore, the operation of determining the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result includes:

[0080] When the distance between the coefficient vector of the newly added candidate feature and one of the cluster center vectors is less than a preset value, the feature cluster corresponding to the cluster center vector whose distance to the coefficient vector of the newly added candidate feature is less than the preset value is taken as the target feature cluster to which the newly added candidate feature belongs.

[0081] When the distance between the coefficient vector of the newly added candidate feature and the multiple cluster center vectors is less than the preset value, the feature cluster corresponding to the cluster center vector with the smallest distance to the coefficient vector of the newly added candidate feature is taken as the target feature cluster to which the newly added candidate feature belongs.

[0082] When the distance between the coefficient vector of the newly added candidate feature and the center vector of each of the clusters is not less than the preset value, a new feature cluster is added as the target feature cluster to which the newly added candidate feature belongs.

[0083] Furthermore, after determining the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result, the processor 1001 can also call the validity analysis program for the newly added business data stored in the memory 1005 to perform the following operations:

[0084] The newly added candidate features are added to the target feature cluster to increase the number of features in the target feature cluster;

[0085] When a feature selection request for a target business model is detected, the effect information of each of the existing features on each business model is obtained, and the target existing features that have a positive effect on the training of the target business model are determined from the existing features based on the effect information.

[0086] New candidate features are obtained from the feature clusters in which the target existing features are located and output as new features of the target business model.

[0087] Furthermore, the operation of determining new candidate features from the new business data and obtaining the feature values ​​of each first sample under the new candidate features to form the candidate feature vector corresponding to the new candidate features when an evaluation request for new business data is detected includes:

[0088] When an evaluation request for new business data is detected, multiple raw features are determined from the new business data.

[0089] New candidate features are obtained by transforming multiple original features using a preset cross-transformation method;

[0090] The feature values ​​of each first sample under the original feature are obtained from the newly added business data. The feature values ​​of each first sample under the original feature are calculated according to the transformation formula corresponding to the preset cross-transformation method to obtain the feature values ​​of each first sample under the newly added candidate feature.

[0091] The feature values ​​of each of the first samples under the newly added candidate feature are used to form the candidate feature vector corresponding to the newly added candidate feature.

[0092] Based on the above structure, various embodiments of the method for analyzing the validity of newly added business data are proposed.

[0093] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the method for analyzing the validity of newly added business data according to the present invention.

[0094] This invention provides an embodiment of a method for validating newly added business data. It should be noted that although the flowchart shows a logical order, in some cases, the steps shown or described may be executed in a different order. In this embodiment, the executing entity of the method for validating newly added business data can be a smartphone, personal computer, server, or other device. For ease of description, the execution entity is omitted from the following description of each embodiment. The method for validating newly added business data includes:

[0095] Step S10: When an evaluation request for new business data is detected, new candidate features are determined from the new business data, feature values ​​of each first sample under the new candidate features are obtained from the new business data to form a candidate feature vector corresponding to the new candidate features, and label values ​​of each first sample under the target label of the business model are obtained to form a first label vector corresponding to the business model.

[0096] Business data is data generated during business processing. Business data includes feature values ​​of multiple samples under at least one characteristic, or feature values ​​of multiple samples under at least one characteristic can be extracted from business data. Newly generated business data can be called new business data. For new business data, we can analyze which business models it positively affects, so that we can use the new business data in those models; that is, we can analyze the effectiveness of the new business data on each business model.

[0097] A business model can refer to a model built using machine learning methods such as neural networks and decision trees, or manually based on certain features, for prediction or classification tasks in business operations. Business data already used or attempted during the business model building process is considered stock business data. Features within this stock business data are called stock features, and a single stock feature is typically applied to different business models. During the business model building process, methods can be used to determine whether the stock features have a positive or negative effect on the model's training, thus providing information on their impact on the business model. Generally, stock features with a positive effect are retained for business model building, while those with a negative effect are discarded. A positive effect typically means that the business model achieves higher accuracy in prediction or classification tasks, while a negative effect generally means that the feature is redundant, not only failing to improve accuracy but also making the business model more complex and redundant.

[0098] When new business data is added, and an evaluation request for that new business data is required for validity analysis, it can be triggered. When an evaluation request for new business data is detected, a feature can be identified from the new business data as a candidate feature for addition.

[0099] New candidate features can be selected directly from the various features (hereinafter referred to as original features) contained in the new business data, or they can be derived from the original features in the new business data through feature derivation operations such as cross-transformation and dummy coding. For example, in a user behavior recommendation model, the number of times a user performs an action each day is multiplied by the day of the week in which that day occurs, and this newly generated value is used as a new feature. As another example, in a business behavior recommendation model, the city and industry of a business are a series of Chinese names. When this feature is input into the model, these series of Chinese names need to be replaced with corresponding numerical values. A common method is dummy coding, which allows the model to effectively distinguish the relationship between businesses and cities.

[0100] After identifying new candidate features, feature values ​​for each sample (hereinafter referred to as the second sample) under the new candidate feature can be obtained from the new business data. These feature values ​​are then used to construct the candidate feature vector corresponding to the new candidate feature. A feature value is the specific value of a feature; the feature value of a sample under a feature refers to the specific value of that sample under that feature. For example, for the feature "age," different users are different samples. If a user's age is 23, then "23 years old" is the feature value of that user under the "age" feature. The dimension of the candidate feature vector, that is, the number of elements, is the number of the first samples. In other words, the number of feature values ​​corresponds to the number of first samples, and the number of elements in the candidate feature vector corresponds to the number of feature values.

[0101] For a newly added candidate feature, some samples in the newly added business data may have null feature values ​​under the newly added candidate feature, that is, no feature values. In one implementation, only the samples in the newly added business data whose feature values ​​are not null under the newly added candidate feature can be selected as the first samples, and then the feature values ​​of each first sample can be used to form a candidate feature vector.

[0102] Further, the label values ​​of each first sample under the target label of the business model are obtained, and the label vector corresponding to the business model is composed of these label values ​​(hereinafter referred to as the first label vector for distinction). The same method is used to obtain the first label vector for each business model across multiple business models. The following example uses one business model. The target label is a label related to the prediction or classification task of the business model, referring to the result that the business model needs to obtain by processing feature data. For example, if the business model is a deposit potential demand recommendation model, the target label is a label indicating whether a user has a deposit need. The target label value is "Y" indicating a deposit need, and "N" indicating no deposit need. Assuming there are two thousand first samples, representing two thousand customers, the first label vector composed of the label values ​​of each first sample under the target label is a two-dimensional array consisting of two thousand "Y" or "N" values. The label values ​​of the first samples under the target label can come from newly added business data.

[0103] It should be noted that the feature values ​​in the candidate feature vector and the label values ​​at the same position in the first label vector come from the same sample, in order to ensure that the two vectors can be used to calculate the correlation coefficient.

[0104] Step S20: Calculate the first correlation coefficient between the candidate feature vector corresponding to the newly added candidate feature and the first label vector corresponding to the multiple business models, and combine the multiple first correlation coefficients to form the coefficient vector corresponding to the newly added candidate feature.

[0105] After obtaining the candidate feature vector corresponding to the newly added candidate feature and the first label vectors corresponding to multiple business models, the correlation coefficients (hereinafter referred to as first correlation coefficients) between the candidate feature vector and each first label vector can be calculated, that is, multiple first correlation coefficients are calculated. The method for calculating the first correlation coefficients between the candidate feature vector and the first label vector can be a conventional correlation coefficient calculation method, such as Pearson correlation coefficient, distance correlation coefficient, etc., and is not limited in this embodiment. The calculated multiple first correlation coefficients are combined into a vector, which is the coefficient vector of the newly added candidate feature pair. It can be understood that each element in the coefficient vector of the newly added candidate feature is the correlation coefficient between the newly added candidate feature and the target labels of different business models, which can be regarded as the performance characteristics of the newly added candidate feature on different business models.

[0106] Step S30: Calculate the distance between the coefficient vector corresponding to the newly added candidate feature and the center vector of each cluster. Determine the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result. Determine the effectiveness of the newly added business data for each business model based on the effectiveness of the existing features in the target feature cluster for each business model. The center vector of each cluster is obtained by clustering the coefficient vectors corresponding to each existing feature in the existing business data.

[0107] After obtaining the coefficient vectors corresponding to the newly added candidate features, the distances between these vectors and the center vectors of each cluster can be calculated. Based on the distance calculation results, the feature cluster to which the newly added candidate feature belongs (hereinafter referred to as the target feature cluster) will be determined. The distance between the coefficient vectors and the center vectors of the clusters can be Euclidean distance. The closer the distance, the more similar the performance characteristics of the newly added candidate feature on different business models are to the performance characteristics of each existing feature in the feature cluster corresponding to the center vector on different business models. Specifically, in one embodiment, after calculating the distances between the coefficient vectors of the newly added candidate features and the center vectors of each cluster, the feature cluster corresponding to the closest center vector can be selected as the target feature cluster. That is, the performance characteristics of the newly added candidate feature are most similar to those of each existing feature in the target feature cluster on different business models.

[0108] After identifying the target feature clusters, the results showing that newly added candidate features belong to the target feature clusters can be output. This allows business personnel to determine the effectiveness of new business data for each business model based on the effectiveness of existing features within the target feature clusters. Furthermore, based on the effectiveness of the new business data for each business model, they can determine whether to apply the new business data to each business model. Specifically, when existing features in the target feature cluster have a positive effect on the training of a certain business model, it can be determined that the new business data also has a positive effect on that business model, and it can be applied to that business model. Conversely, when existing features in the target feature cluster have a negative effect on the training of a certain business model, it can be determined that the new business data also has a negative effect on that business model, and it is not suitable to apply the new business data to that business model.

[0109] Alternatively, after determining the target feature cluster, new candidate features can be added to the target feature cluster. When feature selection is needed for a specific business model, the feature closest to the cluster center vector can be selected from the feature cluster for training the business model.

[0110] Furthermore, in one embodiment, the method further includes:

[0111] Step S40: Obtain existing business data and determine existing characteristics from the existing business data;

[0112] Step S50: Obtain the feature values ​​of each second sample under the existing feature from the existing business data to form the existing feature vector corresponding to the existing feature, and obtain the label values ​​of each second sample under the target label of the business model to form the second label vector corresponding to the business model.

[0113] In this embodiment, the method for obtaining the cluster center vector can specifically be as follows: first, obtain existing business data; then, determine existing features from the existing business data; there are multiple existing features. Existing business data refers to business data that has been used or attempted to be used during the business model construction process.

[0114] For each existing feature, the feature values ​​of each sample (hereinafter referred to as the second sample) under that existing feature are obtained from the existing business data, and the existing feature vector corresponding to that existing feature is constructed using the various feature values. The existing business data can be the same or different for different existing features; the second samples can also be the same or different for different existing features. The specific method for obtaining the existing feature vector of an existing feature can be referred to the method for obtaining the candidate feature vector described above.

[0115] For a given existing feature, some samples in the existing business data may have null feature values ​​under that feature, meaning they have no feature values. In one implementation, only samples in the existing business data whose feature values ​​are not null under that existing feature can be selected as second samples, and then the feature values ​​of each second sample can be used to form the existing feature vector.

[0116] Furthermore, in another embodiment, when it is not limited to selecting only samples from the existing business data whose feature values ​​are not empty under the existing feature as the second samples, the step S50 of obtaining the feature values ​​of each second sample from the existing business data under the existing feature to form the existing feature vector corresponding to the existing feature includes:

[0117] Step S501: Obtain the feature values ​​of each second sample under the existing features from the existing business data;

[0118] Step S502: Replace the empty feature values ​​in each of the obtained feature values ​​with preset values ​​to form the stock feature vector corresponding to the stock feature.

[0119] First, the feature values ​​of each second sample under the existing feature are obtained from the existing business data. At this point, the second sample can come from all samples in the existing business data, or from a subset of samples selected by random sampling or other methods. Some of the second samples may contain samples with empty feature values ​​under the existing feature. The empty feature values ​​are then replaced with preset values. That is, the preset value is used as the feature value of the sample corresponding to the empty feature value under the existing feature. The replaced feature value and the remaining unchanged feature values ​​are used to form the existing feature vector. The preset value can be a value set in advance based on experience, or it can be the mean or median of other non-empty feature values.

[0120] Furthermore, the label values ​​of each second sample under the target label of the business model are obtained, and the label vector corresponding to the business model is composed of these label values ​​(hereinafter referred to as the second label vector for distinction). The business model can be the business model applied to the existing feature. It is understood that when the second sample is the same as the first sample, the first label vector and the second label vector corresponding to the same business model are the same. The method for obtaining the second label vector can be referred to the method for obtaining the first label vector.

[0121] It should be noted that the feature values ​​in the existing feature vector and the label values ​​at the same position in the second label vector come from the same sample, in order to ensure that the two vectors can be used to calculate the correlation coefficient.

[0122] Step S60: Calculate the second correlation coefficient between the stock feature vector corresponding to the stock feature and the second label vector corresponding to the multiple business models, and combine the multiple second correlation coefficients into a coefficient vector corresponding to the stock feature.

[0123] After obtaining the existing feature vector corresponding to the existing feature and the second label vectors corresponding to multiple business models, the correlation coefficients (hereinafter referred to as second correlation coefficients) between the existing feature vector and each second label vector can be calculated, that is, multiple second correlation coefficients are calculated. The calculation method for the second correlation coefficients can refer to the calculation method for the first correlation coefficient. The calculated multiple second correlation coefficients are combined into a vector, which is the coefficient vector corresponding to the existing feature. It can be understood that each element in the coefficient vector of the existing feature is the correlation coefficient between the existing feature and the target labels of different business models, which can be seen as the performance characteristics of the existing feature in different business models.

[0124] When there are multiple stock features, the coefficient vector corresponding to each stock feature is calculated using the same method described above. For the i-th stock feature, its coefficient vector can be represented as v. i = <v i1 v i2 , ..., v ij >, where v ij This represents the correlation coefficient obtained for the i-th stock feature on the j-th business model.

[0125] Step S70: Cluster the coefficient vectors corresponding to the multiple stock features, divide each stock feature into multiple feature clusters, and obtain the cluster center vector corresponding to each feature cluster.

[0126] After obtaining the coefficient vectors corresponding to multiple existing features, clustering can be performed on each coefficient vector to divide each existing feature into multiple feature clusters, and the cluster center vector corresponding to each feature cluster can be obtained. The clustering method can be a conventional clustering method, and this embodiment is not limited to any particular method.

[0127] The correlation coefficient of each existing feature is different on different business models, which means that the existing features have different effects on different business models. In this embodiment, the existing features are divided into multiple feature clusters based on the numerical performance of the coefficient vector. When training the business model in the next time, only the feature closest to the center vector of the cluster needs to be selected for training, which can greatly reduce the number of features in the model and solve the problem of the curse of dimensionality, such as sparse data samples and difficulty in distance calculation in high-dimensional cases.

[0128] Further, in one embodiment, the step S30 of determining the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result includes:

[0129] Step S301: When the distance between the coefficient vector of the newly added candidate feature and one of the cluster center vectors is less than a preset value, the feature cluster corresponding to the cluster center vector whose distance to the coefficient vector of the newly added candidate feature is less than the preset value is taken as the target feature cluster to which the newly added candidate feature belongs.

[0130] After calculating the distances between the coefficient vectors of newly added candidate features and the center vectors of each cluster, the distance values ​​can be compared with a preset value. This preset value can be set as needed. When the distance between the coefficient vector of a newly added candidate feature and the center vector of a cluster is greater than this preset value, it is considered that the newly added candidate feature cannot belong to the feature cluster corresponding to that cluster center vector and has no correlation with any existing features in that feature cluster.

[0131] When the distance between the coefficient vector of a newly added candidate feature and the center vector of only one of the clusters is less than a preset value, the feature cluster corresponding to the center vector of that cluster can be directly used as the target feature cluster to which the newly added candidate feature belongs.

[0132] Step S302: When the distance between the coefficient vector of the newly added candidate feature and the multiple cluster center vectors of each cluster is less than the preset value, the feature cluster corresponding to the cluster center vector with the smallest distance to the coefficient vector of the newly added candidate feature is taken as the target feature cluster to which the newly added candidate feature belongs.

[0133] When the distance between the coefficient vector of the newly added candidate feature and the center vectors of multiple clusters is less than a preset value, the feature cluster corresponding to the center vector of the cluster with the smallest distance to the coefficient vector of the newly added candidate feature can be selected as the target feature cluster to which the newly added candidate feature belongs.

[0134] Step S303: When the distance between the coefficient vector of the newly added candidate feature and the center vector of each of the clusters is not less than the preset value, a new feature cluster is added as the target feature cluster to which the newly added candidate feature belongs.

[0135] When the distance between the coefficient vector of a newly added candidate feature and the center vectors of each cluster is less than a preset value, it indicates that the newly added candidate feature does not belong to any of the existing feature clusters. In this case, a new feature cluster can be added as the target feature cluster to which the newly added candidate feature belongs. Since there are currently no existing features in the target feature cluster, the effectiveness of the newly added candidate feature for a business model can be determined based on the effectiveness of existing features in other feature clusters for that business model. Specifically, if existing features in other feature clusters have a positive effect on the training of a business model, it can be determined that the newly added candidate feature has a negative effect on that business model and is not suitable for application; conversely, if existing features in other feature clusters have a negative effect on the training of a business model, it can be determined that the newly added candidate feature has a positive effect on that business model and can be applied. Alternatively, when a new existing feature is added to the target feature cluster, the effectiveness of the newly added candidate feature for the business model can be determined based on the effectiveness of the newly added existing feature for that business model.

[0136] In this embodiment, when an evaluation request for new business data is detected, new candidate features are determined from the new business data, and candidate feature vectors corresponding to the new candidate features and first label vectors corresponding to the business models are obtained. First correlation coefficients are calculated between the candidate feature vectors corresponding to the new candidate features and the first label vectors corresponding to multiple business models, and the multiple first correlation coefficients are combined to form a coefficient vector corresponding to the new candidate features. The distance between the coefficient vectors corresponding to the new candidate features and the cluster center vectors of each type is calculated. The cluster center vectors of each type are obtained by clustering the coefficient vectors corresponding to each existing feature in the existing business data. The target feature cluster to which the new candidate features belong is determined based on the distance calculation results. The effectiveness of the new business data for each business model is determined based on the effectiveness of the existing features in the target feature cluster for each business model. Then, feature selection is performed based on the effectiveness of the new business data for the business model. Compared to existing analysis methods that evaluate the effectiveness of new candidate features by calculating the correlation between each pair of new candidate features and existing features in the new business data, this embodiment pre-calculates coefficient vectors that characterize the performance of each existing feature on different business models. These coefficient vectors are then clustered to divide the existing features into different classes. When evaluating the effectiveness of new candidate features, only the distance between the feature vector of the new candidate feature and the center vector of each cluster needs to be calculated to classify the new candidate feature into its respective feature cluster. The effectiveness of the new business data for the business model can then be determined by evaluating the effectiveness of the existing features within their respective feature clusters. Since the number of models is much smaller than the number of features, the computational cost of calculating the distance between the candidate feature vector and the cluster center vector is far less than the computational cost of calculating the correlation between each pair of new candidate features and existing features. Therefore, the method for analyzing the effectiveness of new business data in this embodiment has lower computational complexity and higher efficiency.

[0137] Furthermore, based on the first embodiment described above, a second embodiment of the method for analyzing the validity of newly added business data according to the present invention is proposed. In this embodiment, the step of calculating the second correlation coefficient between the existing feature vector corresponding to the existing feature and the second label vector corresponding to the business model in step S60 includes:

[0138] Step S601: Calculate the pairwise distance between each element in the existing feature vector to obtain a first distance matrix, and calculate the pairwise distance between each element in the second label vector to obtain a second distance matrix;

[0139] In this embodiment, a method for calculating the second correlation coefficient between the existing feature vector and the second label vector is proposed. Specifically, the distance between each pair of elements in the existing feature vector can be calculated to obtain a matrix (hereinafter referred to as the first distance matrix). It can be understood that the diagonal elements in the first distance matrix are 0, the element in the first row and second column is the value of the second element minus the first element in the existing feature vector, the element in the first row and third column is the value of the third element minus the first element in the existing feature vector, and so on... the element in the second row and first column is the value of the first element minus the second element in the existing feature vector, the element in the third row and first column is the value of the first element minus the third element in the existing feature vector, and so on. Following the same method, the distance between each pair of elements in the second label vector is calculated to obtain a matrix (hereinafter referred to as the second distance matrix).

[0140] Step S602: Subtract the mean of the same row and the mean of the same column from each element in the first distance matrix and add the mean of the elements in the first distance matrix to obtain the third distance matrix; and subtract the mean of the same row and the mean of the same column from each element in the second distance matrix and add the mean of the elements in the second distance matrix to obtain the fourth distance matrix.

[0141] After obtaining the first distance matrix, the mean of each row and the mean of each column in the first distance matrix can be calculated, as well as the mean of all elements in the first distance matrix (hereinafter referred to as the element mean). For each element in the first distance matrix, the mean of the same row and the mean of the same column are subtracted from the element, where the mean of the same row is the mean of the row containing the element, and the mean of the same column is the mean of the column containing the element; after subtracting the mean of the same row and the mean of the same column from the element, the element mean of the first distance matrix is ​​added back; after calculating for each element, a new matrix (hereinafter referred to as the third distance matrix) is obtained.

[0142] After obtaining the second distance matrix, a new matrix (hereinafter referred to as the fourth distance matrix) is obtained by calculating each element in the second distance matrix in the same way.

[0143] Step S603: Sum the squares of each element in the third distance matrix, divide by the number of columns, and then calculate the square root to obtain the first value; sum the squares of each element in the fourth distance matrix, divide by the number of columns, and then calculate the square root to obtain the second value; multiply the third distance matrix and the fourth distance matrix, sum the elements, divide by the number of columns, and then calculate the square root to obtain the third value.

[0144] After obtaining the third distance matrix, we can calculate the square of each element in the third distance matrix, then sum the squares of each element, divide the sum by the number of columns in the third distance matrix (that is, the number of elements in the existing feature vector), and then calculate the square root to obtain a value (hereinafter referred to as the first value).

[0145] After obtaining the fourth distance matrix, the same method is used to calculate a value for each element in the fourth distance matrix (hereinafter referred to as the second value).

[0146] Furthermore, multiplying the third distance matrix by the fourth distance matrix yields a matrix. Summing the elements in this matrix and dividing by the number of columns, then calculating the square root, yields a numerical value (hereinafter referred to as the third value).

[0147] Step S604: Calculate the square root of the product of the first value and the second value to obtain the fourth value, and divide the third value by the fourth value to obtain the second correlation coefficient between the stock feature vector and the second label vector.

[0148] After calculating the first, second, and third values, the first and second values ​​can be multiplied together and the square root is calculated to obtain a fourth value. The result of dividing the third value by the fourth value is the second correlation coefficient between the existing feature vector and the second label vector. The larger the second correlation coefficient, the higher the correlation between the existing features and the target label.

[0149] The first correlation coefficient between the candidate feature vector and the first label vector can also be calculated using the same method described above. By calculating the second correlation coefficient between the existing feature vector and the second label vector, and using this second correlation coefficient to form the coefficient vector of the existing features, the existing features are clustered according to the coefficient vector, dividing them into different feature clusters. The first correlation coefficient between the candidate feature vector and the first label vector is then calculated, and this first correlation coefficient is used to form the coefficient vector of the new candidate features. The distance between the coefficient vector and the cluster center vector of the feature cluster is calculated, and the new candidate features are assigned to the target feature cluster based on the distance. Through this process, the implicit correlation between the new candidate features and each existing feature in the target feature cluster can be quickly discovered. This allows for the rapid determination of the effectiveness of the new candidate features for each business model based on each existing feature in the target feature cluster, improving the efficiency of feature selection.

[0150] Further, in one embodiment, step S70 includes:

[0151] Step S701: Initialize a preset number of cluster center vectors;

[0152] This embodiment proposes a method for clustering coefficient vectors of existing features. Specifically, a preset number of cluster center vectors can be initialized first. The preset number can be set as needed; a larger number can be set when more detailed feature segmentation is required to improve the accuracy of validity analysis of new business data, and a smaller number can be set when higher efficiency is required for validity analysis of new business data. Initializing the cluster center vectors can be achieved by randomly selecting a preset number of coefficient vectors from the coefficient vectors of each existing feature as the initial cluster center vectors. Each cluster center vector corresponds to a feature cluster; at this point, there are no existing features in the feature clusters.

[0153] Step S702: Calculate the distance between the coefficient vector corresponding to each of the existing features and the center vector of each of the clusters, and assign the multiple existing features to the feature clusters corresponding to the nearest cluster center vectors;

[0154] After determining the cluster center vector, for each existing feature, calculate the distance between the coefficient vector corresponding to the existing feature and each cluster center vector, and assign the existing feature to the feature cluster corresponding to the cluster center vector closest to its coefficient vector.

[0155] Step S703: Average the coefficient vectors corresponding to the existing features in the feature clusters to obtain new cluster center vectors, and check whether the preset clustering termination condition is met.

[0156] After assigning each existing feature to its corresponding feature cluster, for each feature cluster, the coefficient vectors corresponding to each existing feature in that cluster can be averaged to obtain a new cluster center vector. At this point, it can be checked whether the preset clustering termination condition is met. Specifically, the clustering termination condition can be that the distance between the new cluster center vector and the original cluster center vector is less than a certain value, or that the number of iterations reaches a certain number, or that the iteration time reaches a certain time, etc.

[0157] Step S704: If it is determined that the clustering termination condition is met, then the clustering is terminated;

[0158] If the clustering termination condition is met, the clustering can be terminated, and the current feature cluster and the new cluster center vector can be used as the final feature cluster and cluster center vector.

[0159] Step S705: If it is determined that the clustering termination condition is not met, then step S702 is executed again based on the new cluster center vector.

[0160] If it is determined that the clustering termination condition is not met, then the new cluster center vector can be used as the basis to return to step S702 and perform a new round of iterations until the clustering termination condition is met.

[0161] Furthermore, based on the first and / or second embodiments described above, a third embodiment of the method for analyzing the validity of newly added business data according to the present invention is proposed. In this embodiment, after step S30, the method further includes:

[0162] Step A10: Add the newly added candidate features to the target feature cluster to increase the number of features in the target feature cluster;

[0163] When determining the target feature cluster to which a new candidate feature belongs, the new candidate feature can be added to the target feature cluster to increase the number of features in the target feature cluster. That is, the target feature cluster includes both the existing features and the new candidate feature. When there are multiple new candidate features, each new candidate feature is added to its corresponding feature cluster.

[0164] Step A20: When a feature selection request for the target business model is detected, obtain the effect information of each of the existing features on each business model, and determine the target existing features that have a positive effect on the training of the target business model from the existing features based on the effect information.

[0165] When feature selection is required for a target business model, a feature selection request for that target business model can be triggered. Upon detecting a feature selection request, the effect information of the existing features on each business model can be obtained. This effect information can be pre-entered and stored. The effect information may include whether the existing features have a positive or negative effect on the training of the business models. Based on the effect information of each existing feature, the existing features that have a positive effect on the training of the target business model can be identified as target existing features.

[0166] Step A30: Obtain new candidate features from the feature cluster where the target existing features are located and output them as new features of the target business model.

[0167] Once the target existing features are determined, new candidate features can be obtained from the feature clusters containing the target existing features and output as new features for the target business model. The output can be feature information such as the feature identifier or feature name of the new feature, allowing business personnel to select and construct the target business model based on the new feature.

[0168] It should be noted that when business personnel need to select features from multiple new candidate features to build a target business model, they can determine the effectiveness of each new candidate feature based on the correlation between the existing features already applied to the target business model and the new candidate feature. Compared to calculating the correlation between the new candidate feature and each existing feature, this embodiment only needs to determine the feature cluster to which each new candidate feature belongs, and then select the new candidate feature from the feature cluster of the existing feature that has a positive effect on the training of the target business model to build the target business model. This results in lower computational complexity and higher feature selection efficiency.

[0169] Further, in one embodiment, step S10 includes:

[0170] Step S101: When an evaluation request for new business data is detected, determine multiple original features from the new business data;

[0171] Step S102: Transform the multiple original features according to a preset cross-transformation method to obtain new candidate features;

[0172] In this embodiment, the newly added candidate features can be obtained by transforming the original features of the newly added business data. Specifically, when an evaluation request for newly added business data is detected, any number of original features can be selected from the original features in the newly added business data, and these original features can be transformed using a preset cross-transformation method to obtain new candidate features. The preset cross-transformation method can be a method set as needed, such as adding two original features or calculating two original features according to a complex formula. By transforming the original features to obtain new candidate features, and then performing effectiveness analysis on the new candidate features, it is possible to help extract features that are more effective for the business model from the existing features of the newly added business data.

[0173] Step S103: Obtain the feature values ​​of each first sample under the original features from the newly added business data, and calculate the feature values ​​of each first sample under the original features according to the transformation formula corresponding to the preset cross-transformation method to obtain the feature values ​​of each first sample under the newly added candidate features.

[0174] The feature values ​​of each first sample under the original features can be obtained from the newly added business data. For each first sample, the feature values ​​of the sample under the original features are calculated according to the transformation formula corresponding to the preset cross-transformation method to obtain the feature values ​​of the sample under the newly added candidate features.

[0175] Step S104: Combine the feature values ​​of each of the first samples under the newly added candidate feature to form the candidate feature vector corresponding to the newly added candidate feature.

[0176] After obtaining the feature values ​​of each first sample under the newly added candidate features, the feature values ​​can be combined to form the candidate feature vector corresponding to the newly added candidate features.

[0177] Furthermore, this embodiment of the invention also proposes a device for analyzing the validity of newly added business data, the device comprising:

[0178] The acquisition module is used to determine new candidate features from the new business data when an evaluation request for new business data is detected, to obtain the feature values ​​of each first sample under the new candidate features from the new business data to form a candidate feature vector corresponding to the new candidate features, and to obtain the label values ​​of each first sample under the target label of the business model to form a first label vector corresponding to the business model.

[0179] The calculation module is used to calculate the first correlation coefficient between the candidate feature vector corresponding to the newly added candidate feature and the first label vector corresponding to the multiple business models, and to form the coefficient vector corresponding to the newly added candidate feature by combining the multiple first correlation coefficients.

[0180] The analysis module is used to calculate the distance between the coefficient vector corresponding to the newly added candidate feature and the cluster center vectors of each type, and determine the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result, so as to determine the effectiveness of the newly added business data for each of the business models based on the effectiveness of the existing features in the target feature clusters for each of the business models. The cluster center vectors are obtained by clustering based on the coefficient vectors corresponding to each existing feature in the existing business data.

[0181] The extended content of the specific implementation of the new business data validity analysis device of the present invention is basically the same as the various embodiments of the new business data validity analysis method described above, and will not be repeated here.

[0182] Furthermore, this embodiment of the invention also proposes a computer-readable storage medium storing a validity analysis program for newly added business data. When the validity analysis program for newly added business data is executed by a processor, it implements the steps of the validity analysis method for newly added business data as described below.

[0183] The various embodiments of the newly added business data validity analysis device and computer-readable storage medium of the present invention can all refer to the various embodiments of the newly added business data validity analysis method of the present invention, and will not be described again here.

[0184] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0185] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0186] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0187] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for validity analysis of newly added business data, characterized in that, The method includes the following steps: When an evaluation request for new business data is detected, new candidate features are determined from the new business data, feature values ​​of each first sample under the new candidate features are obtained from the new business data to form a candidate feature vector corresponding to the new candidate features, and label values ​​of each first sample under the target label of the business model are obtained to form a first label vector corresponding to the business model. Calculate the first correlation coefficient between the candidate feature vector corresponding to the newly added candidate feature and the first label vector corresponding to the multiple business models respectively, and combine the multiple first correlation coefficients to form the coefficient vector corresponding to the newly added candidate feature. The distance between the coefficient vector corresponding to the newly added candidate feature and the center vector of each type of cluster is calculated. Based on the distance calculation result, the target feature cluster to which the newly added candidate feature belongs is determined. The effectiveness of the newly added business data for each of the business models is determined based on the effectiveness of the existing features in the target feature cluster for each of the business models. Wherein, each cluster center vector is obtained by clustering based on the coefficient vectors corresponding to each existing feature in the existing business data. Before the step of determining new candidate features from the new business data when an evaluation request for new business data is detected, the method further includes: Acquire existing business data and determine existing characteristics from the existing business data; The existing business data is used to obtain the feature values ​​of each second sample under the existing feature to form the existing feature vector corresponding to the existing feature, and the label values ​​of each second sample under the target label of the business model are obtained to form the second label vector corresponding to the business model. Calculate the second correlation coefficient between the existing feature vector corresponding to the existing feature and the second label vector corresponding to the multiple business models respectively, and combine the multiple second correlation coefficients to form the coefficient vector corresponding to the existing feature; Cluster the coefficient vectors corresponding to the multiple stock features, divide each stock feature into multiple feature clusters, and obtain the cluster center vector corresponding to each feature cluster.

2. The method for analyzing the validity of newly added business data as described in claim 1, characterized in that, The steps for calculating the second correlation coefficient between the existing feature vector corresponding to the existing feature and the second label vector corresponding to the business model include: The first distance matrix is ​​obtained by calculating the pairwise distance between each element in the existing feature vector, and the second distance matrix is ​​obtained by calculating the pairwise distance between each element in the second label vector. The third distance matrix is ​​obtained by subtracting the mean of the same row and the mean of the same column from each element of the first distance matrix and then adding the mean of the elements of the first distance matrix; and the fourth distance matrix is ​​obtained by subtracting the mean of the same row and the mean of the same column from each element of the second distance matrix and then adding the mean of the elements of the second distance matrix. The first value is obtained by summing the squares of each element in the third distance matrix, dividing by the number of columns, and then calculating the square root. The second value is obtained by summing the squares of each element in the fourth distance matrix, dividing by the number of columns, and then calculating the square root. The third value is obtained by multiplying the third distance matrix and the fourth distance matrix, summing the elements, dividing by the number of columns, and then calculating the square root. The square root of the product of the first value and the second value is calculated to obtain the fourth value. The third value is divided by the fourth value to obtain the second correlation coefficient between the stock feature vector and the second label vector.

3. The method for analyzing the validity of newly added business data as described in claim 1, characterized in that, The step of clustering the coefficient vectors corresponding to the multiple existing features, dividing each existing feature into multiple feature clusters, and obtaining the cluster center vector corresponding to each feature cluster includes: Initialize a preset number of cluster center vectors; Calculate the distance between the coefficient vector corresponding to each of the existing features and the center vector of each of the clusters, and assign each of the existing features to the feature cluster corresponding to the nearest cluster center vector; The coefficient vectors corresponding to the existing features in the feature clusters are averaged to obtain new cluster center vectors, and it is then checked whether the preset clustering termination condition is met. If the clustering termination condition is met, then the clustering process ends. If it is determined that the clustering termination condition is not met, then based on the new cluster center vector, the step of calculating the distance between the coefficient vector corresponding to the existing feature and each cluster center vector is returned.

4. The method for analyzing the validity of newly added business data as described in claim 1, characterized in that, The step of obtaining the feature values ​​of each second sample under the existing feature from the existing business data to form the existing feature vector corresponding to the existing feature includes: Obtain the feature values ​​of each second sample under the existing features from the existing business data; The empty feature values ​​among the obtained feature values ​​are replaced with preset values ​​to form the stock feature vector corresponding to the stock feature.

5. The method for validity analysis of newly added business data as described in claim 1, characterized in that, The step of determining the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result includes: When the distance between the coefficient vector of the newly added candidate feature and one of the cluster center vectors is less than a preset value, the feature cluster corresponding to the cluster center vector whose distance to the coefficient vector of the newly added candidate feature is less than the preset value is taken as the target feature cluster to which the newly added candidate feature belongs. When the distance between the coefficient vector of the newly added candidate feature and the multiple cluster center vectors is less than the preset value, the feature cluster corresponding to the cluster center vector with the smallest distance to the coefficient vector of the newly added candidate feature is taken as the target feature cluster to which the newly added candidate feature belongs. When the distance between the coefficient vector of the newly added candidate feature and the center vector of each of the clusters is not less than the preset value, a new feature cluster is added as the target feature cluster to which the newly added candidate feature belongs.

6. The method for validity analysis of newly added business data as described in claim 1, characterized in that, After the step of determining the target feature cluster to which the newly added candidate feature belongs based on the distance calculation result, the method further includes: The newly added candidate features are added to the target feature cluster to increase the number of features in the target feature cluster; When a feature selection request for a target business model is detected, the effect information of each of the existing features on each business model is obtained, and the target existing features that have a positive effect on the training of the target business model are determined from the existing features based on the effect information. New candidate features are obtained from the feature clusters in which the target existing features are located and output as new features of the target business model.

7. The method for analyzing the validity of newly added business data as described in any one of claims 1 to 6, characterized in that, The steps of determining new candidate features from the new business data and obtaining the feature values ​​of each first sample under the new candidate features to form the candidate feature vector corresponding to the new candidate features when an evaluation request for new business data is detected include: When an evaluation request for new business data is detected, multiple raw features are determined from the new business data. New candidate features are obtained by transforming multiple original features using a preset cross-transformation method; The feature values ​​of each first sample under the original feature are obtained from the newly added business data. The feature values ​​of each first sample under the original feature are calculated according to the transformation formula corresponding to the preset cross-transformation method to obtain the feature values ​​of each first sample under the newly added candidate feature. The feature values ​​of each of the first samples under the newly added candidate feature are used to form the candidate feature vector corresponding to the newly added candidate feature.

8. A device for analyzing the validity of newly added business data, characterized in that, The device for analyzing the validity of newly added business data includes: a memory, a processor, and a program for analyzing the validity of newly added business data stored in the memory and executable on the processor. When the program for analyzing the validity of newly added business data is executed by the processor, it implements the steps of the method for analyzing the validity of newly added business data as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a validity analysis program for newly added business data. When the validity analysis program for newly added business data is executed by a processor, it implements the steps of the validity analysis method for newly added business data as described in any one of claims 1 to 7.

Images