Number correctness evaluation method and system based on machine learning

Abstract

The present application provides a method and system for evaluating number correctness based on machine learning. The method comprises: obtaining sample data corresponding to numbers and names in a number library, screening and labeling the sample data to obtain a training data set; forming a standard feature vector set based on the training data set; using the standard feature vector set, setting the initial parameter configuration of the XGBoost model; based on feature importance analysis, screening key feature variables from the standard feature vector set, using the screened feature variables to train the XGBoost model to obtain a number correctness evaluation model; using the number correctness evaluation model, setting a probability threshold; using the number correctness evaluation model to predict the numbers in the number library, obtaining a predicted probability value; comparing the predicted probability value with the set probability threshold, and outputting a number correctness evaluation result. The present application significantly improves the objectivity and accuracy of number accuracy evaluation.

Description

Technical Field

[0001] The present application relates to the technical field of data quality management, and in particular to a method and system for evaluating number correctness based on machine learning. Background Art

[0002] Telephone number databases are essential data resources for modern communications and business activities. The accuracy of phone numbers significantly impacts communication service quality, corporate image, and user experience. In applications such as corporate address books, customer management systems, and phone number lookup services, assessing the accuracy of phone number data is a core component of data quality management.

[0003] Traditional number accuracy assessment methods rely primarily on manual verification and simple rule-based assessments. These methods typically rely on manually dialing numbers to verify their validity, or assign fixed weights to single factors such as number update time and source. While these methods can be effective in managing small-scale number databases, they lack systematicity and scientificity, making them difficult to apply to large-scale databases.

[0004] Currently, the most advanced number accuracy assessment technology uses customized ranking values ​​to identify number accuracy. This method assigns different weights to number data from different sources based on preset rules and uses a simple weighted calculation to determine the final accuracy score. This method offers significant improvements over purely manual evaluation, can handle larger number databases, and provides basic accuracy ranking capabilities.

[0005] However, this ranking value evaluation method based on fixed rules and subjective experience has obvious flaws: its data weight assignment relies too much on subjective experience and lacks objective data support; at the same time, as the scale of the number database expands and the data characteristics become increasingly complex, the fixed rules cannot dynamically adapt to data changes and capture new features, resulting in a large deviation between the accuracy evaluation results and the actual situation, and cannot meet the high requirements of modern communications and commercial applications for number accuracy. Summary of the Invention

[0006] In view of this, the present application provides a number correctness evaluation method and system based on machine learning, which solves the problem that the ranking value evaluation method based on fixed rules and subjective experience in the existing technology is too dependent on subjective experience in assigning data weights and cannot dynamically adapt to data changes and capture new features.

[0007] An embodiment of the present application provides a number correctness assessment method based on machine learning, including: obtaining sample data corresponding to numbers and names in a number library, screening and labeling the sample data to obtain a training data set; forming a standard feature vector set based on the training data set; using the standard feature vector set to set the initial parameter configuration of the XGBoost model; based on feature importance analysis, screening key feature variables from the standard feature vector set, using the screened feature variables to train the XGBoost model to obtain a number correctness assessment model; using the number correctness assessment model to set a probability threshold; using the number correctness assessment model to predict the numbers in the number library to obtain a predicted probability value; comparing the predicted probability value with the set probability threshold, and outputting a number correctness assessment result.

[0008] According to one embodiment of the present application, a feature source definition list is determined based on the training data set to form a standard feature vector set, including: extracting the conventional attribute features, call behavior fluctuation features, source association features and historical verification data features of the number from the training data set to form a feature source definition list; based on the feature source definition list, extracting the original feature data of the number, encoding and standardizing the original feature data to obtain a feature original value set.

[0009] According to one embodiment of the present application, the initial parameter configuration of the XGBoost model is set using the canonical feature vector set, including: based on the data distribution characteristics of the canonical feature vector set, setting the objective function type of the model to a binary classification function, the maximum depth parameter of the tree to a preset depth value, and the learning rate parameter to a preset learning rate value to form a basic parameter configuration; according to the basic parameter configuration, setting the sample sampling ratio parameter and the sample balancing processing parameter to obtain the initial parameter configuration.

[0010] According to one embodiment of the present application, key feature variables are screened from the standard feature vector set based on feature importance analysis, including: based on the configured initial parameters, using the standard feature vector set, training the XGBoost model, and calculating the feature importance score through the feature importance evaluation mechanism of the XGBoost model; screening and sorting the feature variables according to the feature importance score, determining a preset number of key feature variables, and constructing an optimized feature variable set.

[0011] According to one embodiment of the present application, the number correctness evaluation model is used to set a probability threshold, including: inputting a test data set into the number correctness evaluation model for verification, calculating the accuracy, precision and recall rate indicators, and forming a model performance evaluation report, wherein the test data set is divided from the training data set according to a preset ratio during the training process of the XGBoost model; using the model performance evaluation report to analyze the impact of different probability thresholds on model performance, and drawing a threshold-performance correspondence curve; according to the threshold-performance correspondence curve, combined with application requirements, setting the probability threshold.

[0012] According to one embodiment of the present application, after outputting the number correctness evaluation result, it also includes: performing scenario processing according to the number correctness evaluation result for the application requirements of the number library management, data cleaning and recommendation system, and outputting the scenario application result; collecting feedback data and new number samples in the scenario application result, updating the number correctness evaluation model, and forming an optimized number correctness evaluation model.

[0013] According to one embodiment of the present application, updating the number correctness evaluation model includes: based on the newly added feedback number samples, using a stochastic gradient descent algorithm with a constant learning rate to update the parameters of the number correctness evaluation model online; inputting text description information, industry background knowledge and historical verification records associated with the number into a large language model, extracting semantic features, and generating an unstructured feature vector; fusing the unstructured feature vector with the original features of the number to form a fusion vector; using the fusion vector to update the number correctness evaluation model to achieve iterative optimization of the number correctness evaluation model.

[0014] According to one embodiment of the present application, after outputting the number correctness assessment result, it also includes: constructing a feature distance calculation matrix for the continuous feature and the categorical feature respectively, and designing a corresponding kernel function to obtain a hybrid feature processing model; using the hybrid feature processing model to process the training data set, optimize the kernel function parameters, and form a Gaussian process model; from the prediction results of the number correctness assessment model, select number samples whose prediction probability values ​​are between a first preset threshold and a second preset threshold, and input them into the Gaussian process model to calculate the prediction uncertainty index to identify number samples that require manual verification.

[0015] According to one embodiment of the present application, feature distance calculation matrices for the number are constructed for continuous features and categorical features respectively, including: according to the continuous features of the number, a radial basis function kernel is used to calculate the distance between features to form a continuous feature distance matrix; according to the categorical features of the number, a Hamming distance kernel is used to calculate the feature dissimilarity to form a categorical feature distance matrix; and the continuous feature distance matrix and the categorical feature distance matrix are weightedly combined to obtain the hybrid feature processing model.

[0016] An embodiment of the present application also provides a number correctness assessment device based on machine learning, including: a training data generation module, used to obtain sample data corresponding to numbers and names in a number library, screen and label the sample data, and obtain a training data set; a feature engineering processing module, used to form a standard feature vector set based on the training data set; a model training module, used to use the normalized feature vector set to set the initial parameter configuration of the XGBoost model, and screen key feature variables based on feature importance analysis, and use the screened feature variables to train the XGBoost model to obtain a number correctness assessment model; a result output module, used to use the number correctness assessment model, set a probability threshold, and predict the correctness of the numbers in the number library, and output the number correctness assessment result.

[0017] An embodiment of the present application also provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned machine learning-based number correctness assessment method.

[0018] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the above-mentioned number correctness assessment method based on machine learning.

[0019] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above-mentioned machine learning-based number correctness assessment method.

[0020] This application has the following technical effects: by using the XGBoost algorithm to replace the traditional fixed rule evaluation, a machine learning evaluation model based on multi-dimensional features is realized, which greatly improves the objectivity and scientificity of the number correctness evaluation; by comprehensively considering multi-dimensional features such as the number's general attributes, call behavior fluctuations, source associations and historical verification, the complex characteristic patterns of the number correctness are fully captured, and the comprehensiveness and accuracy of the evaluation are improved; through the dynamic threshold setting mechanism, the probability threshold can be flexibly adjusted according to the needs of different application scenarios, and accurate screening of incorrect numbers is achieved; through the model update and maintenance mechanism, the evaluation model is ensured to continuously adapt to data changes and new features, and the timeliness and accuracy of the evaluation results are maintained; through scenario-based application processing, the number correctness evaluation results are seamlessly applied to various business scenarios such as number library management, data cleaning and recommendation systems, thereby improving the overall data quality and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0022] Figure 1 1 is a flow chart of a method for evaluating number correctness based on machine learning according to an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of a training data set preparation process according to one embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of a feature engineering process according to an embodiment of the present application;

[0025] Figure 4 This is a schematic diagram of the XGBoost model training and optimization process according to one embodiment of the present application;

[0026] Figure 5 It is a schematic diagram of the model evaluation and threshold setting process according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0029] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0030] The following is a detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the following description is only exemplary and is not intended to limit the scope of protection of the present application.

[0031] like Figure 1 As shown, the embodiment of the present application provides a method for evaluating number correctness based on machine learning, including:

[0032] S1: Obtain sample data corresponding to numbers and names in a number database, filter and label the sample data to obtain a training data set.

[0033] In S1, sample data corresponding to numbers and names in the number database is obtained, and the sample data is screened and labeled to obtain a training data set. This step is the basis of the entire number correctness evaluation method and aims to build a high-quality training data set.

[0034] First, we extract number data and its corresponding name information from an existing number database as the raw data source. Then, based on the correspondence between numbers and names, we screen out accurate numbers (where the numbers and names match) and inaccurate numbers (where the numbers and names do not match), forming an initial sample set. Next, we perform binary labeling on these samples, typically using 1 for accurate numbers and 0 for inaccurate numbers, transforming the problem into a binary classification task. Finally, we partition the dataset into training and test sets in a specific ratio (e.g., 7:3) to ensure a consistent distribution of samples across the two datasets. This process results in a structured and standardized training dataset, laying a solid foundation for subsequent model training.

[0035] Among them, S1 obtains sample data corresponding to numbers and names in the number library, screens and annotates the sample data to obtain a training data set. The specific implementation includes the following sub-steps:

[0036] S1.1: Data Sample Screening: Based on historical data in the number database, accurate numbers (where the number and name match) and inaccurate numbers (where the number and name do not match) are screened according to the correspondence between numbers and names to construct an initial sample set. First, existing number data is selected from the number database as the original data source. This data is analyzed to identify records where the number and name match as accurate number samples, and records where the number and name do not match as inaccurate number samples. In this step, existing verification records, user feedback, or business rules can be used to determine the accuracy of the number.

[0037] S1.2: Data Sample Labeling: Based on the initial sample set selected in step S1.1, each number sample is labeled with a binary value, typically 1 for a correct number (positive sample) and 0 for an incorrect number (negative sample). This binary labeling method transforms the problem into a binary classification task, making it suitable for modeling using machine learning algorithms. The labeling process must ensure label accuracy, as it directly affects the model's learning and predictive performance.

[0038] S1.3: Dataset Partitioning: Based on the sample data labeled in step S1.2, divide the dataset into a training set and a test set according to a specific ratio (e.g., 7:3) to ensure that the distribution of each type of sample in the two datasets is roughly consistent. Stratified sampling is used to ensure that the ratio of positive and negative samples in the training and test sets is consistent with that in the original dataset, thus avoiding model bias caused by uneven sample distribution. The training set is used for model learning and parameter optimization, while the test set is used to evaluate the model's generalization ability and predictive performance.

[0039] S2: Form a set of standard feature vectors based on the training data set.

[0040] First, based on the training data set, the main dimensions of number characteristics are determined, including general number attribute characteristics (such as number type, location, operator, update time, etc.), call behavior fluctuation characteristics (such as recent call frequency, call duration changes, etc.), source association characteristics (such as data source channel, collection method, etc.) and historical verification data characteristics (such as historical verification results, verification time, etc.).

[0041] Then, the raw data of these features are extracted from the number library and related systems, and necessary calculations and conversions are performed.

[0042] Finally, categorical features are encoded (e.g., one-hot encoding or label encoding) and numerical features are normalized (e.g., Z-score normalization or Min-Max normalization) to ensure that all feature data is in a uniform format and within an appropriate range. These processes form a standardized set of feature vectors, providing high-quality input data for model training.

[0043] S3: Using the canonical feature vector set, set the initial parameter configuration of the XGBoost model.

[0044] As a powerful ensemble learning algorithm, the performance of XGBoost depends largely on parameter configuration.

[0045] First, based on the data distribution characteristics of the normalized feature vector set, the model's objective function type is set to a binary classification function (e.g., "binary:logistic"). This is because number accuracy assessment is essentially a binary classification problem. Next, the maximum tree depth parameter is set to a preset value (e.g., 8). This parameter controls the complexity of each decision tree, balancing model complexity and generalization ability. Furthermore, the learning rate parameter is set to a preset value (e.g., 0.1). Although a smaller learning rate slows down model training, it can improve model stability and generalization ability.

[0046] In addition, it is necessary to set the sample sampling ratio parameter (such as 0.8) and the sample balancing processing parameter to address the sample imbalance problem and improve the robustness of the model. These initial parameter configurations provide a good starting point for subsequent model training.

[0047] The specific construction process of the XGBoost model is as follows:

[0048] Model structure: The XGBoost model in this application uses a tree ensemble structure consisting of 100 decision trees. The maximum depth of each tree is set to 8 to balance the complexity and generalization ability of the model.

[0049] Learning parameters: The learning rate is set to 0.1, the regularization parameters α and λ are set to 1 and 2, respectively, to control the overfitting risk of the model.

[0050] Training process: k = 5 fold cross validation method is used, each training iteration is 500 rounds, and the early stopping strategy (early_stopping_rounds=50) is used to avoid overfitting.

[0051] Feature selection: Through the feature importance scoring mechanism, feature variables with importance scores greater than 0.01 were screened out, and the top 141 most important features were finally selected.

[0052] Parameter optimization: A grid search method is used to find the optimal combination in the following parameter space: max_depth[4,6,8,10], min_child_weight[1,3,5], gamma[0,0.1,0.2], subsample[0.6,0.8,1.0].

[0053] S4: Based on the feature importance analysis, key feature variables are screened from the standard feature vector set, and the XGBoost model is trained using the screened feature variables to obtain a number correctness evaluation model.

[0054] This step first trains a basic XGBoost model using a set of canonical feature vectors based on the initial parameter configuration. The goal is to assess the contribution of each feature to the prediction result. By analyzing the XGBoost model's feature importance evaluation mechanism (such as the "feature_importances_" attribute), the importance score of each feature is calculated. Based on this score, the most relevant feature variables (such as "data source type" and "number of months since the last manual data entry") are selected. Feature selection not only improves model training efficiency but also reduces the risk of overfitting. The complete model training process is then carried out based on the selected feature set and optimized parameter configuration. Cross-validation techniques are typically used, followed by grid search or random search to find the optimal parameter combination. Finally, the final model is trained on the entire training set using the optimal parameters, resulting in an evaluation model that effectively distinguishes accurate from inaccurate numbers.

[0055] S5: Using the number correctness evaluation model, set a probability threshold.

[0056] This step aims to determine the optimal decision threshold for model application. First, the test dataset is fed into the trained model to obtain the model's accuracy prediction probability for each number. Performance metrics such as accuracy, precision, and recall are calculated to produce a model performance evaluation report. Next, the impact of different probability thresholds on the model's screening performance is analyzed. Several different thresholds (e.g., from 0.1 to 0.9) are tried. For each threshold, the corresponding performance metrics are calculated, and threshold-performance curves (e.g., threshold-precision curves and threshold-recall curves) are plotted. Finally, based on these curves and incorporating practical application requirements (e.g., prioritizing screening accuracy or pursuing a balanced performance), the optimal probability threshold is determined. For example, in an application scenario where incorrect numbers are screened, a higher threshold (e.g., 0.75) may be required to achieve a 95% screening accuracy rate. This approach provides a scientific basis for model decision-making in real-world applications.

[0057] S6: Use the number correctness evaluation model to predict the numbers in the number database to obtain a prediction probability value.

[0058] S6 uses the number correctness assessment model to predict the numbers in the number database and obtains a prediction probability value. This step applies the trained model to actual data.

[0059] First, the number data in the database is preprocessed, including feature extraction, encoding, and normalization, to ensure that the same processing methods are used as the training data to ensure consistent and accurate predictions. The processed feature vectors are then fed into the trained XGBoost model to obtain a prediction probability for each number. This probability indicates the model's likelihood of identifying the number as accurate, typically ranging from 0 to 1. Batch processing allows for efficient evaluation of all numbers in a large database, enabling rapid and automated number accuracy prediction.

[0060] S7: Compare the predicted probability value with the set probability threshold and output the number correctness evaluation result.

[0061] S7 compares the predicted probability value with the set probability threshold and outputs a number accuracy assessment result. Based on the probability threshold set in S5, the predicted probability value obtained in S6 is compared with the threshold. Generally, if the predicted probability value is greater than or equal to the threshold, the number is determined to be accurate; if the predicted probability value is less than the threshold, the number is determined to be inaccurate. In this way, a clear accuracy assessment result can be obtained for each number in the number database.

[0062] The predicted probability value is compared with the set probability threshold and the number correctness assessment result is output. This is the final step of the method, which converts the model's prediction results into specific decisions.

[0063] Specifically, the predicted probability value obtained in step S6 is compared with the probability threshold set in step S5. Generally, if the predicted probability value is greater than or equal to the threshold, it is determined to be an accurate number; if the predicted probability value is less than the threshold, it is determined to be an inaccurate number.

[0064] For example, if the threshold is set to 0.75, numbers with a predicted probability greater than or equal to 0.75 are determined to be accurate, and numbers with a predicted probability less than 0.75 are determined to be inaccurate.

[0065] In addition to binary classification results, the evaluation results can also include raw predicted probability values, providing more granular accuracy assessment information. The final evaluation results can be output in various formats, such as lists, reports, or visual charts, to meet the needs of different application scenarios. This approach enables accuracy assessment of all numbers in the number database, providing strong support for number database management, data cleansing, and business decision-making.

[0066] like Figure 2 As shown, in S2, a set of standard feature vectors is formed based on the training data set, including:

[0067] S2.1: Extract the number's general attribute features, call behavior fluctuation features, source association features, and historical verification data features from the training data set to form a feature source definition list.

[0068] Based on the training data set generated in step S1.3, the main dimensions of the number features are determined, including the number's general attribute features, call behavior fluctuation features, source association features, and historical verification data features, etc., to form a feature source definition list.

[0069] Specifically, the general attribute characteristics of a number include basic information such as number type, location, operator, and update time; the call behavior fluctuation characteristics include behavioral data such as recent call frequency and changes in call duration; the source association characteristics include related information such as data source channels, collection methods, and provider credibility; the historical verification data characteristics include historical verification results, verification time, verification methods, and other historical data.

[0070] S2.2: Extracting original feature data of the number based on the feature source definition list, encoding and standardizing the original feature data to obtain a set of feature original values.

[0071] Based on the feature source list defined in step S2.1, extract the original feature data from the number library and related systems, and perform necessary calculations and conversions to generate a set of feature raw values. Different extraction and calculation methods are used for different types of features.

[0072] For example, for time-related features, the number of months since the last manual verification can be calculated; for behavioral features, the variance of the weekly query times in the past four weeks can be calculated; for source features, the codes of different source channels can be extracted, etc.

[0073] Subsequent processes require encoding of categorical features and standardization of numerical features to ensure that all feature data is in a uniform format and within an appropriate range. For categorical features (such as number type and source channel), one-hot encoding or label encoding is used to convert them into numerical form. For numerical features, standardization (Z-score standardization) or min-max normalization is used to scale the values ​​to an appropriate range to prevent dimensional differences from affecting model training.

[0074] like Figure 3 As shown, in S3, the set of standard feature vectors is used to set the initial parameter configuration of the XGBoost model, including:

[0075] S3.1: Based on the data distribution characteristics of the standard feature vector set, the objective function type of the model is set to a binary classification function, the maximum depth parameter of the tree is set to a preset depth value, and the learning rate parameter is set to a preset learning rate value, thereby forming a basic parameter configuration.

[0076] Based on the normalized feature vector set output from step S2.3, determine the initial parameter configuration for the XGBoost model. For the objective function type, select "binary:logistic" because number accuracy evaluation is essentially a binary classification problem. For the maximum tree depth parameter "max_depth," set it to a preset depth value (e.g., 8). This parameter controls the complexity of each decision tree. Excessive depth may lead to overfitting, while too small a depth may lead to underfitting. For the learning rate parameter "eta," set it to a preset learning rate value (e.g., 0.1). This is a relatively conservative choice. Although a smaller learning rate slows down model training, it can improve model stability and generalization.

[0077] Based on the data distribution characteristics of the standard feature vector set, the objective function type of the model is set to a binary classification function, the maximum depth parameter of the tree is set to a preset depth value, and the learning rate parameter is set to a preset learning rate value to form a basic parameter configuration.

[0078] In this sub-step, technicians need to set the initial core parameters of the XGBoost model based on the characteristics of the number accuracy prediction task. First, for the objective function type, select "binary:logistic" as the objective function. This is because number accuracy assessment is essentially a binary classification problem, requiring a prediction of whether the number is accurate.

[0079] Secondly, the maximum tree depth parameter "max_depth" is a key parameter that controls the complexity of the decision tree and is usually set to 8. This parameter determines the maximum number of layers that each decision tree can split into. Too large a depth may cause the model to overfit the training data and lose generalization ability, while too small a depth may make the model too simple and unable to capture complex patterns in the data.

[0080] Finally, the learning rate parameter "eta" is typically set to 0.1, which is a relatively conservative choice. A smaller learning rate results in slower model training, but generally improves stability and generalization. Excessively large learning rates can lead to unstable model training or trapping the model in a local optimum. Properly setting these basic parameters provides a scientific starting point for subsequent model training, ensuring the model effectively learns the relationship between number features and accuracy.

[0081] S3.2: According to the basic parameter configuration, set the sample sampling ratio parameter and the sample balancing processing parameter to obtain the initial parameter configuration.

[0082] Based on the basic parameter configuration established in step S3.1, set the sample sampling ratio parameter "subsample" (e.g., 0.8). This means that only 80% of the data is used each time the tree is built. This randomness helps the model better cope with different data distributions. Furthermore, considering the difference in the number of accurate and inaccurate numbers in the sample, set the sample balancing parameter "scale_pos_weight" to balance the sample. This parameter value is typically set to the ratio of the number of negative samples to the number of positive samples to ensure that the model is not biased towards the class with the larger sample size.

[0083] In addition to the basic parameters, some additional parameters need to be set to improve model robustness and handle sample imbalance. First, the subsample parameter is typically set to 0.8, meaning that only 80% of the data samples are randomly used each time a tree is built. This randomness helps reduce the risk of overfitting and makes the model more robust to changes in the data distribution.

[0084] Secondly, because the number of accurate and inaccurate numbers in real-world number databases often differs significantly (a sample imbalance), it's necessary to set the sample balancing parameter "scale_pos_weight" to balance the number. This parameter is typically set to the ratio of the number of negative samples to the number of positive samples to ensure the model isn't biased towards the class with the larger sample size.

[0085] For example, if the number of inaccurate numbers (negative samples) is three times the number of accurate numbers (positive samples), you can set "scale_pos_weight" to 3. Additionally, you can set other parameters, such as the regularization parameters "alpha" and "lambda," to control model complexity and prevent overfitting. By configuring these parameters comprehensively, you form a complete initial parameter configuration, laying a solid foundation for model training.

[0086] like Figure 4 As shown, in S4, based on the feature importance analysis, key feature variables are screened from the set of standard feature vectors, including:

[0087] S4.1: Based on the configured initial parameters, the XGBoost model is trained using the set of standard feature vectors, and a feature importance score is calculated using the feature importance evaluation mechanism of the XGBoost model.

[0088] Based on the configured initial parameters, a basic XGBoost model is trained using a set of canonical feature vectors. The XGBoost model's feature importance evaluation mechanism, "feature_importances_," calculates the importance score of each feature, which reflects the number of times a feature is used as a split point across all trees and the degree to which these splits improve model performance.

[0089] The purpose of this sub-step is not to obtain the final prediction model, but to evaluate the contribution of each feature to the prediction result.

[0090] First, a basic XGBoost model is trained using the initial parameter configuration and normalized feature vector set in step S3. In an XGBoost model, each split point in the decision tree is based on a specific feature, and the degree to which these splits improve model performance directly reflects the importance of that feature. The XGBoost model's built-in "feature_importances_" attribute provides an important score for each feature. This score comprehensively considers the number of times a feature is used as a split point across all trees and the degree to which each split improves the model's objective function (such as the log-likelihood function). Feature importance scores not only provide insight into the model's decision-making mechanisms but also provide an objective basis for subsequent feature selection. This approach allows for a deeper understanding of the impact of different features on number accuracy assessment, enabling more targeted model optimization.

[0091] S4.2: Screen and sort the feature variables according to the feature importance scores, determine a preset number of key feature variables, and construct an optimized feature variable set.

[0092] The feature variables are screened and sorted according to the feature importance scores, a preset number of key feature variables are determined, and an optimized feature variable set is constructed.

[0093] In this sub-step, all feature variables are sorted in descending order based on the feature importance scores calculated in step S4.1. A preset number (e.g., 141) of these feature variables with the highest importance scores are then selected to construct an optimized feature variable set. These selected key features may include "data source type," "number of months since the last manual data update," "last outbound call status," and "weekly query variance over the past four weeks." These features encompass multiple dimensions of a number and comprehensively reflect its accuracy. This feature selection process not only improves model training efficiency (reducing computing resource consumption and training time) but also reduces the risk of overfitting, as irrelevant or redundant features can introduce noise and interfere with the model's learning process. Feature selection also improves model interpretability, providing a clearer understanding of the factors that most impact number accuracy. This step results in a streamlined and effective feature subset, paving the way for subsequent full model training.

[0094] like Figure 5 As shown, in S5, the number correctness assessment model is used to set a probability threshold, including:

[0095] S5.1: Input the test data set into the number correctness evaluation model for verification, calculate the accuracy, precision and recall rate indicators, and form a model performance evaluation report, wherein the test data set is divided from the training data set according to a preset ratio during the training process of the XGBoost model.

[0096] In S5.1, based on the number correctness evaluation model completed in S4, performance testing is performed using the test dataset.

[0097] The test dataset is created by dividing the training dataset in S1.3 according to a preset ratio and is independent of the training dataset. The test dataset is fed into the model to obtain the model's accuracy prediction probability for each number. Performance metrics such as accuracy, precision, and recall are calculated to produce a model performance evaluation report.

[0098] The test data set is input into the number correctness evaluation model for verification, and the accuracy, precision and recall rate indicators are calculated to form a model performance evaluation report, wherein the test data set is divided from the training data set according to a preset ratio during the training process of the XGBoost model.

[0099] After the model training is completed, its performance needs to be comprehensively evaluated to verify the effectiveness and accuracy of the model.

[0100] First, the test dataset (the independent dataset created in step S1.3) is fed into the trained model to obtain the model's probability of accurate prediction for each number. The predicted probabilities are then converted to binary classifications (accurate / inaccurate) based on a preset initial threshold (typically 0.5). By comparing the predicted results with the actual labels, a series of performance metrics are calculated: Accuracy reflects the overall proportion of numbers correctly classified by the model; Precision reflects the proportion of numbers predicted as "accurate" by the model that were actually accurate; Recall reflects the proportion of numbers that were actually accurate that were correctly predicted by the model; and the F1 score is the harmonic average of Precision and Recall, comprehensively considering the balance between the two.

[0101] In addition, ROC curves (Receiver Operating Characteristic curves) and AUC values ​​(Area Under the Curve) can be plotted to more comprehensively evaluate the model's performance at different thresholds. A comprehensive analysis of these evaluation metrics generates a model performance evaluation report, providing a data foundation for subsequent threshold sensitivity analysis.

[0102] S5.2: Use the model performance evaluation report to analyze the impact of different probability thresholds on model performance and draw a threshold-performance correspondence curve.

[0103] In S5.2, based on the model performance evaluation report from step S5.1, analyze the impact of different probability thresholds on model screening performance. Try multiple probability thresholds (e.g., from 0.1 to 0.9, with intervals of 0.05). For each threshold, calculate the corresponding precision, recall, and other metrics. Plot threshold-precision and threshold-recall curves, etc., to form threshold-performance relationship curves.

[0104] The model outputs a probability value (between 0 and 1) that a sample belongs to the positive class. A threshold must be set to convert this probability value into a classification result. Different thresholds can lead to different classification results, affecting model performance metrics such as precision and recall. To find the most appropriate threshold for a practical application, a threshold sensitivity analysis is required.

[0105] In practice, multiple probability thresholds are selected within a reasonable range (e.g., 0.1 to 0.9, with intervals of 0.05). For each threshold, the trained model is used to make predictions on the test dataset and calculate the corresponding performance metrics. For example, when the threshold is set to 0.5, the model may correctly screen 92.3% of incorrect numbers. When the threshold is increased to 0.75, the screening accuracy can reach 95%, but the recall rate may decrease, meaning that some incorrect numbers are not screened out. By plotting the performance metrics at different thresholds, threshold-performance curves are formed, such as threshold-precision curves, threshold-recall curves, and threshold-F1 score curves. This visual display can intuitively reflect the impact of threshold changes on various performance metrics, helping technicians to gain a deeper understanding of the model's behavioral characteristics under different decision boundaries.

[0106] S5.3: According to the threshold-performance correspondence curve and in combination with application requirements, a probability threshold is set.

[0107] In S5.3, the optimal probability threshold is determined based on the threshold-performance curve generated in step S5.2 and in combination with actual application requirements (e.g., prioritizing screening accuracy). For example, in an application scenario where incorrect numbers are screened, accuracy may be more important to avoid incorrectly classifying correct numbers as incorrect. A probability threshold of 0.75 achieves a 95% accuracy rate for incorrect number screening.

[0108] Different application scenarios may prioritize precision and recall differently, so the optimal probability threshold needs to be determined based on specific needs. For example, in a number library management scenario, the accuracy of correctly screening incorrect numbers (i.e., precision) may be more important to avoid mistakenly classifying correct numbers as incorrect. In a recommendation system, recall may be more important to ensure that as many accurate numbers as possible are recommended.

[0109] In an application for screening incorrect numbers, a probability threshold of 0.75 achieves a 95% accuracy rate. This suggests that increasing the threshold can further improve the model's precision, though this may come at the expense of recall. The specific threshold should be selected based on business needs and acceptable risk levels, aiming to find the optimal balance between precision and recall.

[0110] Alternatively, more complex methods can be used to determine the optimal threshold, such as those based on cost functions. For example, one could define the business costs of different types of errors (false positives and false negatives) and then select the threshold that minimizes the overall cost. The resulting threshold, combined with the trained model, forms a complete number correctness assessment solution, ensuring that the model delivers the most relevant assessment results in real-world applications.

[0111] The specific algorithm for threshold setting is as follows:

[0112] Performance index calculation: For each candidate threshold T (from 0.1 to 0.9, with a step size of 0.05), the following performance index is calculated:

[0113] Precision P(T)=TP(T) / (TP(T)+FP(T)),

[0114] Recall rate R(T)=TP(T) / (TP(T)+FN(T)),

[0115] F1 score F1(T)=2·P(T)·R(T) / (P(T)+R(T)),

[0116] Where TP(T), FP(T), and FN(T) are the number of true positives, false positives, and false negatives under the threshold T, respectively.

[0117] Cost function: Define a cost function C(T) that takes into account business needs:

[0118] C(T)=w1·(1-P(T))+w2·(1-R(T)),

[0119] Where w1 and w2 are weight coefficients set according to business requirements, indicating the relative importance of precision and recall.

[0120] Optimal threshold selection: Select the threshold that minimizes the cost function C(T) as the optimal threshold:

[0121] T_opt=argminC(T),

[0122] Threshold Validation: Validate the performance of the selected threshold on an independent validation set to ensure its stability across different datasets.

[0123] In addition, after outputting the number correctness evaluation result in S7, the embodiment of the present application further includes:

[0124] S8: Based on the number correctness evaluation results, scenario processing is performed according to the application requirements of number library management, data cleaning and recommendation system, and scenario application results are output.

[0125] S8 performs scenario-based processing based on the number correctness evaluation results, targeting the application requirements of number library management, data cleaning, and recommendation systems, and outputs scenario application results: for the number library management scenario, the number library can be sorted, graded, or marked based on the evaluation results, so that management personnel can process number data of different quality levels in a targeted manner; for the data cleaning scenario, low-accuracy numbers can be automatically screened out for verification or update based on the evaluation results; for the recommendation system scenario, the number accuracy probability value can be used as a weight factor of the recommendation algorithm to ensure that numbers with high accuracy are recommended first.

[0126] S9: Collect feedback data and newly added number samples in the scenario application results, update the number correctness evaluation model, and form an optimized number correctness evaluation model.

[0127] S9 collects feedback data and newly added number samples from the scenario application results, updates the number correctness assessment model, and forms an optimized number correctness assessment model. In actual application, user feedback, newly added number verification results, and other data can be used as new training samples for regular model updates and optimization. Updates can be triggered based on a fixed cycle (e.g., monthly or quarterly), or based on the number of newly added samples or changes in model performance.

[0128] S9 updates the number correctness assessment model, including:

[0129] S9.1: Based on the newly added feedback number samples, the stochastic gradient descent algorithm with a constant learning rate is used to perform online parameter updates on the number correctness evaluation model.

[0130] Specifically, based on newly added feedback number samples, the number correctness assessment model is updated online using a stochastic gradient descent algorithm with a constant learning rate. Online learning allows the model to be updated instantly as new data arrives, eliminating the need to retrain the entire model. When the system receives a new number verification result, it is immediately input into the model as a training sample, adjusting the model parameters through a single gradient update. This constant learning rate ensures the stability of model parameter updates.

[0131] Based on the newly received feedback number samples, the number correctness assessment model is updated online using a stochastic gradient descent algorithm with a constant learning rate. Traditional model updates typically require batch retraining after collecting sufficient new samples, which is time-consuming and computationally intensive. Online learning technology, however, allows the model to be updated instantly as new data arrives, eliminating the need to retrain the entire model, significantly improving system responsiveness and adaptability.

[0132] In practice, when the system receives new number verification results (such as confirmation of number accuracy obtained through user feedback or external verification), they are immediately input into the model as training samples, and the model parameters are adjusted through a single gradient update. Unlike traditional learning rate decay strategies, a constant learning rate design is employed here to ensure the stability of model parameter updates and continuous adaptability to new patterns. This constant learning rate avoids the problem of a model's ability to adapt to new data reduced due to excessive learning rate decay, making it particularly suitable for scenarios such as number accuracy assessment, where data distribution may change over time. Furthermore, online updates can be combined with window-based strategies to assign higher weights to samples from the most recent period, allowing the model to focus more on recent data patterns and further improving its response to changes in data distribution.

[0133] S9.2: Input the text description information, industry background knowledge, and historical verification records associated with the number into a large language model to extract semantic features and generate an unstructured feature vector.

[0134] The text description information associated with the number, industry background knowledge, and historical verification records are fed into a large language model to extract semantic features and generate an unstructured feature vector. The large language model can extract implicit features from unstructured data such as text descriptions, industry background, and historical records. For example, it can identify the industry type from the company description associated with the number and summarize verification patterns from historical records.

[0135] The large language model inputs text descriptions, industry background knowledge, and historical verification records associated with the number, extracts semantic features, and generates unstructured feature vectors. The XGBoost model primarily processes structured feature data, but in actual number management, a large amount of unstructured textual information associated with numbers exists, such as descriptions of the company associated with the number, industry background knowledge, and notes from historical verification records. This textual data can contain rich semantic information and is crucial for determining number accuracy. LLMs, with their powerful semantic understanding and reasoning capabilities, are able to extract implicit features from this unstructured text.

[0136] During implementation, various text descriptions related to phone numbers are first collected and then fed into a pre-trained large-scale language model (LLM). The LLM is capable of understanding the semantic content of text, such as identifying industry type and operating status from company descriptions and analyzing verification patterns and change patterns from historical records. Through LLM processing, this textual information is converted into fixed-dimensional feature vectors. These vectors capture the semantic information in the text and can reflect multiple dimensions, such as the consistency between the number and name, the company's activity, and the timeliness of the information. This provides a new source of information for subsequent feature fusion and model optimization.

[0137] The large language model used in this application is specifically implemented as follows:

[0138] Model architecture: A pre-trained language model based on Transformer is used, which includes a 12-layer attention mechanism, a hidden layer dimension of 768, and 12 attention heads.

[0139] Input processing: Text information related to the number (such as company descriptions and industry background knowledge) is tokenized and input into the model. The maximum sequence length is set to 256.

[0140] Feature extraction: The [CLS] token output of the last layer of the model is used to obtain a 768-dimensional text representation vector as a semantic feature vector.

[0141] Fine-tuning strategy: Perform domain-adaptive fine-tuning on text corpora related to the number domain, use the Adam optimizer with a learning rate of 2e-5, and train for 10 epochs.

[0142] Reasoning process: The relevant text of each number is batch processed (batch_size=32), semantic features are extracted, and the dimension is reduced to 64 through dimensionality reduction technology (PCA) to maintain a balance with the structured features.

[0143] S9.3: Fusing the unstructured feature vector with the original features of the number to form a fused vector.

[0144] The unstructured feature vector is fused with the original features of the number to form a fused vector. Appropriate feature fusion strategies, such as feature concatenation, weighted combination, or hierarchical fusion, are required to organically combine semantic features with structured features. The fused feature vector contains more comprehensive and richer number information.

[0145] In order to fully utilize the complementary advantages of structured features and unstructured features, it is necessary to effectively fuse these two types of features. Feature fusion is a key step that directly affects the quality and expressiveness of the fused features.

[0146] In practice, three main fusion strategies are commonly used: feature concatenation (directly concatenating two feature vectors into a longer vector by dimension), weighted combination (weighting different feature types and then performing a weighted sum), or hierarchical fusion (processing each feature within its own feature space before fusing at a higher-level semantic level). For number accuracy assessment, feature concatenation is often the most direct and effective approach, but care must be taken to address imbalanced feature dimensions. Dimensionality reduction techniques (such as PCA or t-SNE) can be used to compress high-dimensional features.

[0147] Furthermore, an attention mechanism can be introduced to dynamically adjust the weights of different features based on the characteristics of the current sample, making the fusion process more intelligent and adaptive. The fused feature vector contains multi-dimensional information about the number, including both precise structural attributes (such as time, frequency, and source) and rich semantic descriptions (such as industry background and company status). This provides the model with more comprehensive and in-depth judgment basis, improving the accuracy and robustness of the assessment.

[0148] The specific implementation method of feature fusion is as follows:

[0149] Feature standardization: Z-score standardization is performed on structured features and unstructured features respectively, so that their mean is 0 and standard deviation is 1.

[0150] Dimension matching: Principal component analysis (PCA) is used to reduce the dimension of the unstructured feature vector (768 dimensions) to a dimension similar to that of the structured feature (64 dimensions).

[0151] Feature splicing: directly splice the standardized structured feature vector and the unstructured feature vector after dimensionality reduction to form a fused feature vector.

[0152] Feature weight: The attention mechanism is introduced to dynamically adjust the weights of the two types of features. The specific formula is:

[0153] F 融合 =α·F 结构化 +(1-α)·F 非结构化 ;

[0154] Among them, α is the weight coefficient calculated by the attention network, which is adaptively adjusted according to the characteristics of the current sample.

[0155] Feature selection: The fused feature vector is evaluated again for feature importance to select the feature subset that is most valuable for prediction.

[0156] S9.4: Using the fusion vector, update the number correctness evaluation model to achieve iterative optimization of the number correctness evaluation model.

[0157] Use the fused feature vector to retrain or fine-tune the evaluation model so that the model can make more accurate evaluation judgments by utilizing the newly added semantic features.

[0158] The following is a detailed description of steps S9.1-9.4 and S10.1-10.3, formatted in natural paragraphs:

[0159] S9.1: Based on the newly added feedback number samples, the stochastic gradient descent algorithm with a constant learning rate is used to perform online parameter updates on the number correctness evaluation model.

[0160] Traditional model updates typically require batch retraining after collecting sufficient new samples, a time-consuming and computationally intensive approach. Online learning, on the other hand, allows models to be updated instantly as new data arrives, eliminating the need to retrain the entire model, significantly improving system responsiveness and adaptability.

[0161] In practice, when the system receives new number verification results (such as confirmation of number accuracy obtained through user feedback or external verification), they are immediately input into the model as training samples, and the model parameters are adjusted through a single gradient update. Unlike traditional learning rate decay strategies, a constant learning rate design is employed here to ensure the stability of model parameter updates and continuous adaptability to new patterns. This constant learning rate avoids the problem of a model's ability to adapt to new data reduced due to excessive learning rate decay, making it particularly suitable for scenarios such as number accuracy assessment, where data distribution may change over time. Furthermore, online updates can be combined with window-based strategies to assign higher weights to samples from the most recent period, allowing the model to focus more on recent data patterns and further improving its response to changes in data distribution.

[0162] S9.2: Input the text description information, industry background knowledge, and historical verification records associated with the number into a large language model to extract semantic features and generate an unstructured feature vector.

[0163] While the XGBoost model primarily processes structured feature data, actual number management often involves a significant amount of unstructured textual information associated with numbers, such as descriptions of the company associated with the number, industry background information, and notes from historical verification records. This textual data can contain rich semantic information and is crucial for determining number accuracy.

[0164] Large language models (LLMs) can extract implicit features from these unstructured texts with their powerful semantic understanding and reasoning capabilities. During implementation, various text description information related to the number is first collected, and then these texts are input into a pre-trained large language model. LLMs can understand the semantic content of texts, such as identifying industry types, operating status, and other information from company descriptions, and analyzing verification patterns and change patterns from historical records. Through LLM processing, these text information is converted into fixed-dimensional feature vectors. These vectors capture the semantic information in the text and can reflect features in multiple dimensions, such as the consistency between the number and the name, the activity of the company, and the timeliness of the information, providing a new source of information for subsequent feature fusion and model optimization.

[0165] S9.3: Fusing the unstructured feature vector with the original features of the number to form a fused vector.

[0166] In order to fully utilize the complementary advantages of structured features and unstructured features, it is necessary to effectively fuse these two types of features. Feature fusion is a key step that directly affects the quality and expressiveness of the fused features.

[0167] In practice, three main fusion strategies are commonly used: feature concatenation (directly concatenating two feature vectors into a longer vector by dimension), weighted combination (weighting different feature types and then performing a weighted sum), or hierarchical fusion (processing each feature within its own feature space before fusing at a higher-level semantic level). For number accuracy assessment, feature concatenation is often the most direct and effective approach, but care must be taken to address imbalanced feature dimensions. Dimensionality reduction techniques (such as PCA or t-SNE) can be used to compress high-dimensional features.

[0168] Furthermore, an attention mechanism can be introduced to dynamically adjust the weights of different features based on the characteristics of the current sample, making the fusion process more intelligent and adaptive. The fused feature vector contains multi-dimensional information about the number, including both precise structural attributes (such as time, frequency, and source) and rich semantic descriptions (such as industry background and company status). This provides the model with more comprehensive and in-depth judgment basis, improving the accuracy and robustness of the assessment.

[0169] S9.4: Using the fusion vector, update the number correctness evaluation model to achieve iterative optimization of the number correctness evaluation model.

[0170] Once you have the fused feature vector, you need to effectively apply it to model updates. Depending on the size and nature of the fused features, different update strategies can be employed. If the fused features have similar dimensions to the original features, you can directly replace them with the fused features and retrain or fine-tune the XGBoost model. If the fused features have significantly increased in dimension, you may need to adjust the model structure or parameters to accommodate the new feature space.

[0171] During implementation, an effective approach is to adopt an ensemble learning strategy, retaining the original XGBoost model while training a new model specifically for the fused features. The model then combines the strengths of both models through model fusion (such as weighted averaging or stacking). Furthermore, transfer learning techniques can be used to transfer knowledge from the original model to the new feature space, accelerating the model's adaptation and learning based on the new features.

[0172] Model updates shouldn't be a one-off process; instead, they should form a closed-loop, iterative optimization process. As new data accumulates and the model continues to be applied, the system automatically collects feedback, extracts new features, optimizes the model, and then redeploys the updated model into real-world applications. Through this continuous iteration, the number correctness assessment model can continuously improve itself, adapting to changes in data distribution and the emergence of new features, providing increasingly accurate assessment results.

[0173] In another embodiment, after S7 outputs the number correctness evaluation result, the process further includes:

[0174] S10: For continuous features and categorical features, construct feature distance calculation matrices for the numbers respectively, and design corresponding kernel functions to obtain a hybrid feature processing model.

[0175] S10 constructs a feature distance calculation matrix for the number for each of the continuous feature and the categorical feature, including:

[0176] S10.1: Based on the continuous features of the number, radial basis function kernel is used to calculate the distance between features to form a continuous feature distance matrix.

[0177] Based on the continuous features of the number, a radial basis function kernel is used to calculate the distances between features, forming a continuous feature distance matrix. For continuous features of numbers (such as call frequency and update time), the radial basis function kernel (RBF kernel) can effectively calculate the similarity of samples in the continuous feature space.

[0178] Based on the continuous features of the numbers, radial basis function kernels are used to calculate the distances between features to form a continuous feature distance matrix. The Gaussian process model is a powerful non-parametric Bayesian method that is particularly suitable for handling uncertainty estimation.

[0179] When building a Gaussian process model, the selection and design of the kernel function is a key step, as it determines how the model measures sample similarity. For continuous phone number features (such as call frequency, update time, and query times), the radial basis function (RBF) kernel (also known as the Gaussian kernel) is a widely used choice.

[0180] The form of the RBF kernel function is k(x,x')=exp(-||x-x'||² / 2l²), where ||x-x'|| is the Euclidean distance between samples and l is the length scale parameter that controls the rate at which similarity changes with distance.

[0181] During implementation, all continuous features are first separated from the number features. These features are then preprocessed (such as normalization) to eliminate dimensionality differences. The Euclidean distance between any two samples on these continuous features is then calculated and converted into a similarity metric using the RBF kernel function. This allows the similarity of all pairs of samples in the dataset to be calculated in the continuous feature space, forming an n×n continuous feature distance matrix (where n is the number of samples). This matrix captures the distribution structure of the samples in the continuous feature space and provides a foundational metric for the subsequent Gaussian process model.

[0182] S10.2: Based on the classification features of the number, the feature dissimilarity is calculated using the Hamming distance kernel to form a classification feature distance matrix.

[0183] Based on the number's categorical features, the Hamming distance kernel is used to calculate feature dissimilarity, forming a categorical feature distance matrix. For categorical features (such as source type and number type), the Hamming distance can measure the degree of dissimilarity between two samples in terms of categorical features.

[0184] Based on the categorical features of the numbers, the Hamming distance kernel is used to calculate feature dissimilarity, forming a categorical feature distance matrix. In addition to continuous features, numbers also have many categorical features (such as number type, source channel, industry, etc.), which cannot be directly calculated using Euclidean distance.

[0185] For categorical features, Hamming distance is a suitable metric that counts the number of elements that differ in corresponding positions in two vectors of equal length.

[0186] During implementation, categorical features must first be converted to an appropriate representation. One-hot encoding is typically used to convert categorical variables into binary vectors. Next, for any two sample encoded vectors, the Hamming distance (the number of bits that differ) is calculated. The Hamming distance kernel function can be expressed as k(x,x') = exp(-γ·H(x,x')), where H(x,x') is the Hamming distance and γ is a scaling parameter that controls the influence of distance on similarity.

[0187] In this way, the similarity of categorical features is calculated for all pairs of samples in the dataset, forming a categorical feature distance matrix. This matrix reflects the similarity between samples in the categorical feature space and captures the impact of categorical information on number accuracy. Unlike continuous feature matrices, categorical feature matrices focus more on measuring categorical consistency, providing another perspective for similarity assessment.

[0188] S10.3: Performing a weighted combination of the continuous feature distance matrix and the categorical feature distance matrix to obtain the hybrid feature processing model.

[0189] The continuous feature distance matrix and the classification feature distance matrix are weighted and combined to obtain the hybrid feature processing model. By setting appropriate weight coefficients, the continuous feature distance matrix and the classification feature distance matrix are combined into a composite kernel function matrix as the basis of the Gaussian process model.

[0190] The continuous feature distance matrix and the categorical feature distance matrix are weighted together to obtain the hybrid feature processing model. Continuous and categorical features capture different aspects of number characteristics and need to be effectively combined to form a unified similarity metric. In Gaussian process models, this combination is typically achieved through a linear combination or product of kernel functions.

[0191] This application adopts the weighted linear combination method, that is, k(x,x')=α·k_cont(x,x')+β·k_cat(x,x'), where k_cont is the continuous feature kernel function, k_cat is the classification feature kernel function, and α and β are weight coefficients that control the relative importance of the two types of features.

[0192] During implementation, these weight coefficients can be optimized through methods such as cross-validation or maximizing marginal likelihood to find the combination that best suits the current data. The weighted combination results in a new kernel function matrix that integrates information from both continuous and categorical features, enabling a more comprehensive measure of similarity between samples. This hybrid feature processing model is a core component of building a Gaussian process model, determining how the model understands the data distribution and makes predictions. By properly designing and optimizing this hybrid kernel function, the Gaussian process model can better adapt to the characteristics of phone number data, providing more accurate predictions and more reasonable uncertainty estimates, particularly for edge cases that are difficult to clearly determine using the XGBoost model.

[0193] S11: Utilize the hybrid feature processing model to process the training data set, optimize kernel function parameters, and form a Gaussian process model.

[0194] S11 uses the hybrid feature processing model to process the training dataset, optimize kernel function parameters, and form a Gaussian process model. By maximizing the marginal likelihood function, various parameters in the composite kernel function, such as the length scale parameter of the RBF kernel and the weight coefficient of the Hamming distance kernel, are optimized. This optimization process is typically implemented using numerical optimization techniques such as the conjugate gradient method or the L-BFGS algorithm. The optimized Gaussian process model can not only predict the accuracy probability of a number but also estimate the uncertainty of this prediction.

[0195] S12: From the prediction results of the number correctness evaluation model, select number samples whose prediction probability values ​​are between the first preset threshold and the second preset threshold, input them into the Gaussian process model, and calculate the prediction uncertainty index to identify number samples that require manual verification.

[0196] S12 selects number samples whose prediction probability values ​​are between a first preset threshold and a second preset threshold from the prediction results of the number correctness evaluation model, inputs them into the Gaussian process model, and calculates the prediction uncertainty index to identify number samples that require manual verification.

[0197] When the XGBoost model's predictions for certain numbers fall into the "gray zone" (e.g., probabilities between 0.4 and 0.6), these numbers are fed into a Gaussian process model for further analysis. In addition to outputting the mean accuracy probability, the Gaussian process model also provides the variance or standard deviation, indicating the uncertainty of the prediction. By setting an appropriate uncertainty threshold, we can identify numbers whose accuracy remains uncertain even after evaluation by both models and flag them for human verification.

[0198] An embodiment of the present application also provides a number correctness assessment device based on machine learning, including: a training data generation module, used to obtain sample data corresponding to numbers and names in a number library, screen and label the sample data, and obtain a training data set; a feature engineering processing module, used to form a standard feature vector set based on the training data set; a model training module, used to use the normalized feature vector set to set the initial parameter configuration of the XGBoost model, and screen key feature variables based on feature importance analysis, and use the screened feature variables to train the XGBoost model to obtain a number correctness assessment model; a result output module, used to use the number correctness assessment model, set a probability threshold, and predict the correctness of the numbers in the number library, and output the number correctness assessment result.

[0199] The machine learning-based number correctness assessment method and system in the embodiments of the present application achieve a scientific assessment of number accuracy by utilizing the XGBoost algorithm and multi-dimensional feature engineering, effectively solving the problems of strong subjectivity and poor adaptability of traditional assessment methods, and providing an advanced and reliable technical solution for number data quality management in modern communications and commercial applications.

[0200] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the machine learning-based number correctness assessment method and system described in the above method embodiment. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0201] In addition, an embodiment of the present disclosure also provides a computer program product, which stores a computer program. When the computer program is run by a processor, it executes the steps of the machine learning-based number correctness assessment method and system provided in any of the above embodiments of the present disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0202] The computer program product may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, which may be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK).

[0203] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment and devices can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0204] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0205] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0206] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0207] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A method for evaluating number correctness based on machine learning, characterized in that: include: Obtain sample data corresponding to numbers and names in the number database, filter and annotate the sample data to obtain a training data set; Forming a set of standard feature vectors based on the training data set; Using the canonical feature vector set, setting the initial parameter configuration of the XGBoost model; Based on feature importance analysis, key feature variables are screened from the standard feature vector set, and the screened feature variables are used to train the XGBoost model to obtain a number correctness evaluation model; Using the number correctness assessment model, setting a probability threshold; Using the number correctness assessment model to predict the numbers in the number database to obtain a prediction probability value; Compare the predicted probability value with the set probability threshold and output a number correctness assessment result; After outputting the number correctness evaluation result, the method further includes: For continuous features and categorical features, a feature distance calculation matrix for the number is constructed respectively, and a corresponding kernel function is designed to obtain a hybrid feature processing model; Using the hybrid feature processing model, processing the training data set, optimizing kernel function parameters, and forming a Gaussian process model; Selecting number samples whose prediction probability values ​​are between a first preset threshold and a second preset threshold from the prediction results of the number correctness assessment model, inputting the number samples into the Gaussian process model, and calculating the prediction uncertainty index to identify number samples that require manual verification; The feature distance calculation matrix for the number is constructed for the continuous feature and the categorical feature respectively, including: Based on the continuous features of the number, a radial basis function kernel is used to calculate the distance between features to form a continuous feature distance matrix, including: preprocessing the continuous features of the number to eliminate dimensional differences; calculating the Euclidean distance of any two samples on the continuous features of the number, and converting the Euclidean distance into a similarity measure using an RBF kernel function; forming an n×n continuous feature distance matrix, where n is the number of samples; Based on the categorical features of the number, the feature dissimilarity is calculated using the Hamming distance kernel to form a categorical feature distance matrix, including: using One-Hot Encoding to convert the categorical features of the number into binary vectors; for the encoded vectors of any two samples, their Hamming distance, i.e., the number of different bits, is calculated; for all pairs of samples in the data set, their similarity on the categorical features is calculated to form a categorical feature distance matrix; Performing a weighted combination of the continuous feature distance matrix and the categorical feature distance matrix to obtain the hybrid feature processing model; After outputting the number correctness evaluation result, the method further includes: Based on the number correctness assessment results, scenario-based processing is performed for the application requirements of number library management, data cleaning, and recommendation systems, and scenario application results are output; Collecting feedback data and newly added number samples from the scenario application results, updating the number correctness evaluation model, and forming an optimized number correctness evaluation model; Wherein, updating the number correctness evaluation model includes: Based on the newly added feedback number samples, the stochastic gradient descent algorithm with a constant learning rate is used to perform online parameter updates on the number correctness evaluation model; Input text description information, industry background knowledge, and historical verification records associated with the number into a large language model to extract semantic features and generate an unstructured feature vector. Fusing the unstructured feature vector with the original features of the number to form a fused vector; The fusion vector is used to update the number correctness evaluation model, thereby achieving iterative optimization of the number correctness evaluation model.

2. The method according to claim 1, characterized in that Determine a feature source definition list based on the training data set to form a standard feature vector set, including: Extracting the number's general attribute features, call behavior fluctuation features, source association features, and historical verification data features from the training data set to form a feature source definition list; According to the feature source definition list, the original feature data of the number is extracted, and the original feature data is coded and standardized to obtain a feature original value set.

3. The method according to claim 1, characterized in that Using the canonical feature vector set, the initial parameter configuration of the XGBoost model is set, including: Based on the data distribution characteristics of the canonical feature vector set, the objective function type of the model is set to a binary classification function, the maximum depth parameter of the tree is set to a preset depth value, and the learning rate parameter is set to a preset learning rate value, thereby forming a basic parameter configuration; According to the basic parameter configuration, the sample sampling ratio parameter and the sample balancing processing parameter are set to obtain the initial parameter configuration.

4. The method according to claim 1, wherein Based on the feature importance analysis, key feature variables are screened from the standard feature vector set, including: Based on the configured initial parameters, the XGBoost model is trained using the set of canonical feature vectors, and a feature importance score is calculated using the feature importance evaluation mechanism of the XGBoost model; The feature variables are screened and sorted according to the feature importance scores, a preset number of key feature variables are determined, and an optimized feature variable set is constructed.

5. The method according to claim 1, wherein The adopting the number correctness evaluation model to set a probability threshold comprises: Input the test data set into the number correctness evaluation model for verification, calculate the accuracy, precision and recall rate indicators, and form a model performance evaluation report, wherein the test data set is divided from the training data set according to a preset ratio during the training process of the XGBoost model; Utilize the model performance evaluation report to analyze the impact of different probability thresholds on model performance and draw a threshold-performance correspondence curve; The probability threshold is set according to the threshold-performance correspondence curve and combined with application requirements.

Images