Number classification verification method and related device

Abstract

The invention relates to the technical field of communication, and discloses a number classification verification method and related device.The method comprises the steps that a training data set is obtained, the training data set comprises multiple pieces of training data, the training data comprises marked numbers, feature extraction is conducted on the training data, and multi-dimensional feature vectors are obtained; and performing model training based on the multi-dimensional feature vector to obtain a classification model, obtaining a to-be-detected number, inputting the to-be-detected number into the classification model, detecting the stability of the to-be-detected number, obtaining a detection result, and performing secondary verification on the to-be-detected number when the detection result is determined to be unstable so as to determine a classification result of the to-be-detected number. According to the invention, the number marking accuracy is improved through twice verification of the number.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technology, and in particular to a number classification verification method and related devices. Background Art

[0002] With the rapid development of domestic mobile Internet, smartphones have become an indispensable part of people's daily lives. Number marking and identification services, as a mainstream function of smartphones, provide users with convenient previews of incoming call information and help users preliminarily screen the recipients of calls. However, existing technologies cannot effectively identify some numbers with short usage cycles and frequent changes, which may cause users to make decisions to answer or reject calls based on incorrect information, reducing the user experience. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present application provide a number classification verification method and related devices to solve the problem of inaccurate number recognition and improve the accuracy of number recognition.

[0004] In order to solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a number classification verification method, the method comprising:

[0006] Obtaining a training data set, wherein the training data set includes a plurality of training data, and the training data includes marked numbers;

[0007] Extract features from training data to obtain multi-dimensional feature vectors;

[0008] Perform model training based on multi-dimensional feature vectors to obtain a classification model;

[0009] Get the number to be tested;

[0010] Input the number to be tested into the classification model, detect the stability of the number to be tested, and obtain the test result;

[0011] When the detection result is determined to be unstable, the number to be tested is subjected to secondary verification to determine the classification result of the number to be tested.

[0012] In some embodiments, the dimensions include a first dimension, a second dimension, and a third dimension;

[0013] The first dimension is number characteristics, which include the operator corresponding to the number;

[0014] The second dimension is call behavior, which includes call frequency and the number of peer numbers;

[0015] The third dimension is the marking category, which includes the number of times a number is marked within a period of time.

[0016] In some embodiments, model training is performed based on multi-dimensional feature vectors to obtain a classification model, including:

[0017] Combine the feature vectors corresponding to the first dimension, the second dimension, and the third dimension to obtain a multi-dimensional feature vector;

[0018] The multi-dimensional feature vector is input into the model to obtain a classification model.

[0019] In some embodiments, the number to be tested is input into a classification model, the stability of the number to be tested is detected, and a detection result is obtained, including:

[0020] Pre-judge the number to be tested by the classification model to obtain an initial classification result, wherein the initial classification result includes the first category of the number to be tested and the probability of the first category;

[0021] According to the probability of the first category, the detection result of the number to be tested is determined.

[0022] In some embodiments, the detection result includes stable and unstable, and the detection result of the number to be tested is determined according to the probability of the first category, including:

[0023] If the probability of the first category is greater than or equal to the probability threshold, the detection result of the number to be tested is determined to be stable;

[0024] If the probability of the first category is less than the probability threshold, the detection result of the number to be tested is determined to be unstable.

[0025] In some embodiments, performing secondary verification on the number to be tested to determine the verification result of the number to be tested includes:

[0026] Querying the number to be tested through a webpage to determine a final classification result of the number to be tested, wherein the final classification result is the classification result of the number to be tested;

[0027] or,

[0028] The final classification result of the number to be tested is determined by dialing the number to be tested.

[0029] In some embodiments, the method further comprises:

[0030] The verification result is saved in the marked number library to update the numbers in the marked number library and their corresponding classification results, wherein the marked number library is used to save the classified numbers.

[0031] In a second aspect, an embodiment of the present application provides a number classification verification device, the device comprising:

[0032] A first acquisition module is used to acquire a training data set, wherein the training data set includes a plurality of training data, and the training data includes a known label number;

[0033] The extraction module is used to extract features from the training data to obtain multi-dimensional feature vectors;

[0034] A training module is used to perform model training based on multi-dimensional feature vectors to obtain a classification model;

[0035] The second acquisition module is used to acquire the number to be tested;

[0036] The detection module is used to input the number to be tested into the classification model, detect the stability of the number to be tested, and obtain the detection result;

[0037] The verification module is used to perform a secondary verification on the number to be tested when the detection result is determined to be unstable, so as to determine the classification result of the number to be tested.

[0038] In a third aspect, an embodiment of the present application provides a server, including:

[0039] A processor and a memory, the processor is used to execute the executable program code in the memory, and when the executable program code is executed, the processor executes the instructions of the number classification verification method of the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a number classification verification system, including:

[0041] The server of the third aspect is used to obtain a training data set, train a model based on the training data set, and obtain a classification model;

[0042] The electronic device is connected to the server and is used to run the classification model to prompt the classification result corresponding to the number during the call.

[0043] The beneficial effect of the implementation mode of the present application is: different from the prior art, the implementation mode of the present application provides a number classification verification method, the method comprising: obtaining a training data set, wherein the training data set comprises a plurality of training data, the training data comprises marked numbers, performing feature extraction on the training data to obtain a multi-dimensional feature vector, performing model training based on the multi-dimensional feature vector to obtain a classification model, obtaining a number to be tested, inputting the number to be tested into the classification model, detecting the stability of the number to be tested, obtaining a detection result, and when it is determined that the detection result is unstable, performing a secondary verification on the number to be tested to determine the classification result of the number to be tested. The present application improves the accuracy of number marking by secondary verification of the number. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] One or more embodiments are exemplarily described by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and the figures in the drawings do not constitute proportional limitations unless otherwise stated.

[0045] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0046] Figure 2 It is a flowchart of a number classification verification method provided by an embodiment of the present application;

[0047] Figure 3 yes Figure 2 A detailed flow chart of step S203 in FIG.

[0048] Figure 4 yes Figure 2 A detailed flow chart of step S205 in FIG.

[0049] Figure 5 yes Figure 4 A detailed flow chart of step S2502 in FIG.

[0050] Figure 6 yes Figure 2 A detailed flow chart of step S208 in FIG.

[0051] Figure 7 It is a schematic diagram of a process for updating a tag number library provided in an embodiment of the present application;

[0052] Figure 8 It is a schematic diagram of the overall process of a number classification verification method provided by an embodiment of the present application;

[0053] Fig. 9 It is a structural diagram of a number classification verification system provided by an embodiment of the present application;

[0054] Fig.10 It is a structural diagram of a number classification verification device provided in an embodiment of the present application;

[0055] Fig.11 It is a structural diagram of a server provided in an embodiment of the present application.

[0056] Description of Figure Numbers:

[0057] Label name Label name 100 Application Environment 1001 First acquisition module 10 server 1002 Extraction module 11 processor 1003 Training Module 12 Memory 1004 Second acquisition module 20 Electronic devices 1005 Detection module 900 Number Classification Verification System 1006 Authentication Module 1000 Number classification verification device DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, and all are within the scope of protection of the present application. In addition, the words "first", "second", etc. used in the present application do not limit the data, but only divide the same or similar items with basically the same functions and effects.

[0060] Before introducing the embodiments of the present application, a short message interception processing method known to the inventor of the present application is briefly introduced to facilitate subsequent understanding of the embodiments of the present application.

[0061] Existing smartphone number marking and identification services mainly mark incoming call numbers based on user feedback and preset databases.

[0062] The above solution has the following disadvantages:

[0063] (1) Over-reliance on user-generated labeling leads to frequent mislabeling and malicious labeling, which seriously affects the accuracy of the labeling information and causes users to make decisions to answer or reject calls based on incorrect information.

[0064] (2) Due to the short usage cycle of numbers and frequent changes, especially the rapid changes of marketing and other types of numbers, the existing technology has a lag when updating database information and is difficult to adapt to number changes in a timely manner, which causes users to continue to be disturbed by unnecessary calls.

[0065] In response to the above problems, the present application obtains a training data set, wherein the training data set includes multiple training data, and the training data includes known marked numbers, performs feature extraction on the training data to obtain a multi-dimensional feature vector, performs model training based on the multi-dimensional feature vector to obtain a classification model, obtains a number to be tested, inputs the number to be tested into the classification model, detects the stability of the number to be tested, obtains a test result, and when it is determined that the test result is unstable, performs a secondary verification on the number to be tested to determine the classification result of the number to be tested. The present application improves the accuracy of number recognition by secondary verification of the number.

[0066] The technical solution of the present application is described in detail below in conjunction with the accompanying drawings:

[0067] See also Figure 1 , Figure 1is a schematic diagram of an application environment provided by an embodiment of the present application;

[0068] like Figure 1 As shown, the application environment 100 includes a server 10 and an electronic device 20. The server 10 and the electronic device 20 are connected via a wireless network or a wired network. The wireless network includes wireless networks such as 2G, 3G, 4G, 5G, wireless LAN, and Bluetooth, and the wired network includes wired networks such as serial cables and network cables.

[0069] The server 10 is used for training a model to generate a classification model, and applying the classification model to the electronic device 20, wherein the classification model is used for pre-judging the number to be tested to output a classification result of the number to be tested.

[0070] Among them, server 10 is an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0071] The electronic device 20 is used to run the classification model to automatically identify the category of the current number during a call.

[0072] The electronic device 20 includes a mobile phone, a tablet computer, a smart wearable device (such as a smart watch), and other devices capable of making calls.

[0073] See also Figure 2 , Figure 2 It is a flowchart of a number classification verification method provided by an embodiment of the present application;

[0074] The number classification verification method is applied to a server. Specifically, the execution subject of the number classification verification method is one or at least two processors of the server.

[0075] like Figure 2 As shown, the number classification verification method includes:

[0076] Step S201: obtaining a training data set, wherein the training data set includes a plurality of training data, and the training data includes marked numbers;

[0077] In an embodiment of the present application, the server includes a database, which is used to store a marked number library, and the marked number library is used to store strange call log data, which includes historically marked numbers. The marked numbers include multiple categories, including express delivery, take-out, taxi, sales promotion, harassment, etc. Normally, when an electronic device receives a marked number, the category corresponding to the marked number will be displayed on the screen of the electronic device to prompt the user that the current number is risky.

[0078] In the embodiment of the present application, the numbers in the marked number library are numbers that have been manually verified, that is, the categories corresponding to the numbers in the marked number library have a higher accuracy.

[0079] Specifically, the marked numbers in the marked number library are obtained from the server as training data, and a plurality of marked numbers constitute a training data set, that is, the training data set includes a plurality of training data, and the training data includes the marked numbers.

[0080] Among them, the marked numbers correspond to multiple categories, including express delivery, takeout, taxi, sales promotion, harassment, etc. There are at least 30,000 marked numbers corresponding to each label.

[0081] In an embodiment of the present application, a training data set is used to train a model to obtain a classification model, and the classification model is used to predict the category of a number.

[0082] In an embodiment of the present application, the model is implemented based on a machine learning algorithm. For example, the machine learning algorithm includes an extreme gradient boosting algorithm (eXtreme Gradient Boosting, XGBoost). The XGBoost algorithm is an integrated learning algorithm based on a gradient boosting decision tree (Gradient Boosting Decision Tree, GBDT). The XGBoost algorithm introduces regularization terms and second-order derivative information on the basis of the GBDT algorithm. The XGBoost algorithm iteratively adds prediction trees, and each tree tries to correct the errors of the previous tree, thereby gradually building a powerful integrated model, that is, the classification model of the present application.

[0083] Step S202: extracting features from the training data to obtain a multi-dimensional feature vector;

[0084] In an embodiment of the present application, taking the XGBoost algorithm as an example, the training data is input into the model to train the model.

[0085] Specifically, the XGBoost algorithm is used to extract features from the training data to obtain a multi-dimensional feature vector.

[0086] In the embodiment of the present application, the multiple dimensions include a first dimension, a second dimension, and a third dimension. The first dimension is number characteristics, which include the operator corresponding to the number, the number's location, etc. The second dimension is call behavior, which includes call frequency, the number of peer numbers, changes in time periods, etc. The change in time period refers to the number of calls of a specific number in different time periods (such as working days and weekends, or within a specific time, such as 0:00-6:00, 7:00-9:00, 9:00-18:00, 11:00-13:00, 17:00-19:00, where working days refer to Monday to Friday, and weekends refer to Saturday and Sunday). By obtaining the changes in this time period, the differences in call or query behavior between weekdays and weekends can be compared to understand the usage patterns of specific numbers, the active time of the target user group, and possible business periodicity. For example, some phone numbers show a higher call frequency on weekdays, which may be related to work-related communication activities, while some phone numbers are more active on weekends, which may reflect the leisure or social needs of their user groups. The third dimension is the marking category, which includes the number of times the number is marked within a period of time, such as the number of times the number is marked within 6 months.

[0087] It should be noted that there are also features in other dimensions, which are not listed one by one here.

[0088] Specifically, features are extracted from the training data to obtain a multi-dimensional feature vector, wherein the multi-dimensional feature vector includes a feature vector corresponding to the first dimension, a feature vector corresponding to the second dimension, and a feature vector corresponding to the third dimension. The feature vector includes call frequency, number of opposite end numbers, changes in time periods, call behavior, etc.

[0089] In an embodiment of the present application, feature variables are extracted from training data based on multiple dimensions to obtain multi-dimensional feature vectors, for example, 277 feature vectors are obtained.

[0090] In an embodiment of the present application, the feature vector also includes the number of UIDs, query frequency within a specific time, geographic location, etc., wherein the number of UIDs refers to the number of active users or coverage of a phone number.

[0091] Step S203: Perform model training based on multi-dimensional feature vectors to obtain a classification model;

[0092] Specifically, a multi-dimensional feature vector is input into the model to train the model so that the model can learn the features of the marked numbers and obtain a classification model, wherein the classification model is used to predict the numbers to obtain the classification results of the numbers.

[0093] The classification results include categories and category probabilities. Categories include express delivery, takeout, taxi, sales promotion, harassment, etc. The probability of a category is the possibility of being classified into a certain category. The higher the probability of a category, the higher the possibility that the current number is classified into that category.

[0094] Please refer to Figure 3 , Figure 3 yes Figure 2 A detailed flow chart of step S203 in FIG.

[0095] like Figure 3 As shown, step S203 includes:

[0096] Step S231: combining the feature vectors corresponding to the first dimension, the second dimension, and the third dimension to obtain a multi-dimensional feature vector;

[0097] Specifically, feature fusion is performed on the feature vectors corresponding to the first dimension, the second dimension, and the third dimension to obtain a multi-dimensional feature vector.

[0098] In an embodiment of the present application, feature vectors of multiple dimensions are combined so that the model can learn and capture the complex relationships and interaction effects between features, thereby being able to process data sets with multiple attributes or features.

[0099] Step S232: input the multi-dimensional feature vector into the model to obtain a classification model;

[0100] Specifically, a multi-dimensional feature vector is input into the model to train the model and obtain a classification model.

[0101] In an embodiment of the present application, after the classification model is generated, it is necessary to further evaluate the performance of the model. Specifically, a test data set is obtained, wherein the test data set includes multiple test data, and the test data includes marked numbers. The test data set is input into the classification model to obtain a test result. The performance indicators of the model are calculated based on the test result. The performance indicators include accuracy, precision, recall rate, F1 score, AUC-ROC curve, etc. According to the performance indicators of the model, whether the performance of the model meets the requirements is evaluated. If the performance is not good, it is necessary to readjust the parameters of the model, increase the training data, and perform multiple rounds of training on the model to enhance the learning rate of the model until the model meets the performance requirements, that is, the model can accurately determine the category of the number.

[0102] Step S204: Obtain the number to be tested;

[0103] Specifically, a number to be tested is obtained from a database of a server, and the category of the number to be tested is unknown or known.

[0104] In an embodiment of the present application, when the category of the number to be tested is unknown, the category of the number to be tested can be predicted through the classification model. When the category of the number to be tested is known, the number to be tested is verified again through the classification model to determine whether its category is correct.

[0105] Step S205: input the number to be tested into the classification model, detect the stability of the number to be tested, and obtain the test result;

[0106] Specifically, the number to be tested is input into the classification model, and the initial classification result of the number to be tested is predicted by the classification model, the initial classification result includes the first category of the number to be tested and the probability of the first category, and the detection result of the number to be tested is determined according to the probability of the first category, that is, the stability of the number to be tested is determined according to the probability of the first category, wherein the first category is one of the categories, and the probability of the first category indicates the possibility that the number to be tested belongs to the first category, and the detection results include stable and unstable.

[0107] Among them, a stable number means that the number has a fixed call duration and frequency every day, and its call duration and frequency remain stable for a long time. An unstable number means that the communication behavior or pattern of the number has shown significant fluctuations or abnormalities over a period of time.

[0108] Please refer to the test results of the number to be tested. Figure 4 .

[0109] Please refer to Figure 4 , Figure 4 yes Figure 2 A detailed flow chart of step S205 in FIG.

[0110] like Figure 4 As shown, step S205 includes:

[0111] Step S2501: Pre-judge the number to be tested by using a classification model to obtain an initial classification result, wherein the initial classification result includes a first category of the number to be tested and a probability of the first category;

[0112] Specifically, the number to be tested is predicted through the classification model, and the initial classification result of the number to be tested is output, wherein the initial classification result includes the first category of the number to be tested and the probability of the first category, the first category is one of the categories, and the probability of the first category indicates the possibility that the number to be tested belongs to the first category.

[0113] Step S2502: Determine the detection result of the number to be tested according to the probability of the first category;

[0114] Specifically, a probability threshold is set, and the probability of the first category is compared with the probability threshold to determine the detection result of the number to be tested, that is, to determine whether the number to be tested is a stable number or an unstable number, and obtain the detection result, which includes stable and unstable. For specific steps to determine whether the number to be tested is stable, please refer to Figure 5 .

[0115] Please refer to Figure 5 , Figure 5 yes Figure 4 A detailed flow chart of step S2502 in FIG.

[0116] like Figure 5 As shown, step S2502 includes:

[0117] Step S2521: determining whether the probability of the first category is less than a probability threshold;

[0118] Specifically, determine whether the probability of the first category is less than the probability threshold. If the probability of the first category is greater than or equal to the probability threshold, jump to step S2522; if the probability of the first category is less than the probability threshold, jump to step S2523.

[0119] Step S2522: Determine that the detection result of the number to be tested is stable;

[0120] Specifically, when the probability of the first category is greater than or equal to the probability threshold, the detection result of the number to be tested is determined to be stable. For example, when the first category corresponding to the number to be tested is takeout, the probability of the first category corresponding to the number to be tested is 0.95, and the probability threshold is 0.9. The probability of the first category is greater than the probability threshold, then the first category corresponding to the number to be tested is determined to be takeout.

[0121] Step S2523: Determine that the detection result of the number to be tested is unstable;

[0122] Specifically, when the probability of the first category is less than the probability threshold, the detection result of the number to be tested is determined to be unstable. For example, when the first category corresponding to the number to be tested is takeout, the probability of the first category corresponding to the number to be tested is 0.85, and the probability threshold is 0.9. The probability of the first category is less than the probability threshold, then the first category corresponding to the number to be tested may be takeout.

[0123] In the embodiment of the present application, when it is determined that the detection result of the number to be tested is unstable, it is necessary to further verify the number to be tested to determine the classification result of the number to be tested.

[0124] In the embodiment of the present application, the probability threshold can be adjusted according to actual conditions.

[0125] Step S206: Determine whether the detection result is stable;

[0126] Specifically, when the probability of the first category corresponding to the number to be tested is greater than or equal to the probability threshold, the detection result is stable and jumps to step S207. When the probability of the first category corresponding to the number to be tested is less than the probability threshold, the detection result is unstable and jumps to step S208.

[0127] Step S207: Keep the classification result of the number to be tested unchanged;

[0128] Specifically, when the probability of the first category corresponding to the number to be tested is greater than or equal to the probability threshold, the detection result is stable, and the classification result of the number to be tested remains unchanged, that is, the final classification result of the number to be tested is determined to be the first category.

[0129] Step S208: perform secondary verification on the number to be tested to determine the classification result of the number to be tested; in the embodiment of the present application, when the probability of the first category corresponding to the number to be tested is less than the probability threshold, the detection result is unstable, that is, whether the category of the number to be tested is the first category needs to be further determined.

[0130] Among them, the classification result includes the second category.

[0131] Specifically, the number to be tested is verified twice to determine the classification result of the number to be tested, that is, to determine that the number to be tested is a second category, the second category is one of the categories, the second category may be the same as the first category, or the second category may be different from the first category. For specific verification steps, please refer to Figure 6 .

[0132] Please refer to Figure 6 , Figure 6 yes Figure 2 A detailed flow chart of step S208 in FIG.

[0133] like Figure 6 As shown, step S208 includes:

[0134] Step S281: querying the number to be tested through a web page to determine the final classification result of the number to be tested;

[0135] Specifically, the number to be tested is queried through a web page to determine the final classification result of the number to be tested. The final classification result is the classification result of the number to be tested. For example, on some open websites, such as the E164 number query website, the number to be tested is directly entered on the website, and the website will directly return the query result. The query result includes the specific category, risk level, and usage status (such as normal, empty number, shut down, etc.). The final classification result of the number to be tested can be determined through the query result.

[0136] Step S282: dialing the number to be tested to determine the final classification result of the number to be tested;

[0137] Specifically, directly dial the number to be tested to obtain the outbound dialing result. The outbound dialing result is obtained by observing the call connection status, the other party's response, and abnormal situations during the call (such as voice prompts, busy tones, inability to connect, etc.). Based on the outbound dialing result, the final classification result of the number to be tested can be determined.

[0138] In the embodiment of the present application, the final classification result of the number to be tested can also be determined by manual investigation.

[0139] In the embodiment of the present application, by performing a secondary verification on the number to be tested, the category of the number to be tested can be accurately determined, and unnecessary human and financial expenditures can be reduced.

[0140] In an embodiment of the present application, after the classification result of the number to be tested is confirmed, the number to be tested and the classification result of the number to be tested need to be saved to a marked number library, and the marked number library is updated, so that in actual application, the number can be effectively marked based on the marked number library.

[0141] Please refer to Figure 7 , Figure 7 It is a schematic diagram of a process for updating a tag number library provided in an embodiment of the present application;

[0142] like Figure 7 As shown, the process of updating the tag number library includes:

[0143] Step S701: saving the number to be tested and the second category of the number to be tested to a marked number library to update the numbers in the marked number library and their corresponding classification results, wherein the marked number library is used to store classified numbers;

[0144] Specifically, after the classification result of the number to be tested is determined, the number to be tested and the classification result of the number to be tested are saved in a marked number library, so that the number to be tested corresponds to one category.

[0145] In an embodiment of the present application, in actual application, after the classification model is applied to an electronic device, when the electronic device receives a call, the classification model automatically identifies the category of the current call number and displays the category of the current call number on the display screen of the electronic device.

[0146] In an embodiment of the present application, the classification model can also be used to detect whether the numbers in the marked number library are marked correctly. Specifically, the numbers in the marked number library are input into the classification model, and the classification model can output the classification results corresponding to each number. The classification results include categories and category probabilities. According to the probability of the categories, it can be determined whether the categories corresponding to the numbers originally existing in the marked number library are correct. If not, the categories of the original numbers are modified and the marked number library is updated again. The present application can not only mark unmarked numbers, but also check the numbers in the existing marked number library to supplement and update the marked number library.

[0147] See also Figure 8 , Figure 8 It is a schematic diagram of the overall process of a number classification verification method provided by an embodiment of the present application;

[0148] like Figure 8 As shown, the overall process of the number classification verification method includes:

[0149] Step S801: Obtain a training data set;

[0150] Step S802: extracting features from the training data to obtain a multi-dimensional feature vector;

[0151] Specifically, the XGBoost algorithm is used to extract features from the training data to obtain a multi-dimensional feature vector.

[0152] Step S803: training the model based on the multi-dimensional feature vector;

[0153] Specifically, the multi-dimensional feature vector is input into the model to train the model and obtain a classification model.

[0154] Step S804: determine whether the number to be tested is stable;

[0155] Specifically, it is determined whether the number to be tested is stable through the classification model. If the number to be tested is stable, the process jumps to step S805; if the number to be tested is unstable, the process jumps to step S806.

[0156] Step S805: Determine whether the number to be tested is stable;

[0157] Specifically, when the probability of the first category corresponding to the number to be tested is greater than or equal to the probability threshold, it is determined that the number to be tested is stable.

[0158] Step S806: Determine that the number to be tested is unstable;

[0159] Specifically, when the probability of the first category corresponding to the number to be tested is less than the probability threshold, it is determined that the number to be tested is unstable.

[0160] Step S807: secondary verification to determine the classification result;

[0161] Specifically, when it is determined that the number to be tested is unstable, a secondary verification is performed on the number to be tested to determine a final classification result of the number to be tested.

[0162] Step S808: Update the tag number library;

[0163] Specifically, when it is determined that the number to be tested is stable, the number to be tested and the category corresponding to the number to be tested are saved in the marked number library; when it is determined that the number to be tested is unstable, the final classification result of the number to be tested obtained after the secondary verification is saved in the marked number library.

[0164] See also Fig. 9 , Fig. 9 It is a structural diagram of a number classification verification system provided by an embodiment of the present application;

[0165] like Fig. 9 As shown, the number classification verification system 900 includes a server 10 and an electronic device 20. The server 10 and the electronic device 20 are connected via a wireless network or a wired network.

[0166] The server 10 is connected to the electronic device 20 . The server 10 is used to obtain a training data set, train a model based on the training data set, obtain a classification model, and deploy the classification model in the electronic device 20 .

[0167] The electronic device 20 is connected to the server 10, and is used to run the classification model to prompt the classification result corresponding to the number during the call.

[0168] In the embodiment of the present application, the classification model can improve the number recognition capability of the electronic device 20, so as to prompt the user in time, thereby preventing the user from being continuously and unnecessaryly disturbed.

[0169] See also Fig.10 , Fig.10 It is a structural diagram of a number classification verification device provided in an embodiment of the present application;

[0170] See also Fig.10 , Fig.10 It is a structural diagram of a number classification verification device provided in an embodiment of the present application;

[0171] like Fig.10 As shown, the number classification verification device 1000 includes:

[0172] A first acquisition module 1001 is used to acquire a training data set, wherein the training data set includes a plurality of training data, and the training data includes marked numbers;

[0173] Extraction module 1002, used to extract features from training data to obtain multi-dimensional feature vectors;

[0174] A training module 1003 is used to perform model training based on multi-dimensional feature vectors to obtain a classification model;

[0175] The second acquisition module 1004 acquires the number to be tested;

[0176] The detection module 1005 is used to input the number to be tested into the classification model, detect the stability of the number to be tested, and obtain the detection result;

[0177] The verification module 1006 is used to perform a secondary verification on the number to be tested when the detection result is determined to be unstable, so as to determine the classification result of the number to be tested.

[0178] In an embodiment of the present application, the number classification verification device can be a software module, which includes a number of instructions stored in a memory. The processor can access the memory and call the instructions for execution to complete the number classification verification method of each of the above embodiments.

[0179] In the embodiments of the present application, the number classification verification device can also be constructed by hardware devices. For example, the number classification verification device can be constructed by one or more chips, and each chip can work in coordination with each other to complete the number classification verification method described in the above embodiments. For another example, the number classification verification device can also be constructed by various logic devices, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components.

[0180] The number classification verification device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0181] The number classification verification device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0182] The number classification verification device provided in the embodiment of the present application can achieve Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0183] It should be noted that the above device can execute the number classification verification method provided in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in the device embodiment, please refer to the number classification verification method provided in the embodiment of the present application.

[0184] In the embodiment of the present application, the various modules of the number classification and verification device cooperate with each other to solve the problem of low number recognition rate in the prior art, thereby improving the accuracy of number marking.

[0185] See also Fig.11 , Fig.11 It is a structural diagram of a server provided in an embodiment of the present application.

[0186] like Fig.11 As shown, the server 10 includes one or more processors 11 and a memory 12. Fig.11 A processor 11 is taken as an example.

[0187] The processor 11 and the memory 12 may be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0188] A processor, configured to execute the number classification verification method in any embodiment of the present application, comprising:

[0189] Obtaining a training data set, wherein the training data set includes a plurality of training data, and the training data includes marked numbers;

[0190] Extract features from training data to obtain multi-dimensional feature vectors;

[0191] Perform model training based on multi-dimensional feature vectors to obtain a classification model;

[0192] Get the number to be tested;

[0193] Input the number to be tested into the classification model, detect the stability of the number to be tested, and obtain the test result;

[0194] When the detection result is determined to be unstable, the number to be tested is subjected to secondary verification to determine the classification result of the number to be tested.

[0195] The memory 12 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the number classification verification method in the embodiment of the present invention. The processor 11 executes various functional applications and data processing of the electronic device by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, the number classification verification method of the above method embodiment is implemented.

[0196] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0197] One or more modules are stored in the memory 12, and when executed by one or more processors 11, the number classification verification method in any of the above method embodiments is executed, for example, the number classification verification method described above is executed. Figure 2 The steps shown.

[0198] An embodiment of the present application also provides a computer program product, which includes one or more program codes, and the program code is stored in a non-volatile computer-readable storage medium. The processor of the electronic device reads the program code from the non-volatile computer-readable storage medium, and the processor executes the program code to complete the steps of the number classification verification method provided in the above embodiment.

[0199] Through the description of the above implementation methods, ordinary technicians in this field can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or by hardware related to program code, and the program can be stored in a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium mentioned above can be a read-only memory, a disk or an optical disk, etc.

[0200] Through the description of the above implementation methods, a person of ordinary skill in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the non-volatile computer-readable storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as mentioned above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A number classification verification method, characterized in that: The method comprises: Acquire a training data set, wherein the training data set includes a plurality of training data, and the training data includes marked numbers; Performing feature extraction on the training data to obtain a multi-dimensional feature vector; Performing model training based on the multi-dimensional feature vector to obtain a classification model; Get the number to be tested; Inputting the number to be tested into a classification model, detecting the stability of the number to be tested, and obtaining a detection result; When it is determined that the detection result is unstable, a secondary verification is performed on the number to be tested to determine the classification result of the number to be tested.

2. The method according to claim 1, characterized in that The dimensions include a first dimension, a second dimension, and a third dimension; The first dimension is number characteristics, and the number characteristics include the operator corresponding to the number; The second dimension is call behavior, which includes call frequency and the number of opposite-end numbers; The third dimension is the marking category, which includes the number of times a number is marked within a period of time.

3. The method according to claim 2, characterized in that The performing model training based on the multi-dimensional feature vector to obtain a classification model includes: Combining the feature vectors corresponding to the first dimension, the second dimension, and the third dimension to obtain a multi-dimensional feature vector; The multi-dimensional feature vector is input into the model to obtain the classification model.

4. The method according to claim 1, characterized in that: Inputting the number to be tested into a classification model, detecting the stability of the number to be tested, and obtaining a test result, including: Pre-judgment of the number to be tested is performed using the classification model to obtain an initial classification result, wherein the initial classification result includes a first category of the number to be tested and a probability of the first category; The detection result of the number to be tested is determined according to the probability of the first category.

5. The method according to claim 4, characterized in that The detection result includes stable and unstable, and the determination of the detection result of the number to be tested according to the probability of the first category includes: If the probability of the first category is greater than or equal to the probability threshold, determining that the detection result of the number to be tested is stable; If the probability of the first category is less than the probability threshold, it is determined that the detection result of the number to be tested is unstable.

6. The method according to claim 5, characterized in that The performing secondary verification on the number to be tested to determine the verification result of the number to be tested includes: Querying the number to be tested through a webpage to determine a final classification result of the number to be tested, wherein the final classification result is a classification result of the number to be tested; or, The final classification result of the number to be tested is determined by dialing the number to be tested.

7. The method according to claim 6, characterized in that The classification result includes a second category, and the method further includes: The number to be tested and the second category of the number to be tested are saved in a marked number library to update the numbers in the marked number library and their corresponding classification results, wherein the marked number library is used to save the classified numbers.

8. A number classification verification device, characterized in that: The device comprises: A first acquisition module is used to acquire a training data set, wherein the training data set includes a plurality of training data, and the training data includes a known label number; An extraction module, used to extract features from the training data to obtain a multi-dimensional feature vector; A training module, used for performing model training based on the multi-dimensional feature vector to obtain a classification model; The second acquisition module is used to acquire the number to be tested; A detection module, used for inputting the number to be tested into a classification model, detecting the stability of the number to be tested, and obtaining a detection result; The verification module is used to perform a secondary verification on the number to be tested when it is determined that the detection result is unstable, so as to determine the classification result of the number to be tested.

9. A server, characterized in that: include: A processor and a memory, wherein the processor is used to execute an executable program code in the memory, and when the executable program code is executed, the processor executes instructions of the number classification verification method as described in any one of claims 1 to 7.

10. A number classification verification system, characterized in that: include: The server according to claim 9, used to obtain a training data set, train a model based on the training data set, and obtain a classification model; The electronic device is connected to the server and is used to run the classification model to prompt the classification result corresponding to the number during the call.

Images