Phone number identification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a phone number identification method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: carrying out number identification on a target phone number and target operation service data through an original number identification model to obtain a first number category, carrying out number identification on the target phone number and target outgoing call data of each call through a call regression model of each call to obtain a second number category of each call, performing mean value calculation on the second number category of multiple calls to obtain a reference predicted number category, performing aggregation calculation on the first number category and the reference predicted number category to obtain a target number category of the target telephone number, the target number category being used for indicating that the target telephone number is a normal number or an abnormal number, the accuracy of telephone number identification can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a telephone number recognition method and device, an electronic device and a storage medium. Background Art

[0002] With the widespread application of Internet technology and mobile communication technology, contactless fraud activities carried out with the help of communication tools such as mobile phones and landlines frequently occur. In order to protect the property and privacy of users, it is necessary to identify whether a phone number is a fraudulent number. In related technologies, manual experience rules are used to identify phone numbers. However, manual experience rules are often based on limited subjective experience and cannot cover the characteristics of all phone numbers, resulting in low accuracy in phone number recognition. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide a telephone number recognition method and device, an electronic device and a storage medium, aiming to improve the accuracy of telephone number recognition.

[0004] To achieve the above object, a first aspect of an embodiment of the present application provides a telephone number recognition method, the method comprising:

[0005] Obtaining a target phone number, target operating service data of the target phone number, and target outgoing call data of multiple calls;

[0006] The target telephone number and the target operation service data are identified by an original number identification model to obtain a first number category; wherein the original number identification model is obtained by minimizing the difference between the predicted first number category output based on the sample telephone number and the sample operation service data and the sample number category label;

[0007] The target telephone number and the target outgoing call data of each call are identified by a call regression model for each call, so as to obtain a second number category for each call; wherein the call regression model is obtained by minimizing the difference between the predicted second number category output based on the sample telephone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample telephone number and the sample operation service data and the sample number category label;

[0008] Calculate the average of the second number categories of multiple calls to obtain a reference predicted number category;

[0009] The first number category and the reference predicted number category are aggregated and calculated to obtain a target number category of the target telephone number; wherein the target number category is used to indicate whether the target telephone number is a normal number or an abnormal number.

[0010] In some embodiments, after performing aggregate calculation on the first number category and the reference predicted number category to obtain the target number category of the target phone number, the phone number identification method further includes:

[0011] Obtaining a test telephone number, test operation service data of the test telephone number, test outgoing call data of multiple calls, and a test number category label;

[0012] Obtaining a data quantile threshold according to the test telephone number, the test operation service data, the test outgoing call data of multiple calls, the test number category label, the original number recognition model, and the call regression model of each call;

[0013] The number recognition confidence interval is calculated according to the target number category and the data quantile threshold.

[0014] In some embodiments, obtaining a data quantile threshold according to the test phone number, the test operation service data, the test outgoing call data of multiple calls, the test number category label, the original number recognition model, and the call regression model of each call includes:

[0015] Performing number identification on the test telephone number and the test operation service data by using the original number identification model to obtain a first test number category;

[0016] For each call, performing number recognition on the test telephone number and the test outgoing call data by using the call regression model to obtain a second test number category;

[0017] determining a confidence score for each call based on the test number category label, the first test number category, and the second test number category for each call;

[0018] Sorting the confidence scores of the multiple calls to obtain a score sequence;

[0019] The data quantile threshold is calculated according to the score sequence and a preset confidence level, wherein the preset confidence level is used to indicate the proportion of confidence scores in the score sequence that are less than or equal to the data quantile threshold.

[0020] In some embodiments, determining the confidence score of each call according to the test number category label, the first test number category and the second test number category of each call includes:

[0021] Calculating the difference between the test number category label and the first test number category to obtain a first category difference;

[0022] Calculating the difference between the first category difference and the second test number category to obtain a first score;

[0023] Calculating the difference between the second test number category and the test number category label to obtain a second category difference;

[0024] performing aggregation calculation on the second category difference and the first test number category to obtain a second score;

[0025] The first score and the second score are screened to obtain the confidence score.

[0026] In some embodiments, the calculating the number recognition confidence interval according to the target number category and the data quantile threshold includes:

[0027] Calculate the difference between the target number category and the data quantile threshold to obtain a lower confidence limit value;

[0028] Performing aggregation calculation according to the target number category and the data quantile threshold to obtain an upper confidence limit value;

[0029] An interval is constructed according to the confidence lower limit value and the confidence upper limit value to obtain the number recognition confidence interval.

[0030] In some embodiments, the call regression model for each call is trained according to the following steps:

[0031] Performing number recognition on the sample telephone number and the sample operation service data by using the original number recognition model to obtain the predicted intermediate number category;

[0032] Calculate the difference between the sample number category label and the predicted intermediate number category to obtain the target difference value;

[0033] For each call, performing number recognition on the sample telephone number and the sample outgoing call data by using a preset original regression model to obtain the predicted second number category;

[0034] Perform loss calculation on the target difference and the predicted second number category of each call respectively to obtain first loss data for each call;

[0035] The model parameters of the preset original regression model are updated respectively according to the first loss data of each call to obtain the call regression model of each call.

[0036] In some embodiments, the original number recognition model is trained according to the following steps:

[0037] Performing number recognition on the sample telephone number and the sample operation service data by using a preset number recognition model to obtain the predicted first number category;

[0038] Calculate the loss according to the predicted first number category and the sample number category label to obtain second loss data;

[0039] The model parameters of the preset number recognition model are updated according to the second loss data to obtain the original number recognition model.

[0040] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a telephone number recognition device, the device comprising:

[0041] A data acquisition module, used to acquire a target phone number, target operation service data of the target phone number, and target outgoing call data of multiple calls;

[0042] A first number recognition module is used to perform number recognition on the target telephone number and the target operation service data through an original number recognition model to obtain a first number category; wherein the original number recognition model is obtained by minimizing the difference between the predicted first number category output based on the sample telephone number and the sample operation service data and the sample number category label;

[0043] A second number identification module is used to identify the target telephone number and the target outgoing call data of each call through a call regression model for each call, and obtain a second number category for each call; wherein the call regression model is obtained by minimizing the difference between the predicted second number category output based on the sample telephone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample telephone number and the sample operation service data and the sample number category label;

[0044] A first calculation module is used to calculate the average of the second number categories of multiple calls to obtain a reference predicted number category;

[0045] The second calculation module is used to perform aggregate calculation on the first number category and the reference predicted number category to obtain a target number category of the target telephone number; wherein the target number category is used to indicate whether the target telephone number is a normal number or an abnormal number.

[0046] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0047] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0048] The telephone number identification method, telephone number identification device, electronic device and computer-readable storage medium of the embodiment of the present application obtain the target telephone number, the target operation service data of the target telephone number and the target outgoing call data of multiple calls to determine whether the target telephone number is a fraudulent number based on the target operation service data and the target outgoing call data. In order to solve the problem of inaccurate telephone number identification caused by artificial experience rules, a deep learning model is used for number identification. Different models can capture different telephone number features. In order to accurately determine whether the telephone number is a fraudulent number, the original number identification model and the call regression model of each call are combined to perform the number identification task. The operation service data contains information such as number package and number activation time, which can be used to identify whether the telephone number is abnormal. The target telephone number and the target operation service data are identified by the original number identification model to obtain the first predicted number category. The information provided by the operation service data is limited. In order to further improve the accuracy of telephone number identification, the sample outgoing call data of multiple calls are used as supplementary information. The target telephone number and the target outgoing call data of each call are identified by the call regression model of each call to obtain the second predicted number category of each call. A phone number may have multiple calls, and the call time, call duration and other characteristics of different calls are different. In order to avoid large differences in prediction results caused by different calls, the second predicted number category of multiple calls is averaged to obtain the reference predicted number category. The first predicted number category and the reference predicted number category are aggregated to obtain the target number category of the target phone number. By integrating the number categories predicted by each model, the overall prediction error can be reduced and the accuracy of phone number recognition can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1is a flowchart of a telephone number identification method provided by an embodiment of the present application;

[0050] Figure 2 is a flow chart of the training process of the original number recognition model provided in an embodiment of the present application;

[0051] Figure 3 is a flow chart of the training process of the call regression model for each call provided in an embodiment of the present application;

[0052] Figure 4 is another flow chart of the telephone number identification method provided in an embodiment of the present application;

[0053] Figure 5 yes Figure 4 Flow chart of step S420 in FIG.

[0054] Figure 6 yes Figure 5 Flow chart of step S530 in FIG.

[0055] Figure 7 yes Figure 4 Flow chart of step S430 in FIG.

[0056] Figure 8 is a schematic diagram of the structure of a telephone number identification device provided in an embodiment of the present application;

[0057] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0061] With the widespread application of Internet technology and mobile communication technology, contactless fraud activities carried out with the help of communication tools such as mobile phones and landlines frequently occur. In order to protect the property and privacy of users, it is necessary to identify whether a phone number is a fraudulent number. In related technologies, manual experience rules are used to identify phone numbers. However, manual experience rules are often based on limited subjective experience and cannot cover the characteristics of all phone numbers, resulting in low accuracy in phone number recognition.

[0062] Based on this, the embodiments of the present application provide a telephone number recognition method, a telephone number recognition device, an electronic device and a computer-readable storage medium, aiming to improve the accuracy of telephone number recognition.

[0063] The telephone number identification method, telephone number identification device, electronic device and computer-readable storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the telephone number identification method in the embodiments of the present application is described.

[0064] The telephone number recognition method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The telephone number recognition method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the telephone number recognition method, etc., but is not limited to the above forms.

[0065] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0066] Figure 1 It is an optional flowchart of the telephone number identification method provided in the embodiment of the present application, which may include but is not limited to steps S110 to S150.

[0067] Step S110, obtaining a target phone number, target operating service data of the target phone number, and target outgoing call data of multiple calls;

[0068] Step S120, performing number recognition on the target telephone number and the target operation service data through the original number recognition model to obtain a first number category; wherein the original number recognition model is obtained by minimizing the difference between the predicted first number category output based on the sample telephone number and the sample operation service data and the sample number category label;

[0069] Step S130, performing number recognition on the target telephone number and the target outgoing call data of each call through the call regression model of each call, and obtaining the second number category of each call; wherein the call regression model is obtained by minimizing the difference between the predicted second number category output based on the sample telephone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number recognition model based on the sample telephone number and the sample operation service data and the sample number category label;

[0070] Step S140, calculating the average of the second number categories of multiple calls to obtain a reference predicted number category;

[0071] Step S150, performing aggregation calculation on the first number category and the reference predicted number category to obtain a target number category of the target telephone number; wherein the target number category is used to indicate whether the target telephone number is a normal number or an abnormal number.

[0072] In step S110 of some embodiments, the target telephone number, the target operation service data of the target telephone number, and the target outgoing call data of multiple calls are obtained. The target telephone number is the telephone number to be identified. The target operation service data is the commercial service data provided by the operator for the target telephone number, including the package information of the target telephone number (such as package type, fee, package period, etc.), number activation time, etc. The target outgoing call data of multiple calls is the outgoing call data of the target telephone number for the most recent m times, and the outgoing call data includes the call time, call duration, the number of historical calls with the called phone, etc., where m is an integer greater than 1 or equal to 1. The embodiment of the present application does not limit the number of calls, and the number of calls for different telephone numbers may be different. The outgoing call data is data of call granularity, and usually changes with the change of the call.

[0073] In step S120 of some embodiments, the target operation service data is associated with the target phone number, which is user-granular data and does not change with the call. It can be directly used to determine whether the target phone number is a fraudulent number. The target phone number and the target operation service data are input into the original number recognition model for number recognition to obtain a first number category. The first number category is used to indicate whether the target phone number is a normal number or an abnormal number. The abnormal number is a fraudulent number. The traditional number recognition method uses artificial experience rules for recognition, which requires a high level of artificial professionalism and has high labor and time costs. Compared with the manual recognition method, the original number recognition model greatly improves the efficiency and accuracy of fraudulent number recognition.

[0074] See also Figure 2 In some embodiments, the training process of the original number recognition model may include but is not limited to steps S210 to S230:

[0075] Step S210, performing number recognition on the sample telephone number and the sample operation service data by using a preset number recognition model to obtain a predicted first number category;

[0076] Step S220, performing loss calculation according to the predicted first number category and the sample number category label to obtain second loss data;

[0077] Step S230, updating the model parameters of the preset number recognition model according to the second loss data to obtain the original number recognition model.

[0078] In step S210 of some embodiments, a data set is obtained, which includes telephone numbers, characteristic data of telephone numbers, and number category labels, and the characteristic data is divided into operation service data and outgoing call data. A sub-data set is constructed based on all telephone numbers in the data set, the operation service data corresponding to the telephone numbers, and the number category labels, and the original number recognition model is trained on the sub-data set.

[0079] Obtain sample phone numbers, sample operation service data of sample phone numbers, and sample number category labels from the sub-dataset. The sample phone number is the phone number used for model training, the sample operation service data is the commercial service data provided by the operator for the sample phone number, including the package information of the sample phone number, number activation time, etc., and the sample number category label is the actual number category of the sample phone number, which can be a normal category or an abnormal category. The preset number recognition model is a binary classification model, which can be constructed using a tree model or a deep model. The sample phone number and sample operation service data are input into the preset number recognition model for number recognition to obtain a predicted first number category. The predicted first number category is the number category predicted for the sample phone number by the preset number recognition model.

[0080] In step S220 of some embodiments, based on the cross entropy loss function, a loss calculation is performed on the predicted first number category and the sample number category label, and the loss value obtained by the loss calculation is used as the second loss data to guide the model training process of the preset number recognition model based on the second loss data, so that the predicted first number category is close to the sample number category label. The cross entropy loss function is used to measure the difference between the predicted value and the actual label.

[0081] In step S230 of some embodiments, the second loss data is minimized, and the model parameters of the preset number recognition model are adjusted to obtain the original number recognition model. The model training optimization process of the original number recognition model is expressed as:

[0082] min q ∑ o∈[N] L(q(b i ),y i ),

[0083] Where q represents the preset number recognition model; N is the number of sample phone numbers; i represents the i-th sample phone number; L is the cross entropy loss function; b i is the sample operation service data of the i-th sample phone number; y i is the sample number category label of the i-th sample phone number.

[0084] Through the above steps S210 to S230, an original number recognition model can be trained to perform preliminary number recognition based on the original number recognition model.

[0085] In step S130 of some embodiments, in the related art, the telephone statistical features of a single call are used to predict fraud for a single call, and the prediction is used to directly determine whether the incoming call belongs to a fraud number. However, a phone number may have multiple calls, and different calls have different characteristics such as call time and call duration, which will make the prediction results of the same phone number vary greatly. In addition, the original number recognition model only predicts numbers through operation service data, and the information provided by the operation service data is limited, which may cause the problem of inaccurate number discrimination. In order to improve the accuracy of number recognition, the embodiment of the present application makes full use of the data characteristics of operation service data and outgoing call data, and uses the target outgoing call data of multiple calls as supplementary information to make up for the shortcomings of untimely update of operation service data and less available information. For each call, the target phone number and the target outgoing call data of the current call are identified through the call regression model of the current call, and the second number category of the current call is output.

[0086] See also Figure 3In some embodiments, the training process of the call regression model for each call may include but is not limited to steps S310 to S350:

[0087] Step S310, performing number recognition on the sample telephone number and the sample operation service data through the original number recognition model to obtain a predicted intermediate number category;

[0088] Step S320, performing difference calculation based on the sample number category label and the predicted intermediate number category to obtain a target difference value;

[0089] Step S330, for each call, number recognition is performed on the sample telephone number and the sample outgoing call data by using a preset original regression model to obtain a predicted second number category;

[0090] Step S340, respectively calculating the loss of the target difference and the predicted second number category of each call to obtain first loss data of each call;

[0091] Step S350, updating the model parameters of the preset original regression model according to the first loss data of each call, to obtain a call regression model for each call.

[0092] In step S310 of some embodiments, the data set includes N data samples, each data sample includes a telephone number, feature data of the telephone number, and a number category label, the feature data includes operation service data and outgoing call data, and the data set is divided into a training set and a test set by randomly sampling the telephone numbers, and the training set is applied to the model training process, and the test set is applied to the model testing process.

[0093] If the data set is represented by D 1 ={({O i,j} k j＝1 ,b i ,y i )} i∈[N] , where k is the number of calls, k and the number of calls m of the target phone number can be the same or different, k is greater than or equal to m, j represents the jth call, o i,j is the jth outgoing call data of the ith phone number, b i is the operation service data of the ith phone number, y i is the number category label of the ith phone number, N is the number of samples in the data set, and the training set is represented as N 1 is the number of samples in the training set, and the test set is expressed as The number of samples in the test set is NN 1 .

[0094] The call regression model is trained on the training set, and sample telephone numbers, sample operation service data of the sample telephone numbers, sample outgoing call data of multiple calls, and sample number category labels are obtained from the training set. The sample telephone numbers and sample operation service data are identified through the original number recognition model to determine whether the sample telephone number is a fraud number and obtain the predicted intermediate number category.

[0095] In step S320 of some embodiments, in order to measure the difference between the predicted value and the true value of the original number recognition model, the sample number category label and the predicted intermediate number category are subtracted to obtain a target difference.

[0096] In step S330 of some embodiments, a phone number usually has multiple calls, and each call can be used as a basis for determining whether the phone number is a fraudulent number, so a call regression model based on multiple calls is designed. K preset original regression models are designed as weak learners, and the preset original regression model is a binary classification model. The embodiment of the present application does not limit the preset original regression model, which can be a tree model or a deep model. For each call, the sample phone number and the sample outgoing call data of the current call are identified by the preset original regression model of the current call to obtain the predicted second number category of the current call.

[0097] In step S340 of some embodiments, for each call, a loss calculation is performed on the target difference and the predicted second number category of the current call based on the cross-entropy loss function to obtain the first loss data of the current call, so that the preset original regression model can learn the prediction error of the original number recognition model through differential learning, so that the preset original regression model can focus on learning the data features that the original number recognition model cannot capture, thereby improving the overall prediction accuracy of the model.

[0098] In step S350 of some embodiments, for each call, the first loss data of the current call is minimized, and the model parameters of the preset original regression model of the current call are updated to obtain a call regression model of the current call.

[0099] For the sth call regression model, the data used is The training process is expressed as:

[0100]

[0101] Among them, g s is the preset original regression model; L is the cross entropy loss function; o i,s b is the sample outgoing call data of the sth call of the i-th sample phone number; b i is the sample operation service data of the i-th sample phone number; yi is the sample number category label of the i-th sample phone number; N 1 is the number of samples in the training set.

[0102] Through the above steps S310 to S350, a call regression model for each call can be obtained, so that accurate number prediction can be performed for each call and the prediction error of the original number recognition model can be compensated.

[0103] In step S140 of some embodiments, a phone number has multiple calls, and the call time, call duration and other characteristics of different calls are different, so that the number prediction results of multiple calls of the same phone number are different. In order to avoid large differences in number prediction results of different calls, the second number categories of multiple calls are averaged to obtain a reference prediction number category. The average calculation process is defined as:

[0104]

[0105] Where m is the number of calls to the target phone number, 1≤m≤k; g s is the call regression model for the sth call; O s The target outgoing call data for the target phone number for the sth time.

[0106] In step S150 of some embodiments, the first number category and the reference predicted number category are summed to obtain the target number category of the target phone number. The target number category is used to indicate whether the target phone number is a normal number or an abnormal number. The calculation formula of the target number category is expressed as:

[0107]

[0108] Where b is the target operation service data of the target phone number; q represents the original number recognition model; g s is the call regression model of the sth call; m is the number of calls to the target phone number; o s The target outgoing call data for the target phone number for the sth time.

[0109] In the related art, the prediction method based on artificial intelligence only outputs the predicted probability of the number category, and the predicted probability cannot accurately reflect the trustworthiness of the prediction result, which makes it difficult to accurately evaluate the prediction result. If the abnormal phone number with incorrect prediction is directly blocked or other disabling measures are taken, it will bring a very bad user experience. The embodiment of the present application adopts a common prediction framework to calculate the confidence of this prediction based on the original number recognition model and the call regression model of each call. The confidence calculation process is described in detail below.

[0110] See also Figure 4In some embodiments, after step S150, the telephone number identification method may further include but is not limited to steps S410 to S430:

[0111] Step S410, obtaining a test phone number, test operation service data of the test phone number, test outgoing call data of multiple calls, and a test number category label;

[0112] Step S420, obtaining a data quantile threshold according to the test phone number, the test operation service data, the test outgoing call data of multiple calls, the test number category label, the original number recognition model, and the call regression model of each call;

[0113] Step S430, calculating the number recognition confidence interval according to the target number category and the data quantile threshold.

[0114] In step S410 of some embodiments, from the test set Get the test phone number, the test operation service data of the test phone number, the test outbound call data of multiple calls, and the test number category label.

[0115] In step S420 of some embodiments, based on the original number recognition model and the call regression model of each call, credibility learning is performed through test phone numbers, test operation service data, test outgoing call data of multiple calls, and test number category labels to obtain data quantile thresholds. The data quantile threshold is used to measure the value of a specific position in the data distribution, for example, the median is the 50% quantile. The data quantile threshold provides a reliability guarantee for the confidence, so that the confidence calculation is performed based on the data quantile threshold.

[0116] In step S430 of some embodiments, in order to evaluate the prediction accuracy of the target number category, a number recognition confidence interval is calculated based on the target number category and the data percentile threshold. The number recognition confidence interval is a confidence interval of the number discrimination result, which represents the possible range of parameter values ​​of the target number category. The smaller the span of the range, the higher the credibility of the number prediction result.

[0117] Through the above steps S410 to S430, the confidence range of the telephone number identification result can be obtained to evaluate the credibility of the number prediction and realize the reliable use of the prediction result.

[0118] See also Figure 5 In some embodiments, step S420 may include but is not limited to steps S510 to S550:

[0119] Step S510, performing number recognition on the test telephone number and the test operation service data through the original number recognition model to obtain a first test number category;

[0120] Step S520, for each call, performing number recognition on the test phone number and the test outgoing call data through a call regression model to obtain a second test number category;

[0121] Step S530, determining a confidence score for each call based on the test number category label, the first test number category, and the second test number category for each call;

[0122] Step S540, sorting the confidence scores of the multiple calls to obtain a score sequence;

[0123] Step S550, calculating the data quantile threshold according to the score sequence and the preset confidence level; wherein the preset confidence level is used to indicate the proportion of confidence scores in the score sequence that are less than or equal to the data quantile threshold.

[0124] In step S510 of some embodiments, when calculating the data quantile threshold, the test phone number and the test operation service data are input into the original number recognition model for number recognition to obtain a first test number category. The first test number category is the number category for the test phone number predicted by the original number recognition model.

[0125] In step S520 of some embodiments, for each call, the test phone number and the test outgoing call data of the current call are identified by the call regression model of the current call to obtain a second test number category of the current call. The second test number category is the number category for the test phone number predicted by the call regression model based on the test outgoing call data.

[0126] In step S530 of some embodiments, for each call, the confidence score of the corresponding call is calculated based on the test number category label, the first test number category and the second test number category of the corresponding call. The confidence score is used to indicate the prediction credibility of the second test number category of the test phone number in the current call output by the call regression model, so as to perform confidence verification on the call regression model. The confidence score adopts the prediction error metric, and the smaller the confidence score, the more credible the number discrimination result of this call is.

[0127] In step S540 of some embodiments, in order to obtain the data distribution of the confidence scores and quickly locate a specific quantile, the confidence scores of multiple calls are sorted in order from small to large to determine the order of the confidence scores and obtain a score sequence. The score sequence is a set of confidence scores, and the score sequence is expressed as:

[0128]

[0129] Among them, S iis the i-th confidence score in the score sequence; k is the number of calls; N is the number of samples in the data set; N 1 is the number of samples in the training set; NN 1 is the number of samples in the test set; (NN 1 )k is the number of confidence scores in the score sequence.

[0130] In step S550 of some embodiments, the preset confidence level represents the degree of confidence, which is used to indicate the probability that the confidence level score in the score sequence is less than or equal to the data quantile threshold, and can be represented by 1-α. The number of confidence levels in the score sequence is obtained, and the number is multiplied by the preset confidence level to obtain the target position. The confidence level score of the target position is obtained from the score sequence to obtain the data quantile threshold.

[0131] Through the above steps S510 to S550, the data quantile threshold can be obtained, so as to calculate the number recognition confidence interval according to the data quantile threshold, and then evaluate the accuracy of the number recognition result.

[0132] See also Figure 6 In some embodiments, step S530 may include but is not limited to steps S610 to S650:

[0133] Step S610, calculating the difference between the test number category label and the first test number category to obtain a first category difference;

[0134] Step S620, calculating the difference between the first category difference and the second test number category to obtain a first score;

[0135] Step S630, calculating the difference between the second test number category and the test number category label to obtain a second category difference;

[0136] Step S640, performing aggregation calculation on the second category difference and the first test number category to obtain a second score;

[0137] Step S650: Screen the first score and the second score to obtain a confidence score.

[0138] In step S610 of some embodiments, in order to measure the difference between the predicted value output by the original number recognition model for the test phone number and the true value, the test number category label is subtracted from the first test number category to obtain a first category difference. The first category difference is the difference between the test number category label and the first test number category.

[0139] In step S620 of some embodiments, in order to measure the output prediction error of the current call regression model for differential learning, for the current call, the first category difference is subtracted from the second test number category of the current call to obtain a first score for the current call. The larger the first score, the larger the output prediction error, and the smaller the confidence of the discrimination result output by the call regression model.

[0140] In step S630 of some embodiments, in order to measure the difference between the predicted value output by the current call regression model for the test phone number and the true value, the second test number category of the current call is subtracted from the test number category label to obtain a second category difference. The second category difference is the difference between the second test number category and the test number category label.

[0141] In step S640 of some embodiments, in order to measure the comprehensive prediction error of the discrimination results output by the original number recognition model and the call regression model of the current call for the test phone number, the second category difference and the first test number category are added to obtain a second score for the current call. The larger the second score, the larger the comprehensive prediction error, and the smaller the confidence of the discrimination result output by the call regression model.

[0142] In step S650 of some embodiments, the maximum value is selected from the first score of the current call and the second score of the current call to obtain the confidence score of the current call. The larger the confidence score, the less reliable the discrimination result.

[0143] Calculate the confidence score of the ith test phone number in the sth call. The calculation formula of the confidence score is defined as:

[0144] S(o i,s ,b i )=max{y i -q(b i )-g s (o i,s ),g s (o i,s )-y i +q(b i )},

[0145] Where S is the test operation service data based on the test phone number b i And test outgoing call data o i,s The confidence score obtained; y i is the test number category label; q is the original number recognition model; g s is the call regression model for the sth call.

[0146] Through the above steps S610 to S650, a confidence score can be obtained to evaluate the credibility of the number identification result based on the confidence score.

[0147] See also Figure 7 In some embodiments, step S430 may include but is not limited to steps S710 to S730:

[0148] Step S710, performing difference calculation according to the target number category and the data quantile threshold to obtain a confidence lower limit value;

[0149] Step S720, performing aggregation calculation according to the target number category and the data quantile threshold to obtain an upper confidence limit value;

[0150] Step S730, constructing an interval according to the lower confidence limit value and the upper confidence limit value to obtain a number recognition confidence interval.

[0151] In step S710 of some embodiments, in order to obtain the lower limit of the confidence interval, the target number category is subtracted from the data quantile threshold, and the subtraction result is compared with 0. If the subtraction result is greater than or equal to 0, the subtraction result is used as the confidence lower limit value. If the subtraction result is less than 0, 0 is used as the confidence lower limit value.

[0152] In step S720 of some embodiments, in order to obtain the upper limit of the confidence interval, the target number category and the data quantile threshold are added, and the result of the addition is compared with 1. If the result of the addition is greater than or equal to 1, 1 is used as the upper limit of the confidence level. If the result of the addition is less than 1, the result of the addition is used as the upper limit of the confidence level.

[0153] In step S730 of some embodiments, the confidence lower limit value is used as the left endpoint of the interval, and the confidence upper limit value is used as the right endpoint of the interval to construct the interval and obtain the number recognition confidence interval. The probability that the target number category is within the number recognition confidence interval is greater than or equal to the preset confidence 1-α. The calculation formula of the number recognition confidence interval is expressed as:

[0154]

[0155] Where b is the target operation service data of the target phone number; q represents the original number recognition model; g s is the call regression model of the sth call; m is the number of calls to the target phone number; o s The target outgoing call data for the target phone number for the sth time; is the data quantile threshold.

[0156] The data quantile threshold is obtained based on the confidence score, which is based on the prediction error metric. Therefore, the smaller the interval span of the number recognition confidence interval, the higher the credibility of the number identification result. The number recognition confidence interval gradually changes with the increase of the number call count m, so as to perform a dynamic confidence evaluation of number identification with gradually adjusted credibility, so as to better distinguish abnormal phone numbers and realize reliable identification of phone numbers.

[0157] Through the above steps S710 to S730, a confidence interval can be provided for the prediction result, so as to realize reliable identification of the telephone number based on the confidence interval.

[0158] See also Figure 8 The embodiment of the present application also provides a telephone number recognition device, which can implement the above telephone number recognition method, and the telephone number recognition device includes:

[0159] The data acquisition module 810 is used to acquire the target phone number, the target operation service data of the target phone number, and the target outgoing call data of multiple calls;

[0160] The first number recognition module 820 is used to perform number recognition on the target telephone number and the target operation service data through the original number recognition model to obtain a first number category; wherein the original number recognition model is obtained by minimizing the difference between the predicted first number category output based on the sample telephone number and the sample operation service data and the sample number category label;

[0161] The second number identification module 830 is used to identify the target phone number and the target outgoing call data of each call through the call regression model of each call, and obtain the second number category of each call; wherein the call regression model is based on the sample phone number and the sample outgoing call data of the corresponding call. The predicted second number category is output by minimizing the difference between the target difference and the original number identification model based on the sample phone number and the sample operation service data. The difference between the predicted intermediate number category and the sample number category label;

[0162] A first calculation module 840 is used to calculate the average of the second number categories of multiple calls to obtain a reference predicted number category;

[0163] The second calculation module 850 is used to perform aggregate calculation on the first number category and the reference predicted number category to obtain a target number category of the target phone number; wherein the target number category is used to indicate whether the target phone number is a normal number or an abnormal number.

[0164] The specific implementation of the telephone number recognition device is substantially the same as the specific implementation of the telephone number recognition method described above, and will not be described in detail herein.

[0165] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above telephone number recognition method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0166] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0167] The processor 910 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0168] The memory 920 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 920, and the processor 910 calls and executes the telephone number recognition method of the embodiment of this application;

[0169] Input / output interface 930, used to implement information input and output;

[0170] Communication interface 940, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0171] bus 950 , which transmits information between the various components of the device (e.g., processor 910 , memory 920 , input / output interface 930 , and communication interface 940 );

[0172] The processor 910 , the memory 920 , the input / output interface 930 , and the communication interface 940 are connected to each other in communication within the device via a bus 950 .

[0173] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned telephone number recognition method is implemented.

[0174] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0176] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0177] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0179] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0180] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0181] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0184] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0185] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A telephone number recognition method, characterized in that: The method comprises: Obtaining a target phone number, target operating service data of the target phone number, and target outgoing call data of multiple calls; The target telephone number and the target operation service data are identified by an original number identification model to obtain a first number category; wherein the original number identification model is obtained by minimizing the difference between the predicted first number category output based on the sample telephone number and the sample operation service data and the sample number category label; The target telephone number and the target outgoing call data of each call are identified by a call regression model for each call, so as to obtain a second number category for each call; wherein the call regression model is obtained by minimizing the difference between the predicted second number category output based on the sample telephone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample telephone number and the sample operation service data and the sample number category label; Calculate the average of the second number categories of multiple calls to obtain a reference predicted number category; The first number category and the reference predicted number category are aggregated and calculated to obtain a target number category of the target telephone number; wherein the target number category is used to indicate whether the target telephone number is a normal number or an abnormal number.

2. The method according to claim 1, characterized in that After performing aggregate calculation on the first number category and the reference predicted number category to obtain the target number category of the target phone number, the phone number identification method further includes: Obtaining a test telephone number, test operation service data of the test telephone number, test outgoing call data of multiple calls, and a test number category label; Obtaining a data quantile threshold according to the test telephone number, the test operation service data, the test outgoing call data of multiple calls, the test number category label, the original number recognition model, and the call regression model of each call; The number recognition confidence interval is calculated according to the target number category and the data quantile threshold.

3. The method according to claim 2, characterized in that The acquiring of a data quantile threshold according to the test telephone number, the test operation service data, the test outgoing call data of multiple calls, the test number category label, the original number recognition model, and the call regression model of each call includes: Performing number identification on the test telephone number and the test operation service data by using the original number identification model to obtain a first test number category; For each call, performing number recognition on the test telephone number and the test outgoing call data by using the call regression model to obtain a second test number category; determining a confidence score for each call based on the test number category label, the first test number category, and the second test number category for each call; Sorting the confidence scores of the multiple calls to obtain a score sequence; The data quantile threshold is calculated according to the score sequence and a preset confidence level, wherein the preset confidence level is used to indicate the proportion of confidence scores in the score sequence that are less than or equal to the data quantile threshold.

4. The method according to claim 3, wherein determining the confidence score of each call according to the test number category label, the first test number category and the second test number category of each call comprises: Calculating the difference between the test number category label and the first test number category to obtain a first category difference; Calculating the difference between the first category difference and the second test number category to obtain a first score; Calculating the difference between the second test number category and the test number category label to obtain a second category difference; performing aggregation calculation on the second category difference and the first test number category to obtain a second score; The first score and the second score are screened to obtain the confidence score.

5. The method according to claim 2, characterized in that: The calculating the number recognition confidence interval according to the target number category and the data quantile threshold comprises: Calculate the difference between the target number category and the data quantile threshold to obtain a lower confidence limit value; Performing aggregation calculation according to the target number category and the data quantile threshold to obtain an upper confidence limit value; An interval is constructed according to the confidence lower limit value and the confidence upper limit value to obtain the number recognition confidence interval.

6. The method according to any one of claims 1 to 5, characterized in that: The call regression model for each call is trained according to the following steps: Performing number recognition on the sample telephone number and the sample operation service data by using the original number recognition model to obtain the predicted intermediate number category; Calculate the difference between the sample number category label and the predicted intermediate number category to obtain the target difference value; For each call, performing number recognition on the sample telephone number and the sample outgoing call data by using a preset original regression model to obtain the predicted second number category; Perform loss calculation on the target difference and the predicted second number category of each call respectively to obtain first loss data for each call; The model parameters of the preset original regression model are updated respectively according to the first loss data of each call to obtain the call regression model of each call.

7. The method according to any one of claims 1 to 5, characterized in that: The original number recognition model is trained according to the following steps: Performing number recognition on the sample telephone number and the sample operation service data by using a preset number recognition model to obtain the predicted first number category; Calculate the loss according to the predicted first number category and the sample number category label to obtain second loss data; The model parameters of the preset number recognition model are updated according to the second loss data to obtain the original number recognition model.

8. A telephone number recognition device, characterized in that: The device comprises: A data acquisition module, used to acquire a target phone number, target operation service data of the target phone number, and target outgoing call data of multiple calls; A first number recognition module is used to perform number recognition on the target telephone number and the target operation service data through an original number recognition model to obtain a first number category; wherein the original number recognition model is obtained by minimizing the difference between the predicted first number category output based on the sample telephone number and the sample operation service data and the sample number category label; A second number identification module is used to identify the target telephone number and the target outgoing call data of each call through a call regression model for each call, and obtain a second number category for each call; wherein the call regression model is obtained by minimizing the difference between the predicted second number category output based on the sample telephone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample telephone number and the sample operation service data and the sample number category label; A first calculation module is used to calculate the average of the second number categories of multiple calls to obtain a reference predicted number category; The second calculation module is used to perform aggregate calculation on the first number category and the reference predicted number category to obtain a target number category of the target telephone number; wherein the target number category is used to indicate whether the target telephone number is a normal number or an abnormal number.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images