Telephone number identification method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a telephone number identification method and device, electronic equipment and storage medium, belonging to the technical field of artificial intelligence. The method comprises: performing number identification on the target telephone number and target operation service data through an original number identification model to obtain a first number category, performing number identification on the target telephone number and target outgoing call data of each call through a call regression model of each call to obtain a second number category of each call, performing mean value calculation on the second number categories of multiple calls to obtain a reference predicted number category, and performing aggregation calculation on the first number category and the reference predicted number category to obtain a target number category of the target telephone number. The target number category is used to indicate that the target telephone number is a normal number or an abnormal number, and the accuracy of telephone number identification can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a telephone number identification method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the wide application of Internet technology and mobile communication technology, non-contact fraud activities are frequently implemented by means of communication tools such as mobile phones and fixed phones. In order to protect the property and privacy safety of users, it is necessary to identify whether a telephone number is a fraud number. In related technologies, artificial experience rules are used for telephone number identification. However, artificial experience rules are often based on limited subjective experience and cannot cover the characteristics of all telephone numbers, resulting in low accuracy of telephone number identification. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a telephone number identification method and device, an electronic device and a storage medium, which aims to improve the accuracy of telephone number identification.

[0004] To achieve the above purpose, a first aspect of the embodiments of the present application provides a telephone number identification method, which comprises:

[0005] obtaining a target telephone number, target operation service data of the target telephone number and target outgoing call data of multiple calls;

[0006] performing number identification on the target telephone number and the target operation service data by using an original number identification model to obtain a first number category, wherein the original number identification model is obtained through difference minimization learning between a predicted first number category output by the original number identification model based on sample telephone numbers and sample operation service data and a sample number category label;

[0007] performing number identification on the target telephone number and the target outgoing call data of each call by using a call regression model of each call to obtain a second number category of each call, wherein the call regression model is obtained through difference minimization learning between a predicted second number category output by the call regression model based on the sample telephone numbers and sample outgoing call data of corresponding calls and a target difference value, and the target difference value is a difference value between a predicted intermediate number category output by the original number identification model based on the sample telephone numbers and the sample operation service data and the sample number category label;

[0008] performing mean calculation on the second number categories of the multiple calls to obtain a reference predicted number category;

[0009] The first number category and the reference predicted number category are aggregated to obtain the target number category of the target phone number; wherein, the target number category is used to indicate whether the target phone number is a normal number or an abnormal number.

[0010] In some embodiments, after performing an aggregation calculation on the first number category and the reference predicted number category to obtain the target number category of the target phone number, the phone number identification method further includes:

[0011] Obtain the test phone number, the test operation service data of the test phone number, the test outbound call data of multiple calls, and the test number category label;

[0012] Based on the test phone number, the test operation service data, the test outbound call data from multiple calls, the test number category label, the original number recognition model, and the call regression model for each call, obtain the data quantile threshold;

[0013] Calculate the number identification confidence interval based on the target number category and the data quantile threshold.

[0014] In some embodiments, obtaining the data quantile threshold based on the test phone number, the test operation service data, the test outbound call data from multiple calls, the test number category label, the original number recognition model, and the call regression model for each call includes:

[0015] The test phone number and the test operation service data are identified using the original number recognition model to obtain a first test number category.

[0016] For each call, the call regression model is used to identify the test phone number and the test outgoing call data to obtain a second test number category;

[0017] The confidence score for each call is determined based on the test number category label, the first test number category, and the second test number category for each call.

[0018] The confidence scores from multiple calls are sorted to obtain a score sequence;

[0019] The data quantile threshold is calculated based on the score sequence and a preset confidence level; wherein the preset confidence level is used to indicate the proportion of confidence scores in the score sequence that are less than or equal to the data quantile threshold.

[0020] In some embodiments, determining the confidence score for each call based on the test number category label, the first test number category, and the second test number category for each call includes:

[0021] The difference between the test number category label and the first test number category is calculated to obtain the first category difference;

[0022] The difference between the first category difference and the second test number category is calculated to obtain a first score;

[0023] The difference between the second test number category and the test number category label is calculated to obtain the second category difference;

[0024] The second category difference and the first test number category are aggregated and calculated to obtain the second score;

[0025] The confidence score is obtained by filtering the first score and the second score.

[0026] In some embodiments, calculating the number identification confidence interval based on the target number category and the data quantile threshold includes:

[0027] The lower confidence limit is obtained by calculating the difference based on the target number category and the data quantile threshold.

[0028] The confidence limit is obtained by performing aggregation calculations based on the target number category and the data quantile threshold.

[0029] The confidence interval for number identification is obtained by constructing an interval based on the lower confidence limit and the upper confidence limit.

[0030] In some embodiments, the call regression model for each call is trained according to the following steps:

[0031] The original number recognition model is used to identify the sample phone numbers and the sample operational service data to obtain the predicted intermediate number category.

[0032] The target difference is obtained by calculating the difference between the sample number category label and the predicted intermediate number category;

[0033] For each call, the sample phone numbers and the sample outgoing call data are identified using a preset original regression model to obtain the predicted second number category;

[0034] Loss calculations are performed on the target difference and the predicted second number category for each call to obtain the first loss data for each call;

[0035] The model parameters of the preset original regression model are updated based on the first loss data of each call to obtain the call regression model for each call.

[0036] In some embodiments, the original number recognition model is trained according to the following steps:

[0037] The sample phone numbers and sample operational service data are identified using a preset number recognition model to obtain the predicted first number category;

[0038] The second loss data is obtained by calculating the loss based on the predicted first number category and the sample number category label;

[0039] The preset number recognition model is updated with model parameters based on the second loss data to obtain the original number recognition model.

[0040] To achieve the above objectives, a second aspect of this application provides a telephone number identification device, the device comprising:

[0041] The data acquisition module is used to acquire the target phone number, the target operation service data of the target phone number, and the target outgoing call data of multiple calls.

[0042] The first number recognition module is used to recognize the target phone number and the target operation service data through the original number recognition model to obtain the first number category; wherein, the original number recognition model is obtained by minimizing the difference between the predicted first number category output by the sample phone number and the sample operation service data and the sample number category label.

[0043] The second number identification module is used to identify the target phone number and the target outgoing call data of each call using a call regression model for each call, and obtain the second number category for each call; wherein, the call regression model is obtained by minimizing the difference between the predicted second number category output by the sample phone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample phone number and the sample operation service data and the sample number category label;

[0044] The first calculation module is used to calculate the average value of the second number category from multiple calls to obtain the reference predicted number category;

[0045] The second calculation module is used to perform aggregate calculations on the first number category and the reference predicted number category to obtain the target number category of the target phone number; wherein, the target number category is used to indicate whether the target phone number is a normal number or an abnormal number.

[0046] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0047] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0048] The telephone number identification method, telephone number identification device, electronic device, and computer-readable storage medium of this application acquire a target telephone number, target operational service data of the target telephone number, and target outgoing call data from multiple calls to determine whether a target telephone number is a fraudulent number based on the target operational service data and the target outgoing call data. To address the problem of inaccurate telephone number identification caused by manual experience rules, a deep learning model is used for number identification. Different models can capture different telephone number features. To accurately determine whether a telephone number is a fraudulent number, the original number identification model and the call regression model for each call are combined for the number identification task. Operational service data includes information such as number packages and number activation time, which can be used to identify whether a telephone number is abnormal. The original number identification model is used to identify the target telephone number and the target operational service data to obtain a first predicted number category. Since the information provided by the operational service data is limited, sample outgoing call data from multiple calls is used as supplementary information to further improve the accuracy of telephone number identification. The call regression model for each call is used to identify the target telephone number and the target outgoing call data for each call to obtain a second predicted number category for each call. A phone number may have multiple calls, each with different call times and durations. To avoid significant discrepancies in prediction results due to different calls, the average of the second predicted number categories from multiple calls is calculated to obtain a reference predicted number category. The first predicted number category and the reference predicted number category are then aggregated to obtain the target number category for the target phone number. By combining the number categories predicted by various models, the overall prediction error can be reduced, and the accuracy of phone number recognition can be improved. Attached Figure Description

[0049] Figure 1This is a flowchart of the telephone number recognition method provided in the embodiments of this application;

[0050] Figure 2 This is a flowchart of the training process of the original number recognition model provided in the embodiments of this application;

[0051] Figure 3 This is a flowchart illustrating the training process of the call regression model for each call provided in the embodiments of this application;

[0052] Figure 4 This is another flowchart of the telephone number recognition method provided in the embodiments of this application;

[0053] Figure 5 yes Figure 4 The flowchart of step S420 in the middle;

[0054] Figure 6 yes Figure 5 The flowchart of step S530 in the text;

[0055] Figure 7 yes Figure 4 The flowchart of step S430 in the middle;

[0056] Figure 8 This is a schematic diagram of the telephone number recognition device provided in the embodiments of this application;

[0057] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0061] With the widespread application of internet and mobile communication technologies, contactless fraud using mobile phones, landlines, and other communication tools is occurring frequently. To protect users' assets and privacy, it is necessary to identify whether a phone number is fraudulent. One related technology uses manual experience rules for phone number identification. However, manual experience rules are often based on limited subjective experience and cannot cover all the characteristics of phone numbers, resulting in low accuracy in phone number identification.

[0062] Based on this, embodiments of this application provide a telephone number recognition method, a telephone number recognition device, an electronic device, and a computer-readable storage medium, aiming to improve the accuracy of telephone number recognition.

[0063] The telephone number recognition method, telephone number recognition device, electronic device, and computer-readable storage medium provided in this application are specifically described through the following embodiments. First, the telephone number recognition method in the embodiments of this application is described.

[0064] The telephone number recognition method provided in this application relates to the field of artificial intelligence technology. The telephone number recognition method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the telephone number recognition method, but is not limited to the above forms.

[0065] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0066] Figure 1 This is an optional flowchart of the telephone number recognition method provided in the embodiments of this application, which may include, but is not limited to, steps S110 to S150.

[0067] Step S110: Obtain the target phone number, the target operation service data of the target phone number, and the target outbound call data from multiple calls;

[0068] Step S120: The target phone number and target operation service data are identified by the original number recognition model to obtain the first number category; wherein, the original number recognition model is obtained by minimizing the difference between the predicted first number category output by the sample phone number and sample operation service data and the sample number category label.

[0069] Step S130: The target phone number and the target outgoing call data of each call are identified by the call regression model for each call to obtain the second number category of each call; wherein, the call regression model is obtained by minimizing the difference between the predicted second number category output by the sample phone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample phone number and the sample operation service data and the sample number category label;

[0070] Step S140: Calculate the average of the second number categories from multiple calls to obtain the reference predicted number category;

[0071] Step S150: Aggregate the first number category and the reference predicted number category to obtain the target number category of the target phone number; wherein, the target number category is used to indicate whether the target phone number is a normal number or an abnormal number.

[0072] In step S110 of some embodiments, the target phone number, the target operation service data of the target phone number, and the target outgoing call data from multiple calls are acquired. The target phone number is the phone number to be identified. The target operation service data is the commercial service data provided by the operator for the target phone number, including the package information of the target phone number (such as package type, cost, package duration, etc.), the number activation time, etc. The target outgoing call data from multiple calls is the outgoing call data of the target phone number from the most recent m calls. The outgoing call data includes the call time, call duration, and the number of historical calls with the called phone number, where m is an integer greater than or equal to 1. This application embodiment does not limit the number of calls; the number of calls can be different for different phone numbers. The outgoing call data is call-granular data and usually changes with the changes in the call.

[0073] In step S120 of some embodiments, the target operational service data is associated with the target phone number. This data is user-level and does not change with calls, so it can be directly used to determine whether the target phone number is a fraudulent number. The target phone number and target operational service data are input into the original number recognition model for number recognition, resulting in a first number category. The first number category indicates whether the target phone number is a normal number or an abnormal number; an abnormal number is a fraudulent number. Traditional number recognition methods rely on manual experience and rules, which requires a high level of human expertise and incurs high labor and time costs. Compared to manual recognition methods, the original number recognition model significantly improves the efficiency and accuracy of fraudulent number identification.

[0074] Please see Figure 2 In some embodiments, the training process of the original number recognition model may include, but is not limited to, steps S210 to S230:

[0075] Step S210: Use a preset number recognition model to identify the sample phone numbers and sample operational service data to obtain a predicted first number category;

[0076] Step S220: Calculate the loss based on the predicted first number category and the sample number category label to obtain the second loss data;

[0077] Step S230: Update the model parameters of the preset number recognition model based on the second loss data to obtain the original number recognition model.

[0078] In step S210 of some embodiments, a dataset is obtained, which includes telephone numbers, feature data of telephone numbers, and number category labels. The feature data is divided into operational service data and outbound call data. A sub-dataset is constructed based on all telephone numbers in the dataset, the operational service data corresponding to the telephone numbers, and the number category labels. The original number recognition model is trained on the sub-dataset.

[0079] The dataset extracts sample phone numbers, sample operational service data for these phone numbers, and sample number category labels. The sample phone numbers are used for model training. The sample operational service data consists of commercial service data provided by operators to these phone numbers, including package information and activation time. The sample number category labels represent the true category of the sample phone numbers, which can be either normal or abnormal. The preset number recognition model is a binary classification model, which can be constructed using a tree model or a deep learning model. The sample phone numbers and sample operational service data are input into the preset number recognition model for number recognition, resulting in a predicted first number category. The predicted first number category is the number category predicted by the preset number recognition model for the sample phone numbers.

[0080] In step S220 of some embodiments, a loss is calculated on the predicted first number category and the sample number category label based on the cross-entropy loss function. The loss value obtained from the loss calculation is used as second loss data to guide the model training process of the preset number recognition model, so that the predicted first number category is close to the sample number category label. The cross-entropy loss function is used to measure the difference between the predicted value and the actual label.

[0081] In step S230 of some embodiments, the second loss data is minimized, and the model parameters of the preset number recognition model are adjusted to obtain the original number recognition model. The model training and optimization process of the original number recognition model is expressed as follows:

[0082] min q ∑ o∈[N] L(q(b i ),y i ),

[0083] Where q represents the preset number recognition model; N is the number of sample phone numbers; i represents the i-th sample phone number; L is the cross-entropy loss function; b i For the sample operation service data of the i-th sample phone number; y i Let be the sample number category label for the i-th sample phone number.

[0084] Through the above steps S210 to S230, the original number recognition model can be trained to perform preliminary number recognition based on the original number recognition model.

[0085] In step S130 of some embodiments, in related technologies, telephone statistical features of a single call are used to predict fraud in a single call, and the incoming call is directly determined to be a fraudulent number based on this prediction. However, a phone number may have multiple calls, and different calls have different characteristics such as call time and call duration, which can lead to significant differences in the prediction results for the same phone number. Furthermore, the original number identification model only uses operational service data for number prediction, and the information provided by operational service data is limited, which can result in inaccurate number identification. To improve the accuracy of number identification, this application embodiment fully utilizes the data characteristics of operational service data and outgoing call data, using target outgoing call data from multiple calls as supplementary information to compensate for the shortcomings of untimely updates and limited available information in operational service data. For each call, the target phone number and the target outgoing call data of the current call are identified using the call regression model of the current call, and a second number category of the current call is output.

[0086] Please see Figure 3In some embodiments, the training process of the call regression model for each call may include, but is not limited to, steps S310 to S350:

[0087] Step S310: Use the original number recognition model to identify the sample phone numbers and sample operation service data to obtain the predicted intermediate number category;

[0088] Step S320: Calculate the difference based on the sample number category label and the predicted intermediate number category to obtain the target difference;

[0089] Step S330: For each call, the sample phone number and sample outgoing call data are identified using a preset original regression model to obtain a predicted second number category;

[0090] Step S340: Calculate the loss for the target difference and the predicted second number category for each call to obtain the first loss data for each call;

[0091] Step S350: Update the model parameters of the preset original regression model according to the first loss data of each call to obtain the call regression model for each call.

[0092] In step S310 of some embodiments, the dataset includes N data samples, each of which includes a telephone number, feature data of the telephone number, and a number category label. The feature data includes operational service data and outbound call data. The dataset is divided into a training set and a test set by randomly sampling the telephone numbers. The training set is applied to the model training process, and the test set is applied to the model testing process.

[0093] If the dataset is represented as D1={({O i,j} k j＝1 ,b i ,y i )} i∈[N] Where k is the number of calls, and k and the number of calls m to the target phone number can be the same or different, k is greater than or equal to m, j represents the j-th call, o i,j For the j-th outgoing call data of the i-th phone number, b i For the operational service data of the i-th phone number, y i Let be the number category label for the i-th phone number, and N be the number of samples in the dataset. Then the training set is represented as: N1 is the number of samples in the training set, and the test set is represented as... The number of samples in the test set is N-N1.

[0094] The call regression model is trained on the training set. Sample phone numbers, sample operational service data of sample phone numbers, sample outbound call data of multiple calls, and sample number category labels are obtained from the training set. The original number recognition model is used to identify the sample phone numbers and sample operational service data to determine whether the sample phone numbers are fraudulent numbers and to obtain the predicted intermediate number category.

[0095] In step S320 of some embodiments, in order to measure the difference between the predicted value and the true value of the original number recognition model, the sample number category label and the predicted intermediate number category are subtracted to obtain the target difference.

[0096] In step S330 of some embodiments, a phone number typically has multiple calls, and each call can serve as a basis for determining whether the phone number is a fraudulent number. Therefore, a call regression model based on multiple calls is designed. K preset primitive regression models are designed as weak learners, and these preset primitive regression models are binary classification models. This application embodiment does not limit the preset primitive regression models; they can be tree models or deep models. For each call, the preset primitive regression model of the current call is used to perform number identification on the sample phone number and the sample outgoing call data of the current call to obtain the predicted second number category of the current call.

[0097] In step S340 of some embodiments, for each call, loss calculation is performed on the target difference and the predicted second number category of the current call based on the cross-entropy loss function to obtain the first loss data of the current call. This is used to enable the preset original regression model to learn the prediction error of the original number recognition model through differential learning, so that the preset original regression model can focus on learning the data features that the original number recognition model cannot capture, thereby improving the overall prediction accuracy of the model.

[0098] In step S350 of some embodiments, for each call, the first loss data of the current call is minimized, and the model parameters of the preset original regression model of the current call are updated to obtain the call regression model of the current call.

[0099] For the s-th call regression model, the data used is The training process is represented as follows:

[0100]

[0101] Among them, g s The original regression model is set up; L is the cross-entropy loss function; o i,s b is the sample outgoing call data of the s-th call to the i-th sample phone number; i For the sample operation service data of the i-th sample phone number; y iLet N1 be the sample number category label for the i-th sample phone number; N1 is the number of samples in the training set.

[0102] Through steps S310 to S350, a call regression model for each call can be obtained, thereby enabling accurate number prediction for each call and compensating for the prediction error of the original number recognition model.

[0103] In step S140 of some embodiments, a phone number has multiple calls, and the call times, durations, and other characteristics of these calls differ, resulting in different number prediction results for the same phone number across multiple calls. To avoid significant differences in number prediction results across different calls, an average is calculated for the second number category of the multiple calls to obtain a reference predicted number category. The process of calculating the average is defined as follows:

[0104]

[0105] Where m is the number of calls to the target phone number, 1≤m≤k; g s For the call regression model of the s-th call; O s The target outbound call data for the target phone number at the s-th time.

[0106] In step S150 of some embodiments, the first number category and the reference predicted number category are summed to obtain the target number category of the target phone number. The target number category indicates whether the target phone number is a normal number or an abnormal number. The formula for calculating the target number category is expressed as:

[0107]

[0108] Where b represents the target operational service data for the target phone number; q represents the original number identification model; g s Let m be the call regression model for the s-th call; m is the number of calls to the target phone number; o s The target outbound call data for the target phone number at the s-th time.

[0109] In related technologies, AI-based prediction methods only output the predicted probability of number categories. However, the predicted probability cannot accurately reflect the reliability of the prediction results, making it difficult to accurately evaluate the results. Imposing direct blocking or other disabling measures on abnormal phone numbers with incorrect predictions would result in a very poor user experience. This application's embodiments employ a common-type prediction framework, calculating the confidence level of the prediction based on the original number identification model and the call regression model for each call. The calculation process of the confidence level is described in detail below.

[0110] Please see Figure 4In some embodiments, after step S150, the telephone number identification method may also include, but is not limited to, steps S410 to S430:

[0111] Step S410: Obtain the test phone number, test operation service data of the test phone number, test outbound call data of multiple calls, and test number category label;

[0112] Step S420: Based on the test phone number, test operation service data, test outbound call data from multiple calls, test number category labels, original number recognition model, and call regression model for each call, obtain the data quantile threshold.

[0113] Step S430: Calculate the number identification confidence interval based on the target number category and data quantile threshold.

[0114] In step S410 of some embodiments, from the test set Obtain test phone numbers, test operation service data for test phone numbers, test outbound call data from multiple calls, and test number category tags.

[0115] In step S420 of some embodiments, based on the original number recognition model and the call regression model for each call, confidence learning is performed using test phone numbers, test operational service data, test outbound call data from multiple calls, and test number category labels to obtain a data quantile threshold. The data quantile threshold measures the value at a specific position in the data distribution; for example, the median is the 50th quantile. The data quantile threshold provides a reliability guarantee for confidence levels, enabling confidence calculations based on the data quantile threshold.

[0116] In step S430 of some embodiments, in order to evaluate the prediction accuracy of the target number category, a number identification confidence interval is calculated based on the target number category and the data quantile threshold. The number identification confidence interval is the confidence interval of the number discrimination result, which represents the range in which the parameter value of the target number category may exist. The smaller the span of this range, the higher the credibility of the number prediction result.

[0117] Through the above steps S410 to S430, the confidence range of the telephone number identification result can be obtained to assess the credibility of this number prediction and realize the reliable use of the prediction result.

[0118] Please see Figure 5 In some embodiments, step S420 may include, but is not limited to, steps S510 to S550:

[0119] Step S510: The test phone numbers and test operation service data are identified using the original number recognition model to obtain the first test number category;

[0120] Step S520: For each call, the test phone number and test outgoing call data are identified using a call regression model to obtain the second test number category;

[0121] Step S530: Determine the confidence score for each call based on the test number category label, the first test number category, and the second test number category for each call;

[0122] Step S540: Sort the confidence scores of multiple calls to obtain a score sequence;

[0123] Step S550: Calculate the data quantile threshold based on the score sequence and the preset confidence level; wherein the preset confidence level is used to indicate the proportion of confidence scores in the score sequence that are less than or equal to the data quantile threshold.

[0124] In step S510 of some embodiments, when calculating the data quantile threshold, the test phone number and test operation service data are input into the original number recognition model for number recognition to obtain a first test number category. The first test number category is the number category predicted by the original number recognition model for the test phone number.

[0125] In step S520 of some embodiments, for each call, the test phone number and the test outgoing call data of the current call are identified using the call regression model of the current call to obtain a second test number category for the current call. The second test number category is the number category predicted by the call regression model for the test phone number based on the test outgoing call data.

[0126] In step S530 of some embodiments, for each call, a confidence score is calculated based on the test number category label, the first test number category, and the second test number category of the corresponding call. The confidence score indicates the reliability of the call regression model's prediction for the test phone number in the second test number category of the current call, thus validating the call regression model's confidence. The confidence score uses a prediction error metric; a smaller confidence score indicates a more reliable number identification result for this call.

[0127] In step S540 of some embodiments, in order to obtain the data distribution of confidence scores and quickly locate specific quantiles, the confidence scores of multiple calls are sorted in ascending order to determine the order of the confidence scores, thus obtaining a score sequence. The score sequence is a set of confidence scores, and the score sequence is represented as follows:

[0128]

[0129] Among them, S iLet be the i-th confidence score in the score sequence; k be the number of calls; N be the number of samples in the dataset; N1 be the number of samples in the training set; N-N1 be the number of samples in the test set; and (N-N1)k be the number of confidence scores in the score sequence.

[0130] In step S550 of some embodiments, a preset confidence level represents the degree of confidence, used to indicate the probability that a confidence score in the score sequence is less than or equal to a data quantile threshold, and can be represented by 1-α. The number of confidence scores in the score sequence is obtained, and this number is multiplied by the preset confidence level to obtain the target position. The confidence score of the target position is obtained from the score sequence to obtain the data quantile threshold.

[0131] Through the above steps S510 to S550, the data quantile threshold can be obtained, and the number identification confidence interval can be calculated based on the data quantile threshold, thereby evaluating the accuracy of the number identification result.

[0132] Please see Figure 6 In some embodiments, step S530 may include, but is not limited to, steps S610 to S650:

[0133] Step S610: Calculate the difference between the test number category label and the first test number category to obtain the first category difference;

[0134] Step S620: Calculate the difference between the first category difference and the second test number category to obtain the first score;

[0135] Step S630: Calculate the difference between the second test number category and the test number category label to obtain the second category difference;

[0136] Step S640: Aggregate the differences in the second category and the first test number category to obtain the second score;

[0137] Step S650: Filter the first score and the second score to obtain the confidence score.

[0138] In step S610 of some embodiments, in order to measure the difference between the predicted value and the true value output by the original number recognition model for the test phone number, the test number category label is subtracted from the first test number category to obtain the first category difference. The first category difference is the difference between the test number category label and the first test number category.

[0139] In step S620 of some embodiments, in order to measure the output prediction error of the current call regression model performing differential learning, for the current call, the first category difference is subtracted from the second test number category of the current call to obtain a first score for the current call. The larger the first score, the larger the output prediction error, and the lower the confidence of the discrimination result output by the call regression model.

[0140] In step S630 of some embodiments, in order to measure the difference between the predicted value and the actual value output by the current call regression model for the test phone number, the second test number category of the current call is subtracted from the test number category label to obtain the second category difference. The second category difference is the difference between the second test number category and the test number category label.

[0141] In step S640 of some embodiments, in order to measure the combined prediction error of the discrimination results output by the original number recognition model and the call regression model for the current call for the test phone number, the second category difference and the first test number category are added together to obtain the second score of the current call. The larger the second score, the larger the combined prediction error, and the lower the confidence level of the discrimination results output by the call regression model.

[0142] In step S650 of some embodiments, the maximum value between the first score and the second score of the current call is selected to obtain the confidence score of the current call. The higher the confidence score, the less reliable the judgment result.

[0143] Calculate the confidence score of the i-th test phone number on the s-th call. The formula for calculating the confidence score is defined as follows:

[0144] S(o i,s ,b i )=max{y i -q(b i )-g s (o i,s ),g s (o i,s )-y i +q(b i )},

[0145] Where S represents test operation service data based on test phone numbers. i And test outgoing call data o i,s The obtained confidence score; y i q represents the test number category label; q represents the original number recognition model; g represents the test number category label. s Let be the call regression model for the s-th call.

[0146] Through the above steps S610 to S650, a confidence score can be obtained, and the reliability of the number identification result can be evaluated based on the confidence score.

[0147] Please see Figure 7 In some embodiments, step S430 may include, but is not limited to, steps S710 to S730:

[0148] Step S710: Calculate the difference based on the target number category and data quantile threshold to obtain the lower confidence limit.

[0149] Step S720: Perform aggregation calculation based on the target number category and data quantile threshold to obtain the upper limit of confidence.

[0150] Step S730: Construct an interval based on the lower confidence limit and the upper confidence limit to obtain the number identification confidence interval.

[0151] In step S710 of some embodiments, in order to obtain the lower bound of the confidence interval, the target number category is subtracted from the data quantile threshold, and the result of the subtraction is compared with 0. If the result of the subtraction is greater than or equal to 0, the result of the subtraction is used as the lower bound of the confidence level. If the result of the subtraction is less than 0, 0 is used as the lower bound of the confidence level.

[0152] In step S720 of some embodiments, in order to obtain the upper bound of the confidence interval, the target number category and the data quantile threshold are added together, and the sum of the sum and 1 is compared. If the sum is greater than or equal to 1, then 1 is taken as the upper limit of confidence. If the sum is less than 1, then the sum is taken as the upper limit of confidence.

[0153] In step S730 of some embodiments, the lower confidence limit is used as the left endpoint of the interval, and the upper confidence limit is used as the right endpoint of the interval to construct the interval, thus obtaining the number identification confidence interval. The probability that the target number category lies within the number identification confidence interval is greater than or equal to a preset confidence level of 1-α. The formula for calculating the number identification confidence interval is expressed as:

[0154]

[0155] Where b represents the target operational service data for the target phone number; q represents the original number identification model; g s Let m be the call regression model for the s-th call; m is the number of calls to the target phone number; o s The target outgoing call data for the target phone number at the s-th time; This is the data quantile threshold.

[0156] The data quantile threshold is derived from the confidence score, which in turn is based on a prediction error metric. Therefore, the smaller the span of the number identification confidence interval, the higher the reliability of the number identification result. The number identification confidence interval gradually changes as the number of calls (m) increases, enabling dynamic confidence assessment of number identification with gradually adjusted reliability. This allows for better differentiation of abnormal phone numbers and achieves reliable phone number identification.

[0157] Through the above steps S710 to S730, a confidence interval can be provided for the prediction result, so as to achieve reliable identification of telephone numbers based on the confidence interval.

[0158] Please see Figure 8 This application also provides a telephone number identification device that can implement the above-described telephone number identification method. The telephone number identification device includes:

[0159] The data acquisition module 810 is used to acquire the target phone number, the target operation service data of the target phone number, and the target outgoing call data of multiple calls.

[0160] The first number recognition module 820 is used to recognize the target phone number and target operation service data through the original number recognition model to obtain the first number category; wherein, the original number recognition model is obtained by minimizing the difference between the predicted first number category output by the sample phone number and sample operation service data and the sample number category label.

[0161] The second number recognition module 830 is used to recognize the target phone number and the target outgoing call data of each call through the call regression model of each call to obtain the second number category of each call; wherein, the call regression model is obtained by minimizing the difference between the predicted second number category output by the sample phone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number recognition model based on the sample phone number and the sample number category label;

[0162] The first calculation module 840 is used to calculate the average of the second number category from multiple calls to obtain the reference predicted number category;

[0163] The second calculation module 850 is used to perform aggregate calculations on the first number category and the reference predicted number category to obtain the target number category of the target phone number; wherein, the target number category is used to indicate whether the target phone number is a normal number or an abnormal number.

[0164] The specific implementation of this telephone number recognition device is basically the same as the specific embodiment of the telephone number recognition method described above, and will not be repeated here.

[0165] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned telephone number recognition method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0166] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0167] The processor 910 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to achieve the technical solutions provided in the embodiments of this application.

[0168] The memory 920 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 920 and is called and executed by the processor 910 to execute the telephone number recognition method of the embodiments of this application.

[0169] The input / output interface 930 is used to implement information input and output;

[0170] The communication interface 940 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0171] Bus 950 transmits information between various components of the device (e.g., processor 910, memory 920, input / output interface 930, and communication interface 940);

[0172] The processor 910, memory 920, input / output interface 930 and communication interface 940 are connected to each other within the device via bus 950.

[0173] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described telephone number recognition method.

[0174] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0175] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0176] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0179] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0180] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0182] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0185] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A telephone number recognition method, characterized in that, The method includes: Acquire the target phone number, the target operation service data of the target phone number, and the target outbound call data from multiple calls; The target phone number and the target operational service data are identified using an original number recognition model to obtain a first number category; wherein, the original number recognition model is obtained by minimizing the difference between the predicted first number category output by the sample phone number and the sample operational service data and the sample number category label. The target phone number and the target outgoing call data of each call are identified by a call regression model for each call to obtain a second number category for each call; wherein, the call regression model is obtained by minimizing the difference between the predicted second number category output by the sample phone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample phone number and the sample operation service data and the sample number category label; The average value of the second number category from multiple calls is calculated to obtain the reference predicted number category; The first number category and the reference predicted number category are aggregated to obtain the target number category of the target phone number; wherein, the target number category is used to indicate whether the target phone number is a normal number or an abnormal number; The call regression model for each call is trained according to the following steps: The original number recognition model is used to identify the sample phone numbers and sample operational service data to obtain the predicted intermediate number category; the difference between the sample number category label and the predicted intermediate number category is calculated to obtain the target difference; for each call, a preset original regression model is used to identify the sample phone numbers and sample outbound call data to obtain the predicted second number category; the loss is calculated for the target difference and the predicted second number category for each call to obtain the first loss data for each call; the model parameters of the preset original regression model are updated based on the first loss data for each call to obtain the call regression model for each call; The original number recognition model is trained according to the following steps: The sample phone numbers and sample operational service data are identified using a preset number recognition model to obtain the predicted first number category; loss is calculated based on the predicted first number category and the sample number category label to obtain second loss data; the preset number recognition model is updated with the second loss data to obtain the original number recognition model.

2. The method according to claim 1, characterized in that, After performing the aggregation calculation on the first number category and the reference predicted number category to obtain the target number category of the target phone number, the phone number identification method further includes: Obtain the test phone number, the test operation service data of the test phone number, the test outbound call data of multiple calls, and the test number category label; Based on the test phone number, the test operation service data, the test outbound call data from multiple calls, the test number category label, the original number recognition model, and the call regression model for each call, obtain the data quantile threshold; Calculate the number identification confidence interval based on the target number category and the data quantile threshold.

3. The method according to claim 2, characterized in that, The step of obtaining data quantile thresholds based on the test phone number, the test operation service data, the test outbound call data from multiple calls, the test number category label, the original number recognition model, and the call regression model for each call includes: The test phone number and the test operation service data are identified using the original number recognition model to obtain a first test number category. For each call, the call regression model is used to identify the test phone number and the test outgoing call data to obtain a second test number category; The confidence score for each call is determined based on the test number category label, the first test number category, and the second test number category for each call. The confidence scores from multiple calls are sorted to obtain a score sequence; The data quantile threshold is calculated based on the score sequence and a preset confidence level; wherein the preset confidence level is used to indicate the proportion of confidence scores in the score sequence that are less than or equal to the data quantile threshold.

4. The method according to claim 3, wherein determining the confidence score for each call based on the test number category label, the first test number category, and the second test number category for each call comprises: The difference between the test number category label and the first test number category is calculated to obtain the first category difference; The difference between the first category difference and the second test number category is calculated to obtain a first score; The difference between the second test number category and the test number category label is calculated to obtain the second category difference; The second category difference and the first test number category are aggregated and calculated to obtain the second score; The confidence score is obtained by filtering the first score and the second score.

5. The method according to claim 2, characterized in that, The step of calculating the number identification confidence interval based on the target number category and the data quantile threshold includes: The lower confidence limit is obtained by calculating the difference based on the target number category and the data quantile threshold. The confidence limit is obtained by performing aggregation calculations based on the target number category and the data quantile threshold. The confidence interval for number identification is obtained by constructing an interval based on the lower confidence limit and the upper confidence limit.

6. A telephone number identification device, characterized in that, The device includes: The data acquisition module is used to acquire the target phone number, the target operation service data of the target phone number, and the target outgoing call data of multiple calls. The first number recognition module is used to recognize the target phone number and the target operation service data through the original number recognition model to obtain the first number category; wherein, the original number recognition model is obtained by minimizing the difference between the predicted first number category output by the sample phone number and the sample operation service data and the sample number category label. The second number identification module is used to identify the target phone number and the target outgoing call data of each call using a call regression model for each call, and obtain the second number category for each call; wherein, the call regression model is obtained by minimizing the difference between the predicted second number category output by the sample phone number and the sample outgoing call data of the corresponding call and the target difference, and the target difference is the difference between the predicted intermediate number category output by the original number identification model based on the sample phone number and the sample operation service data and the sample number category label; The first calculation module is used to calculate the average of the second number category from multiple calls to obtain the reference predicted number category; The second calculation module is used to perform aggregate calculations on the first number category and the reference predicted number category to obtain the target number category of the target phone number; wherein, the target number category is used to indicate whether the target phone number is a normal number or an abnormal number; The call regression model for each call is trained according to the following steps: The original number recognition model is used to identify the sample phone numbers and sample operational service data to obtain the predicted intermediate number category; the difference between the sample number category label and the predicted intermediate number category is calculated to obtain the target difference; for each call, a preset original regression model is used to identify the sample phone numbers and sample outbound call data to obtain the predicted second number category; the loss is calculated for the target difference and the predicted second number category for each call to obtain the first loss data for each call; the model parameters of the preset original regression model are updated based on the first loss data for each call to obtain the call regression model for each call; The original number recognition model is trained according to the following steps: The sample phone numbers and sample operational service data are identified using a preset number recognition model to obtain the predicted first number category; loss is calculated based on the predicted first number category and the sample number category label to obtain second loss data; the preset number recognition model is updated with the second loss data to obtain the original number recognition model.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 5.

Images