Model training method and apparatus, and electronic device

Abstract

The application discloses a model training method and device and electronic equipment. The method comprises the following steps: obtaining first historical call data corresponding to an operator network element operation domain and second historical call data corresponding to an operator computing domain; obtaining signaling features and call behavior features conforming to ultra-short call behaviors from the first historical call data and the second historical call data; and training a machine learning model according to the signaling features and the call behavior features to obtain a target machine learning model, wherein the machine learning model is used for identifying target call data corresponding to a target number and determining whether the target number is a suspected fraudulent number. The application solves the technical problems of low recognition accuracy and poor security caused by recognizing fraudulent calls based on the incoming call label reminder provided by the mobile phone manager in the related art.

Description

Technical Field

[0001] This application relates to the field of data recognition, and more specifically, to a model training method, apparatus, and electronic device. Background Technology

[0002] Combating telecommunications harassment and fraud is a systematic project characterized by its long-term nature, recurring nature, and complexity. Fraudsters exploit technological vulnerabilities, management gaps, and weak user awareness to commit fraud, posing a severe challenge to the governance efforts.

[0003] Currently, mobile security apps provided by internet security companies can offer caller ID alerts, but these methods have low accuracy and cannot block incoming calls, prevent answering calls, or prevent potential fraudulent activities. Furthermore, the deployment of such software itself poses data security risks.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a model training method, apparatus, and electronic device to at least solve the technical problems of low identification accuracy and poor security caused by the use of caller ID tags provided by mobile phone management software for fraud call identification in related technologies.

[0006] According to one aspect of the embodiments of this application, a model training method is provided, comprising: acquiring first historical call detail record (CDR) data corresponding to the operator's network element operation domain and second historical CDR data corresponding to the operator's computing domain; acquiring signaling features and call behavior features that conform to ultra-short call behavior from the first historical CDR data and the second historical CDR data; training a machine learning model based on the signaling features and call behavior features to obtain a target machine learning model, wherein the machine learning model is used to identify target CDRs corresponding to a target number and determine whether the target number is a suspected fraudulent number.

[0007] Optionally, after obtaining the target machine learning model, the method further includes: obtaining the target number to be analyzed, analyzing the target call detail records corresponding to the target number based on the machine learning model, and determining whether the target number is a suspected fraudulent number.

[0008] Optionally, before analyzing the target call detail records (CDRs) corresponding to the target number based on the machine learning model, the process includes: determining the call attributes of the target CDRs, wherein the call attributes include: local attributes or outbound call attributes, wherein the number of local calls corresponding to the local attribute is greater than the number of current calls corresponding to the outbound call attribute; determining whether to update the target machine learning model based on the call attributes, wherein, if it is determined that the target machine learning model should be updated, adjusting the first weight of the first signaling feature and the first call behavior feature corresponding to the first historical CDR data, and the second weight of the second signaling feature and the second call behavior feature corresponding to the second historical CDR data during the training process of the target machine learning model; training the target machine learning model using the adjusted first weight of the first signaling feature and the first call behavior feature, and the adjusted second weight of the second signaling feature and the second call behavior feature, to obtain a new target training model.

[0009] Optionally, determining whether to update the target machine learning model based on call attributes includes: obtaining the third weights of the first signaling feature and the first call behavior feature during the training process of the target machine learning model in historical time periods, and the fourth weights of the second signaling feature and the second call behavior feature corresponding to the second historical call detail record data; if the call attribute is a local attribute, and the third weight is less than the preset weight, or the fourth weight is greater than the preset weight, then the target learning model is updated.

[0010] Optionally, determining whether to update the target machine learning model based on the call attributes includes: obtaining the third weights of the first signaling feature and the first call behavior feature during the training process of the target machine learning model in historical time periods, and the fourth weights of the second signaling feature and the second call behavior feature corresponding to the second historical call detail record data; if the call attribute is an outbound call attribute, and the third weight is greater than the preset weight, or the fourth weight is less than the preset weight, then the target learning model is updated.

[0011] Optionally, after determining whether the target number is a suspected fraudulent number, the method further includes: if the target number is determined to be a suspected fraudulent number, obtaining the detection indicators corresponding to the suspected fraudulent number, wherein the detection indicators include: short-ring percentage, call failure rate, and dialing destination number dispersion; if at least two of the short-ring percentage, call failure rate, and dialing destination number dispersion are greater than their corresponding preset values, the suspected fraudulent number is determined to be a fraudulent number, and the fraudulent number is sent to the operator for the operator to shut down the fraudulent number.

[0012] Optionally, the machine learning model is trained based on signaling features and call behavior features to obtain the target machine learning model, including: determining the user features corresponding to positive and negative samples based on signaling features and call behavior features, wherein positive samples include numbers that conform to the fraud type, and negative samples include numbers that are not marked as fraud; extracting features from the user features, wherein the extracted features include: number features, basic features, and extended features; using the random forest algorithm to obtain the Gini index corresponding to each of the number features, basic features, and extended features, wherein the smaller the Gini index, the lower the impurity; ranking each feature based on the Gini index, and selecting the top N features as target features; and training the machine learning model based on the target features to obtain the target machine learning model.

[0013] Optionally, after determining whether the target number is a suspected fraudulent number, the method further includes: if the target number is determined to be a suspected fraudulent number, performing cluster analysis on the types of suspected fraudulent numbers, determining the fraud type to which each suspected fraudulent number belongs, generating prompt information for different fraud types, and sending the prompt information to the terminal on the target side.

[0014] According to another aspect of the embodiments of this application, a method for identifying suspected fraudulent numbers is also provided, including: obtaining a target number to be analyzed; analyzing the target call detail records (CDRs) corresponding to the target number based on a machine learning model to determine whether the target number is a suspected fraudulent number, wherein the machine learning model is trained based on signaling features and call behavior features, and the signaling features and call behavior features are obtained by filtering first historical CDR data corresponding to the operator network element operation domain and second historical CDR data corresponding to the operator computing domain.

[0015] According to another aspect of the embodiments of this application, a model training apparatus is also provided, comprising: a first acquisition module, configured to acquire first historical call detail record (CDR) data corresponding to the operator's network element operation domain and second historical CDR data corresponding to the operator's computing domain; a second acquisition module, configured to acquire signaling features and call behavior features that conform to ultra-short call behavior from the first historical CDR data and the second historical CDR data; and a determination module, configured to train a machine learning model based on the signaling features and call behavior features to obtain a target machine learning model, wherein the machine learning model is used to identify the target CDR corresponding to the target number and determine whether the target number is a suspected fraudulent number.

[0016] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute any model training method.

[0017] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement any model training method.

[0018] In this embodiment, a machine learning model is used to identify suspected fraudulent calls. This involves acquiring first historical call detail records (CDRs) corresponding to the operator's network element operation domain and second historical CDRs corresponding to the operator's computing domain. Signaling features and call behavior features consistent with ultra-short call behavior are extracted from these two data sets. The machine learning model is then trained based on these features to obtain a target machine learning model. This model is used to identify the target CDRs corresponding to the target number, determining whether the target number is a suspected fraudulent number. This achieves the goal of identifying suspected fraudulent calls based on a learning model, thereby improving the identification rate of suspected fraudulent calls, enhancing terminal security, and preventing data and privacy leaks. This solves the problems of low accuracy and poor security in related technologies that rely on caller ID tags provided by mobile security apps for fraudulent call identification. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a flowchart illustrating an optional model training method according to this application;

[0021] Figure 2 This is a schematic diagram used to represent the proportion of short-duration ringing in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram used to represent the call failure rate in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram used in this application embodiment to represent the dispersion of dialed remote number;

[0024] Figure 5 This is a schematic diagram of the overall modeling process in an embodiment of this application;

[0025] Figure 6 This is a flowchart illustrating the model training process in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the data flow process during the modeling and detection process in this embodiment of the application;

[0027] Figure 8 This is a schematic diagram of a process for identifying suspected fraudulent phone numbers according to an embodiment of this application;

[0028] Figure 9 This is a flowchart illustrating an optional method for identifying suspected fraudulent phone numbers according to an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of an optional model training device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] According to an embodiment of this application, an embodiment of a model training method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] Figure 1 This is a model training method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0034] Step S102: Obtain the first historical call detail record (CDR) data corresponding to the operator network element operation domain and the second historical CDR data corresponding to the operator computing domain.

[0035] It should be noted that the above-mentioned operator network element operation domain is the O domain (operation support system), and the above-mentioned operator computing domain is the B domain (business support system).

[0036] Step S104: Obtain signaling features and call behavior features that match the ultra-short call behavior from the first historical call detail record data and the second historical call detail record data;

[0037] Step S106: Train the machine learning model based on signaling features and call behavior features to obtain the target machine learning model. The machine learning model is used to identify the target call detail records corresponding to the target number and determine whether the target number is a suspected fraudulent number.

[0038] This model training method employs a machine learning-based approach to identify suspected fraudulent calls. It acquires first historical call detail records (CDRs) corresponding to the operator's network element operation domain and second historical CDRs corresponding to the operator's computation domain. Signaling features and call behavior features consistent with ultra-short call behavior are extracted from these two data sets. The machine learning model is then trained based on these features to obtain a target machine learning model. This model is used to identify the target CDRs corresponding to the target number, determining whether the target number is a suspected fraudulent number. This achieves the goal of identifying suspected fraudulent calls based on a learning model, thereby improving the identification rate of suspected fraudulent calls, enhancing terminal security, and preventing data and privacy leaks. Furthermore, it addresses the low accuracy and poor security issues associated with relying on caller ID tags provided by mobile security apps for fraudulent call identification in related technologies.

[0039] In some embodiments of this application, after obtaining the target machine learning model, the numbers in the call detail records can be identified in real time. Therefore, the target number to be analyzed can be obtained, and the target call detail records corresponding to the target number can be analyzed based on the machine learning model to determine whether the target number is a suspected fraudulent number.

[0040] To maximize the accuracy of model recognition, in some optional embodiments, before analyzing the target call detail records (CDRs) corresponding to the target number based on the machine learning model, it can be determined whether to update the model based on the call attributes of the target CDRs. This is to determine the model most suitable for the current scenario. This can be achieved as follows: Determine the call attributes of the target CDRs, where the call attributes include local attributes or outbound call attributes, wherein the number of local calls corresponding to the local attribute is greater than the number of current calls corresponding to the outbound call attribute; determine whether to update the target machine learning model based on the call attributes, wherein, if it is determined that the target machine learning model should be updated, adjust the first weight of the first signaling feature and the first call behavior feature corresponding to the first historical CDR data, and the second weight of the second signaling feature and the second call behavior feature corresponding to the second historical CDR data during the training process; train the target machine learning model using the adjusted first weight of the first signaling feature and the first call behavior feature, and the adjusted second weight of the second signaling feature and the second call behavior feature, to obtain a new target training model.

[0041] It should be noted that B-domain call detail records have the following characteristics: (1) More fields: They contain more call information fields (such as whether roaming, roaming location, call charges, IMEI / IMSI, etc.) and user information fields (such as the type of package used by the user, etc.); (2) Large latency: Since the billing call detail records need to be filled and issued by the operator group, the latency is relatively high. Usually, the billing call detail records of yesterday are issued every day, which cannot prevent fraud incidents in the middle of the event; (3) Complete data: For numbers belonging to the operator of this province, all call records of the number can be obtained regardless of whether they roam within or outside the province, and the behavior of numbers roaming outside the province can be completely analyzed and detected.

[0042] O-domain call detail records have the following characteristics: (1) As long as a mobile phone user of the same operator makes a call to a landline or mobile number within the province, inter-provincial, or internationally, a corresponding signaling will be generated within the province. The signaling is collected quickly and has strong real-time performance. (2) Low latency: After the call detail records are collected and output, they can be analyzed in real time. They can output fraud events and fraudulent numbers in near real-time, which is convenient for sending reminder text messages and coordinating with the public security bureau to conduct follow-up visits and block fraud. (3) Narrow scope: Since O-domain call detail records can only collect relevant calls of users within the province, if the fraudulent number is roaming outside the province, only the call records between the fraudulent number and users within the province can be collected. It is impossible to obtain all the calls of the fraudulent number for analysis. Therefore, it mainly targets fraudulent numbers and harassing numbers within the province for detection.

[0043] Based on the above call detail record (CDR) characteristics, given the high number of local calls, the weight of the O-domain CDRs during training should be increased. Therefore, as an optional implementation method, determining whether to update the target machine learning model based on call attributes includes: obtaining the third weights of the first signaling feature and the first call behavior feature during the historical training process of the target machine learning model, and the fourth weights of the second signaling feature and the second call behavior feature corresponding to the second historical CDR data; when the call attribute is local (i.e., a high number of local calls; it should be noted that intra-provincial calls are generally considered local calls), if the third weight is less than the preset weight, or the fourth weight is greater than the preset weight, then the target learning model is updated. In the target machine learning model to be used, if the third weight corresponding to the O-domain was small during the original training process, the third weight corresponding to the O-domain should be readjusted to increase it, and then trained again to obtain a new model. Then, data recognition is performed based on the new model.

[0044] As another optional implementation, determining whether to update the target machine learning model based on call attributes includes: obtaining the third weights of the first signaling feature and the first call behavior feature during the historical training process of the target machine learning model, and the fourth weights of the second signaling feature and the second call behavior feature corresponding to the second historical call detail record data; if the call attribute is outbound (i.e., there are many outbound calls; it should be noted that calls outside the province can generally be classified as outbound calls), if the third weight is greater than the preset weight, or the fourth weight is less than the preset weight, then the target learning model is updated. Similarly, if the fourth weight corresponding to the B domain is small during the original training process of the target machine learning model to be used, the fourth weight can be adjusted, and the model can be retrained. The aforementioned preset weight can be 50%.

[0045] Caller ID alerts, typically provided in related technologies, cannot block incoming calls, prevent answering calls, or prevent potential fraudulent activities. To address these issues, in an optional embodiment of this application, a blocking measure can be implemented upon identifying a suspected fraudulent number. Specifically, upon determining that a target number is a suspected fraudulent number, detection indicators corresponding to the suspected fraudulent number can be obtained. These indicators include: short-ring percentage, call failure rate, and caller ID dispersion. If at least two of these indicators exceed their corresponding preset values, the suspected fraudulent number is identified as a fraudulent number and sent to the operator for blocking.

[0046] Specifically, (1) the percentage of short-duration ringing can be found in [reference needed]. Figure 2 The horizontal axis represents the natural sequence number of suspicious numbers, and the vertical axis represents the percentage of short-running calls for the corresponding numbers. The higher the percentage of short-running calls, the higher the probability of a suspected fraudulent call. Verification shows that the percentage of short-running calls for suspicious numbers is generally above 0.9.

[0047] (2) For call failure rate, please refer to Figure 3 The horizontal axis represents the natural sequence number of suspicious phone numbers, and the vertical axis represents the call failure rate of the corresponding number. The higher the call failure rate, the higher the probability of a suspected scam call. Verification shows that the call failure rate of suspicious numbers is generally above 0.7%.

[0048] (3) For dialing frequency dispersion, please refer to [reference needed]. Figure 4 The horizontal axis represents the natural sequence number of suspicious numbers, and the vertical axis represents the dispersion of the corresponding dialed numbers. The higher the dispersion, the higher the probability of a suspected fraudulent call. Verification shows that the dispersion of the dialed numbers for suspicious numbers is generally above 0.6.

[0049] In some optional embodiments of this application, training a machine learning model based on signaling features and call behavior features to obtain a target machine learning model includes: determining user features corresponding to positive and negative samples based on signaling features and call behavior features, wherein positive samples include numbers that conform to fraud types, and negative samples include numbers that are not marked as fraud; extracting features from user features, wherein the extracted features include: number features, basic features, and extended features; using a random forest algorithm to obtain the Gini index corresponding to each of the number features, basic features, and extended features, wherein the smaller the Gini index, the lower the impurity; ranking each feature based on the Gini index, and selecting the top N features as target features; and training the machine learning model based on the target features to obtain the target machine learning model.

[0050] As an optional implementation, after determining whether the target number is a suspected fraudulent number, the method further includes: if the target number is determined to be a suspected fraudulent number, performing cluster analysis on the types of suspected fraudulent numbers, determining the fraud type to which each suspected fraudulent number belongs, generating prompt information for different fraud types, and sending the prompt information to the terminal on the target side.

[0051] Figure 5 and Figure 6 The overall modeling process and the training model are shown respectively, such as Figure 5 , Figure 6 As shown, the modeling includes:

[0052] 1. Sample Extraction: Positive sample extraction includes extracting numbers that match the modeled fraud type based on set conditions, while negative sample extraction includes numbers that are not labeled under the sampling conditions. Positive Sample Extraction: Set the following conditions to perform positive sample extraction.

[0053] (1) Based on the case report data, examine the case details registered by the police in the sample and find the fraudster's number that matches the fraud type in this modeling;

[0054] (2) Based on the reporting number provided by a company in a certain city, identify the fraudulent phone number that matches the fraud type in this modeling;

[0055] (3) Based on the reporting number on the 12321 reporting platform, identify the fraudulent phone number that matches the type of fraud investigated in this case;

[0056] (4) Based on the numbers marked on some Internet software labels, find the fraudulent numbers that match the type of fraud investigated in this case.

[0057] Negative sample extraction: Set sampling conditions and extract negative samples. Based on information from a certain city, randomly select a batch of seemingly normal user numbers from different number segments. Then query the Internet crawler data to see if these numbers are marked as harassment or fraud (excluding express delivery, food delivery, and real estate agencies). If they are not marked, they are considered to meet the characteristics of negative samples.

[0058] 2. Feature Extraction: Features are extracted from the samples. The extracted features include number features, basic features, and extended features extracted based on the B and O domain call detail records, as detailed below:

[0059] (1) Number characteristics: Filter out call detail records (CDRs) where the caller is a mobile or landline and the recipient is a mobile phone.

[0060] 2) Basic features: Select the fields of caller ID, called number, call start time, call end time, call duration, ringing duration, caller ID province, and caller ID city.

[0061] (3) Extended features: Add the time interval between two calls, the number of calls during off-peak hours (0-8 o'clock and 21-23 o'clock are off-peak hours), the number of failed calls, the average call interval, the standard deviation of the call interval, the peak call time, the number of short rings, the total call duration, and the number of called numbers. (4) Output the numbers and related data according to the protocol standard.

[0062] Further expansion of the features can yield the following:

[0063] (1) Daily feature extension: Based on the user's call behavior on the same day, analyze the user's basic features, including number length, number type, and affiliated operator; (2) Multi-day feature extension: Based on 7 days of data, find out the call behavior of consecutive days, find out the number of active days of fraudulent numbers in 7 days, etc.; (3) Statistical feature extension: Based on the same day or 7 days, calculate the call duration of the caller and the callee, the caller ratio, etc.

[0064] Based on the feature fields of the initial screening using 7-day historical billing records:

[0065]

[0066]

[0067]

[0068] Feature fields selected during the initial screening on the same day:

[0069]

[0070]

[0071]

[0072] 3. Algorithm Selection: For algorithm selection, tree-based classification algorithms are prioritized for in-depth data mining and detection of numerous call behavior features based on phone numbers, enabling accurate detection of fraudulent numbers in specific scenarios. Each node in a decision tree represents a condition related to a certain feature, aiming to divide the dataset in two according to different response variables. Impurity is used to determine the node (optimal condition). For classification problems, Gini impurity or information gain is typically used; for regression problems, variance or least squares fitting is usually employed. When training the decision tree, the reduction in tree impurity for each feature can be calculated. For a forest of decision trees, the average reduction in impurity for each feature can be calculated, and this average reduction is used as the feature selection value. Feature selection using tree-based algorithms can be achieved through multi-fold cross-validation, accumulating the importance of features at each fold. Features with a cumulative importance value of 0 are considered for removal. Choosing tree-based classification algorithms allows for better output of corresponding feature importance, improving the interpretability of the fraud scenario. Furthermore, tree-based algorithms are insensitive to redundant or low-contribution features during training, minimizing their impact on model prediction results.

[0073] 4. Model training:

[0074] (1) Based on the extracted features and the selected appropriate algorithm, a model is constructed, and the signaling features and user call behavior under different scenarios are analyzed and abstracted to refine the feature weights and thresholds.

[0075] (2) Evaluate the effectiveness of the training and test sets, and analyze the detection effect of suspected numbers. For example, based on the statistics of base station aggregation behavior, analyze abnormal aggregation behaviors such as different numbers aggregating in a small number of base stations to realize the judgment of abnormal call scenarios.

[0076] (3) For users whose short call numbers are suspected of being fraudulent in the initial screening, a high success rate indicates effectiveness; otherwise, continue to adjust the model's feature values ​​until accurate matching is achieved. Validate the trained model on the test set and conduct a pre-launch evaluation. Once the launch conditions are met, solidify the model.

[0077] The statistical characteristics accumulated from the modeling scenarios in this application can be evaluated using the following methods:

[0078] (1) Manual analysis: Since fraudulent call behaviors in different scenarios have their own unique characteristics, for example, the main characteristics of pornography fraud are often concentrated in nighttime calls. Therefore, for new scenarios, if none of the calls in the samples are made at night, there is no need to introduce statistical features related to nighttime calls. Similarly, if the number in the modeling scenario only has active call behavior on the current day, there is no need to introduce features related to call behavior over multiple days.

[0079] (2) Training and evaluation: For common call behavior features, sample data is extracted for model training, and the importance of the features is evaluated through methods such as K-fold analysis. If the feature importance is below the preset threshold in most cases, the feature is judged to be a redundant feature and is removed.

[0080] (3) Model performance: If the model is trained using only existing features, but the precision or recall rate on the test set does not meet the requirements for going live, it means that the existing features are insufficient to effectively distinguish between normal numbers and fraudulent numbers in this scenario. In this case, manual analysis of the sample data is required to extract more new features.

[0081] Data flow during the modeling and detection process, such as Figure 7 As shown, the process includes: (1) DPI data collection; (2) preprocessing of call detail records (CDR) files and storage; (3) writing CDRs to HDFS (distributed file system); (4) detection of one-ring numbers; (5) output of detection results; (6) data push; (7) front-end display.

[0082] User profiles are created by expanding features through feature engineering, and then modeled and analyzed using algorithms in LightGBM. LightGBM is a lightweight decision tree framework based on a decision tree learning algorithm that supports distributed learning. Its advantages include lower memory consumption and lower data partitioning complexity. While it employs a leaf-wise growth strategy, repeatedly splitting the leaf with the largest splitting gain (usually also the largest data volume) can lead to overfitting by growing a relatively deep decision tree. Therefore, LightGBM adds a maximum depth limit on top of the leaf-wise approach to prevent overfitting while maintaining high efficiency.

[0083] 5. Evaluation and parameter adjustment

[0084] Finally, based on the lightGBM algorithm, the acquired data from cases involving legitimate users and normal users were used as positive and negative samples to expand call behavior features, which were then used for model training. Finally, the importance of the trained model was evaluated, and the following 16 features that significantly contributed to the model's performance and were particularly effective in distinguishing numbers involved in cases were selected, as shown in the table below:

[0085] Experimental value assignment and testing were conducted based on the feature library in the table above:

[0086]

[0087] (1) General characteristics:

[0088] The user's seven-day outgoing call rate is greater than or equal to 0.73; the user's seven-day call rate shows that the location of the local end and the other end are inconsistent, which is greater than or equal to 0.95; the user's seven-day outgoing call rate shows that the number of times the mobile phone number was dialed is greater than or equal to 0.92.

[0089] (2) Feature branch 1:

[0090] The user made 38 or more outgoing calls in a single day; the number of different call types (lac_ci) in a single day was less than or equal to 3; the average duration of each outgoing call in a single day was greater than or equal to 20 seconds and less than or equal to 160 seconds; the number of different locations of the calling party in a single day was greater than or equal to 25; the user made 250 or more outgoing calls in seven days; the user's total outgoing call duration in seven days was greater than or equal to 11,000; the number of different locations of the calling party in seven days was greater than or equal to 38; the user was active for more than or equal to 6 days; and the user's total call charges in seven days were greater than or equal to 130 yuan.

[0091] (3) Feature branch 2:

[0092] The user's total outgoing call duration on the same day is greater than or equal to 1000 seconds; the user's outgoing call destination has 4 or more different locations on the same day; the user's average outgoing call duration on the same day is greater than or equal to 45 seconds; the user's average call charges per active day are greater than or equal to 9 yuan; the user's average total outgoing call duration per active day is greater than or equal to 1300 seconds; and the user's long-distance call charges for the past seven days are 0.

[0093] Figure 8 This is a schematic diagram of a process for identifying suspected fraudulent phone numbers according to an embodiment of this application, such as... Figure 8 As shown, the process includes:

[0094] Step S810: Obtain historical billing call detail records (CDRs) of fraudulent numbers from historical telecommunications network fraud incidents from the operator's O and B domains. These historical CDRs primarily originate from the operator's identified fraudulent number set, 12321 complaint numbers, police report numbers, and some internet-labeled numbers. Select fields from the user CDR data that meet the input conditions for ultra-short call features. These fields include: calling number, calling number type, called number, called number type, call start time, call end time, call duration, ringing duration, calling province, calling city, local cell number, remote cell number, local base station, and remote base station.

[0095] Step S812: Based on the characteristics of ultra-short call (USC) calls, relevant fields are selected from the O domain to extract signaling features and user call behavior to construct a model. Based on the characteristics of USC calls, relevant fields are selected from the B domain to extract signaling features and user call behavior to construct a model. Sample extraction is performed by extracting call detail records (CDRs) from the operator's B and O domains through data flow. Positive and negative samples are extracted to form a user sample library. Based on the low latency characteristic of the O domain, the following fields are selected: calling number, calling number type, called number, called number type, calling province, calling city, local cell number, local base station, and peer base station.

[0096] Based on the low latency characteristic of domain B, the following fields were selected: caller ID, called ID, call start time, call end time, call duration, ringing duration, caller's province, and caller's city. Positive samples were extracted from police reports, reports from a company in a certain city, and reports from the 12321 reporting platform to identify fraudulent numbers matching the type of fraud investigated. Negative samples were extracted from a subset of numbers tagged with certain internet labels, with a sampling condition of 20%, to identify fraudulent numbers matching the type of fraud investigated. A batch of seemingly normal user numbers was randomly selected from different number ranges, and then internet crawler data was queried to check if these numbers were marked as harassment or fraud (excluding delivery services and real estate agents). If no such markings were found, they were considered to meet the negative sample characteristics.

[0097] Features can be expanded to include, for example, the time interval between two calls, the number of calls during off-peak hours (0-8 AM and 9-11 PM), the number of failed calls, the average call interval, the standard deviation of the call interval, the peak call time, the number of short-term rings, the total call duration, and the number of called numbers.

[0098] Further expanding on the features, for example, analyzing a user's call behavior on a given day can reveal basic user characteristics, including number length, number type, and carrier affiliation. For instance, multi-day feature expansion can use 7 days of data to identify consecutive days of call behavior, such as the number of active days for fraudulent numbers within that 7-day period. Statistical feature expansion can use data from the current day or 7 days to calculate call duration and caller-caller ratio. Further expanding on the features, another example is algorithm selection: choosing a suitable classification algorithm for modeling based on the samples and features, with tree-based classification being the preferred approach.

[0099] Step S814 involves modeling and evaluating the performance on the training and test sets to analyze the detection effectiveness of the initial screening of suspected numbers. First, the call behavior characteristics of fraudulent numbers are statistically analyzed, and initial screening conditions and thresholds for each fusion scenario are comprehensively set. Abstract signaling features and user call behavior in different scenarios are analyzed, and feature weights and thresholds are refined. The features involved include multiple dimensions such as the ratio of callers to recipients, average call duration, call area dispersion, concentrated time points, and the proportion of repeated calls. Further in-depth analysis of user behavior is conducted to clarify key features and weights. Based on the established model, the model features and weights are continuously trained using big data machine learning. For positive and negative samples, the feature weights are optimized through machine learning on the model training set.

[0100] By employing machine learning algorithms such as Random Forest and Light-GBM (Gradient Boosting Decision Tree), and conducting in-depth data mining and detection based on a large number of call behavior features of phone numbers, we can achieve accurate detection of fraudulent phone numbers in specific scenarios.

[0101] Feature engineering is used to expand features and create user profiles. Different machine learning methods have different characteristics. For example, modeling and analysis can be performed using algorithms in LightGBM. LightGBM is a lightweight GB framework that uses decision tree-based learning algorithms and supports distributed processing.

[0102] A leaf-wise growth strategy is adopted, where the leaf with the largest splitting gain (generally also the largest data volume) is selected from all current leaves and split, and this process is repeated; however, this can result in a relatively deep decision tree, leading to overfitting. Continuous optimization yields 16 key features to form a feature library for ultra-short call harassment numbers. Experimental examples of conversations are assigned for processing and selection, with the caller being a mobile phone user and meeting general feature conditions:

[0103] 1. The user's seven-day outgoing call ratio is greater than or equal to 0.73; 2. The user's seven-day call location is inconsistent between the local and remote locations, which is greater than or equal to 0.95; 3. The user's seven-day outgoing call ratio for mobile phone numbers is greater than or equal to 0.92; For mobile phone users, the following conditions apply: 1. The user's daily outgoing call count is greater than or equal to 38; 2. The user's daily outgoing call lac_ci type is less than or equal to 3; 3. The user's daily average call duration is greater than or equal to 20 seconds and less than or equal to 160 seconds; 4. The user's daily remote location type is greater than or equal to 25; 5. The user's seven-day outgoing call count is greater than or equal to 250; 6. The user's seven-day total outgoing call duration is greater than or equal to 11,000; 7. The user's seven-day remote location type is greater than or equal to 38; 8. The user has been active for more than 6 days; 9. The user's seven-day total call charges are greater than or equal to 130 yuan.

[0104] Specifically, the user selects between mobile phones and landlines for processing. The conditions for mobile phone users are as follows: 1. The total duration of the user's outgoing calls on the same day is greater than or equal to 1000 seconds; 2. The location of the user's outgoing call destination on the same day is greater than or equal to 4; 3. The average duration of each outgoing call on the same day is greater than or equal to 45 seconds;

[042] 4. The average cost of the user's calls per active day is greater than or equal to 9 yuan; 5. The average total duration of the user's outgoing calls per active day is greater than or equal to 1300 seconds; 6. The user's long-distance call charges for the past seven days are 0.

[0105] Call detail records (CDRs) are streamlined and reorganized, removing redundant fields and adding new ones. CDRs are partitioned by number, ensuring balanced data volume across each partition. This overcomes computational resource limitations, supporting long-term data analysis of CDRs by anti-fraud programs. CDR deduplication and reorganization further overcome computational resource limitations, enabling long-term data analysis of CDRs by anti-fraud programs.

[0106] Step S816: Model results are detected, whitelisted users and historical duplicate data are filtered, and the model results are used to detect the final suspected short call fraud numbers.

[0107] Figure 9 This is another method for identifying suspected fraudulent phone numbers provided according to embodiments of this application, such as... Figure 9 As shown, the method includes:

[0108] S902, Obtain the target number to be analyzed;

[0109] S904, based on a machine learning model, analyzes the target call detail records (CDRs) corresponding to the target number to determine whether the target number is a suspected fraudulent number. The machine learning model is trained based on signaling features and call behavior features. The signaling features and call behavior features are obtained by filtering the first historical CDR data corresponding to the operator's network element operation domain and the second historical CDR data corresponding to the operator's computing domain.

[0110] This method acquires the target number to be analyzed, and then analyzes the target call detail records (CDRs) corresponding to the target number based on a machine learning model to determine whether the target number is a suspected fraudulent number. The machine learning model is trained based on signaling features and call behavior features, which are obtained by filtering first historical CDR data corresponding to the operator's network element operation domain and second historical CDR data corresponding to the operator's computation domain. This achieves the goal of identifying suspected fraudulent calls based on a learning model, thereby improving the identification rate of suspected fraudulent calls, enhancing terminal security, and preventing data and other privacy leaks. Furthermore, it solves the technical problems of low accuracy and poor security in related technologies that rely on caller ID tags provided by mobile phone management apps for fraudulent call identification.

[0111] Figure 10 This is a model training apparatus according to an embodiment of this application, such as... Figure 10 As shown, the device includes:

[0112] The first acquisition module 1002 is used to acquire the first historical call detail record data corresponding to the operator network element operation domain and the second historical call detail record data corresponding to the operator computing domain.

[0113] The second acquisition module 1004 is used to acquire signaling features and call behavior features that conform to ultra-short call behavior from the first historical call detail data and the second historical call detail data.

[0114] The determination module 1006 is used to train the machine learning model based on signaling features and call behavior features to obtain the target machine learning model. The machine learning model is used to identify the target call detail records corresponding to the target number and determine whether the target number is a suspected fraudulent number.

[0115] In this device, the first acquisition module 1002 is used to acquire the first historical call detail record (CDR) data corresponding to the operator's network element operation domain and the second historical CDR data corresponding to the operator's computing domain; the second acquisition module 1004 is used to acquire signaling features and call behavior features that conform to ultra-short call behavior from the first historical CDR data and the second historical CDR data; the determination module 1006 is used to train a machine learning model based on the signaling features and call behavior features to obtain a target machine learning model, wherein the machine learning model is used to identify the target CDR corresponding to the target number and determine whether the target number is a suspected fraudulent number, thereby achieving the purpose of identifying suspected fraudulent calls based on the learning model, thereby improving the identification rate of suspected fraudulent calls, improving terminal security, and avoiding the leakage of data and other privacy technologies, and thus solving the technical problems of low identification accuracy and poor security caused by the caller ID reminder provided by the mobile phone manager in related technologies.

[0116] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute any model training method.

[0117] Specifically, the aforementioned storage medium is used to store program instructions for the following functions, thereby implementing the following functions:

[0118] The system acquires the first historical call detail record (CDR) data corresponding to the operator's network element operation domain and the second historical CDR data corresponding to the operator's computing domain; it extracts signaling features and call behavior features that conform to ultra-short call behavior from the first and second historical CDR data; it trains the machine learning model based on the signaling features and call behavior features to obtain the target machine learning model, wherein the machine learning model is used to identify the target CDR corresponding to the target number and determine whether the target number is a suspected fraudulent number.

[0119] Optionally, in this embodiment, the storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of the storage medium include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0120] An electronic device is provided according to an embodiment of the present application, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described model training methods.

[0121] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0122] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0123] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0127] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0128] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A model training method, characterized in that, include: Obtain the first historical call detail record (CDR) data corresponding to the operator's network element operation domain, and the second historical CDR data corresponding to the operator's computing domain; Obtain signaling features and call behavior features that match ultra-short call behavior from the first historical call detail record data and the second historical call detail record data; The machine learning model is trained based on the signaling features and the call behavior features to obtain a target machine learning model, wherein the machine learning model is used to identify the target call detail records corresponding to the target number and determine whether the target number is a suspected fraudulent number; After obtaining the target machine learning model, the method further includes: obtaining the target number to be analyzed, analyzing the target call detail record corresponding to the target number based on the machine learning model, and determining whether the target number is the suspected fraudulent number; Before analyzing the target call detail records (CDRs) corresponding to the target number based on the machine learning model, the process includes: determining the call attributes of the target CDRs, wherein the call attributes include: local attributes or outbound call attributes, wherein when the call attribute is the local attribute, there are more local calls, and when the call attribute is the outbound call attribute, there are more outbound calls; determining whether to update the target machine learning model based on the call attributes, wherein, if it is determined that the target machine learning model should be updated, adjusting the first weight of the first signaling feature and the first call behavior feature corresponding to the first historical CDR data, and the second weight of the second signaling feature and the second call behavior feature corresponding to the second historical CDR data during the training process; training the target machine learning model using the adjusted first weight of the first signaling feature and the first call behavior feature, and the adjusted second weight of the second signaling feature and the second call behavior feature, to obtain a new target training model.

2. The method according to claim 1, characterized in that, Determining whether to update the target machine learning model based on the call attributes includes: The third weights of the first signaling feature and the first call behavior feature during the training of the target machine learning model in historical time periods are obtained, as well as the fourth weights of the second signaling feature and the second call behavior feature corresponding to the second historical call detail record data. If the call attribute is the local attribute, and the third weight is less than the preset weight, or the fourth weight is greater than the preset weight, then the target machine learning model is updated.

3. The method according to claim 1, characterized in that, Determining whether to update the target machine learning model based on the call attributes includes: The third weights of the first signaling feature and the first call behavior feature during the training of the target machine learning model in historical time periods are obtained, as well as the fourth weights of the second signaling feature and the second call behavior feature corresponding to the second historical call detail record data. If the call attribute is an outbound call attribute, and the third weight is greater than the preset weight, or the fourth weight is less than the preset weight, then the target machine learning model is updated.

4. The method according to claim 1, characterized in that, After determining whether the target number is the suspected fraudulent number, the method further includes: If the target number is determined to be the suspected fraudulent number, the detection indicators corresponding to the suspected fraudulent number are obtained, wherein the detection indicators include: short ringing rate, call failure rate and dispersion of dialed numbers; If at least two of the following detection indicators—the short-running ringing rate, the call failure rate, and the dispersion of the dialed number—are greater than their corresponding preset values, the suspected fraudulent number is determined to be a fraudulent number, and the fraudulent number is sent to the operator for the operator to shut down the fraudulent number.

5. The method according to claim 1, characterized in that, The machine learning model is trained based on the signaling features and the call behavior features to obtain the target machine learning model, including: User characteristics corresponding to positive and negative samples are determined based on the signaling characteristics and the call behavior characteristics, wherein the positive samples include numbers that conform to the fraud type, and the negative samples include numbers that are not marked as fraud. Feature extraction is performed on the user characteristics, wherein the extracted features include: number features, basic features, and extended features; The Gini index corresponding to each of the number features, basic features, and extended features is obtained using the random forest algorithm. The smaller the Gini index, the lower the impurity. The features are sorted based on the Gini index, and the top N features are selected as target features. The machine learning model is then trained based on the target features to obtain the target machine learning model.

6. The method according to claim 1, characterized in that, After determining whether the target number is the suspected fraudulent number, the method further includes: If the target number is determined to be the suspected fraud number, cluster analysis is performed on the type of the suspected fraud number to determine the fraud type to which each suspected fraud number belongs. Prompt information is generated for different fraud types and sent to the terminal on the target side.

7. A method for identifying suspected fraudulent phone numbers, characterized in that, include: Obtain the target number to be analyzed; The machine learning model is used to analyze the target call detail records (CDRs) corresponding to the target number to determine whether the target number is a suspected fraudulent number. The machine learning model is trained based on signaling features and call behavior features. The signaling features and call behavior features are obtained by filtering the first historical CDR data corresponding to the operator network element operation domain and the second historical CDR data corresponding to the operator computing domain. Before analyzing the target call detail records (CDRs) corresponding to the target number based on the machine learning model, the process includes: determining the call attributes of the target CDRs, wherein the call attributes include: local attributes or outbound call attributes, wherein when the call attribute is the local attribute, there are more local calls, and when the call attribute is the outbound call attribute, there are more outbound calls; determining whether to update the target machine learning model based on the call attributes, wherein, if it is determined that the target machine learning model should be updated, adjusting the first weights of the first signaling feature and the first call behavior feature corresponding to the first historical CDR data, and the second weights of the second signaling feature and the second call behavior feature corresponding to the second historical CDR data during the training process; training the target machine learning model using the adjusted first weights of the first signaling feature and the first call behavior feature, and the adjusted second weights of the second signaling feature and the second call behavior feature, to obtain a new target training model.

8. A model training device, characterized in that, include: The first acquisition module is used to acquire the first historical call detail record (CDR) data corresponding to the operator network element operation domain and the second historical CDR data corresponding to the operator computing domain. The second acquisition module is used to acquire signaling features and call behavior features that conform to ultra-short call behavior from the first historical call detail record data and the second historical call detail record data. The determination module is used to train a machine learning model based on the signaling features and the call behavior features to obtain a target machine learning model, wherein the machine learning model is used to identify the target call detail records (CDRs) corresponding to the target number and determine whether the target number is a suspected fraudulent number; after obtaining the target machine learning model, the module further includes: acquiring the target number to be analyzed, analyzing the target CDRs corresponding to the target number based on the machine learning model, and determining whether the target number is the suspected fraudulent number; before analyzing the target CDRs corresponding to the target number based on the machine learning model, the module includes: determining the call attributes of the target CDRs, wherein the call attributes include: local attributes or outbound call attributes, wherein, when the call attribute is the local attribute, the local... If the number of calls is high, and the call attribute is the outbound call attribute, then the number of calls from other locations is high. Based on the call attribute, it is determined whether to update the target machine learning model. If it is determined that the target machine learning model should be updated, the first weights of the first signaling feature and the first call behavior feature corresponding to the first historical call detail record (CDR) data, and the second weights of the second signaling feature and the second call behavior feature corresponding to the second historical CDR data, are adjusted during the training process. The target machine learning model is then trained using the adjusted first weights of the first signaling feature and the first call behavior feature, and the adjusted second weights of the second signaling feature and the second call behavior feature, to obtain a new target training model.

9. A non-volatile storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to execute the model training method according to any one of claims 1 to 6.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the model training method as described in any one of claims 1 to 6.

Images