System for phishing number pre-detection and its operation method

Abstract

A method of operation for a phishing number pre-detection system operating with at least one processor, wherein a model is trained to predict the probability of voice phishing of a roaming outgoing call using terminal activity information including roaming service usage history information by phone number and phone numbers reported for voice phishing. A specific roaming outgoing call originating from a specific country via a roaming service is obtained, and the model is used to obtain the probability of voice phishing of the specific roaming outgoing call from the terminal activity information of the specific roaming outgoing call. Based on the probability of voice phishing obtained from the terminal activity information of the specific roaming outgoing call and the phishing risk level of the country of origin of the specific roaming outgoing call, it is determined whether the specific roaming outgoing call is a voice phishing call.

Description

Technology Field

[0001] The present invention relates to a phishing number pre-detection system and a method of operation thereof, which detects suspected voice phishing numbers in advance by utilizing a caller's call service usage pattern or roaming service usage pattern. Background Technology

[0002] Telecommunications financial fraud refers to fraudulent methods that intentionally deceive a target in a telephone or communication environment for purposes such as demanding money or stealing personal information. Among these, voice phishing is a general term for fraudulent methods carried out in a voice call environment.

[0003] With the recent rise in voice phishing crimes and the resulting increasing damage, there have been various attempts to detect voice phishing. For instance, the KoBERT model, a Korean version of the BERT (Bidirectional Encoder Representations from Transformers) model, has been used for voice phishing detection; however, this technology faced difficulties in actual application due to its low accuracy.

[0004] Furthermore, with the recent development of Large Language Model (LLM) technology that converts speech into text and detects voice phishing based on text, methods for detecting voice phishing during a call are being proposed. However, while these methods may enable real-time detection, they cannot detect voice phishing in advance or at an early stage.

[0005] Furthermore, most voice phishing attempts originate from overseas, where perpetrators deceive victims by manipulating phone numbers to disguise international calls as domestic ones. Victims fall victim to these scams because they proceed with the call based solely on the information displayed as a domestic number, yet there is a problem in that this cannot be detected in advance. The problem to be solved

[0006] Accordingly, the present invention provides a phishing number pre-detection system and a method of operation thereof that can detect voice phishing in advance by determining abnormal usage patterns through the caller's call service usage patterns and roaming service usage patterns without directly utilizing sensitive personal information such as an individual's call history or call counterpart information. means of solving the problem

[0007] A method of operation of a phishing number pre-detection system operating with at least one processor, which is one feature of the present invention for achieving the technical problem of the present invention, comprises the steps of: training a model that predicts the probability of voice phishing of a roaming outgoing call using terminal activity information including roaming service usage history information by phone number and phone numbers reported for voice phishing; obtaining a specific roaming outgoing call originating from a specific country through a roaming service; obtaining the probability of voice phishing of the specific roaming outgoing call from terminal activity information of the specific roaming outgoing call using the model; and determining whether the specific roaming outgoing call is a voice phishing call based on the probability of voice phishing obtained from the terminal activity information of the specific roaming outgoing call and the phishing risk level of the country of origin of the specific roaming outgoing call.

[0008] The above training step may include a step of calculating a voice phishing rate by country based on the roaming service usage history information, and a step of setting a threshold value to determine whether a roaming outgoing call originating from each country is a voice phishing call using the voice phishing risk level by country calculated based on the above voice phishing rate by country.

[0009] The step of setting the threshold may include: a step of calculating the voice phishing rate by country based on the roaming service usage history information, based on terminal activity information labeled with voice phishing information by country and terminal activity information labeled with normal information by country; a step of setting a plurality of candidate thresholds within a preset threshold range according to the acquired standard conditions; and a step of setting one of the plurality of candidate thresholds as the optimal threshold for each country based on the performance indicator calculated for each of the plurality of candidate thresholds.

[0010] The step of setting the optimal threshold value may include the performance indicator including the recall and F1 score calculated for each candidate threshold value, and setting the threshold value with the lowest false positive rate among the candidate threshold values ​​with calculated recalls satisfying the acquired criteria conditions as the optimal threshold value for each country.

[0011] If there is no candidate threshold with a calculated recall rate satisfying the above criteria conditions, the method may include the step of setting the candidate threshold with the highest calculated F1-score as the optimal threshold for each country.

[0012] The step of training the above model may include the step of generating training data in which the terminal activity information further includes communication service usage history information, and based on the communication service usage history information, daily activity amount regarding communication service usage by phone number and initial activity amount regarding communication service usage used for a preset period after subscribing to the communication service by phone number.

[0013] The step of training the above model can be performed such that when the training data is input into the model, the model identifies a daily activity pattern based on the daily activity amount, identifies an initial usage pattern on pre-set dates after subscribing to the communication service based on the initial activity amount, and trains the model to output a probability value of voice phishing based on the daily activity pattern and the initial usage pattern.

[0014] The step of training the above model may further include the step of obtaining country change information by phone number by checking the number of daily visited countries and the number of times the country is changed from the roaming service usage history, and the step of training the model to identify country change patterns by inputting the country change information by phone number into the model.

[0015] The step of determining whether the specific roaming outgoing call is a voice phishing call may include the step of notifying the receiving terminal that the specific roaming outgoing call is a voice phishing call if the voice phishing probability is greater than or equal to a threshold set in the specific country.

[0016] As another feature of the present invention for achieving the technical problem of the present invention, a phishing number pre-detection system comprises a processor equipped with a model for detecting whether a roaming outgoing call is a voice phishing call, wherein the processor trains a model that predicts the probability of voice phishing of a roaming outgoing call using terminal activity information including roaming service usage history information by phone number and phone numbers reported for voice phishing, and when a specific roaming outgoing call originating from a specific country through a roaming service is acquired, the processor obtains the probability of voice phishing of the specific roaming outgoing call from the terminal activity information of the specific roaming outgoing call using the model, and determines whether the specific roaming outgoing call is a voice phishing call based on the probability of voice phishing obtained from the terminal activity information of the specific roaming outgoing call and the phishing risk level of the country of origin of the specific roaming outgoing call.

[0017] The processor can calculate a voice phishing rate by country based on the roaming service usage history information, and use the voice phishing risk level by country calculated based on the voice phishing rate by country to set a threshold value for determining whether a roaming outgoing call originating from each country is a voice phishing call.

[0018] The processor calculates the voice phishing rate for each country based on the roaming service usage history information, the terminal activity information labeled with voice phishing information for each country, and the terminal activity information labeled with normal information for each country, sets a plurality of candidate threshold values ​​within a preset threshold range according to the acquired standard conditions, and sets one of the plurality of candidate threshold values ​​as the optimal threshold value for each country based on the performance indicator calculated for each of the plurality of candidate threshold values.

[0019] The processor can set the threshold with the lowest false positive rate among the candidate thresholds with calculated recalls and F1 scores for each candidate threshold as the optimal threshold for each country, wherein the performance indicators include recalls and F1 scores calculated for each candidate threshold and satisfy the acquired criteria conditions.

[0020] If there is no candidate threshold with a calculated recall satisfying the above criteria condition, the processor may set the candidate threshold with the highest calculated F1-score as the optimal threshold for each country.

[0021] The processor can generate as training data daily activity amounts for communication service usage by phone number and initial activity amounts for communication service usage used for a preset period after subscribing to the communication service by phone number, based on communication service usage history information further included in the terminal activity information.

[0022] When the training data is input into the model, the processor can be trained to identify a daily activity pattern based on the daily activity amount, identify an initial usage pattern on preset dates after subscribing to the communication service based on the initial activity amount, and output a probability value of voice phishing based on the daily activity pattern and the initial usage pattern.

[0023] The processor can identify the number of countries visited per day and the number of times the country is changed from the roaming service usage history to obtain country change information by phone number, and input the country change information by phone number into the model to train it to identify country change patterns.

[0024] The processor can detect that the roaming outgoing call is a voice phishing call and notify the receiving terminal that it is voice phishing if the probability is greater than or equal to a threshold set for the specific country. Effects of the invention

[0025] According to the present invention, suspected voice phishing numbers can be detected in advance before a call is connected without concerns about infringement of personal information.

[0026] In addition, by dynamically selecting suspected voice phishing countries based on data used for pre-detection of voice phishing, rather than fixing them to specific countries, it is possible to actively respond to changing voice phishing patterns.

[0027] In addition, by analyzing in detail the cross-border movement patterns and activities in suspected countries of callers who attempted voice phishing, it is possible to distinguish between general roaming users and suspected voice phishing numbers with high accuracy.

[0028] In addition, by analyzing usage patterns from the activation of the terminal until the first use of the roaming service, suspected voice phishing numbers can be identified early, and damage caused by this can be prevented in advance. Brief explanation of the drawing

[0029] FIG. 1 is an example of an environment in which a phishing number pre-detection system according to an embodiment of the present invention is applied. FIGS. 2 and FIGS. 3 are structural diagrams of a phishing number pre-detection system according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating the operation method of a phishing number pre-detection system according to an embodiment of the present invention. FIG. 5 is a flowchart of a method for a phishing number pre-detection system to calculate a voice phishing rate by country according to an embodiment of the present invention. FIG. 6 is a flowchart of a method for a phishing number pre-detection system to calculate activity levels per phone number according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating a method for a phishing number pre-detection system to generate user-specific country change information according to an embodiment of the present invention. FIG. 8 is a flowchart of a method for a phishing number pre-detection system to dynamically set a threshold value according to an embodiment of the present invention. FIG. 9 is a hardware configuration diagram of a computing device according to an embodiment of the present invention. Specific details for implementing the invention

[0030] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0031] Throughout the specification, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0032] Hereinafter, a phishing number pre-detection system and a method of operation thereof according to an embodiment of the present invention will be described in detail with reference to the drawings.

[0033] FIG. 1 is an example of an environment in which a phishing number pre-detection system according to an embodiment of the present invention is applied.

[0034] As illustrated in FIG. 1, the phishing number pre-detection system (100) is linked with a communication service usage history management server (200), a report information management server (300), and a phishing notification service providing server (400).

[0035] The communication service usage history management server (200) is a server managed by a telecommunications company that provides mobile communication services, and manages terminal information held by each subscriber using the communication service and communication service usage history information regarding the use of voice / text services by the subscribers using the terminal. Here, the terminal information is described as an example of the terminal's phone number, but can be used as various identification information (e.g., IMEI, etc.) that can distinguish the terminal.

[0036] Additionally, the communication service usage history management server (200) manages roaming service usage history information of subscribers who have subscribed to the roaming service and used the roaming service overseas. At this time, the roaming service usage history information includes various roaming service usage histories such as the date of subscription to the roaming service, the start date of the roaming service, the end date of the roaming service, and the country where the service was used.

[0037] The report information management server (300) is a server managed by relevant institutions that perform duties for voice phishing prevention or manage information reported as voice phishing, such as the National Police Agency, the Financial Supervisory Service, or the Korea Internet & Security Agency (KISA). When a victim who has suffered voice phishing damage reports it, the report information management server (300) manages information related to the victim, as well as information related to the phishing criminal, such as the phone number of the terminal used by the voice phishing criminal, the originating region, and the time of origination, and various other information not described above.

[0038] The phishing number pre-detection system (100) obtains communication service usage history information corresponding to a phone number registered in the report information management server (300) (hereinafter referred to as a ‘voice phishing phone number’ for convenience of explanation) from the communication service usage history management server (200).

[0039] Then, information indicating that the phone number is a voice phishing number (hereinafter referred to as a "voice phishing label" for convenience of explanation) is labeled on the roaming service usage history information for the voice phishing phone number. At this time, the phishing number pre-detection system (100) is described as an example in which a label indicating that it is voice phishing is labeled on the roaming service usage history for the voice phishing phone number, and a normal usage label indicating that it is normal roaming service usage is labeled on the roaming service usage history for a phone number that is not voice phishing.

[0040] Additionally, the phishing number pre-detection system (100) collects and pre-processes information managed by the communication service usage history management server (200). An artificial intelligence model (hereinafter referred to as the 'phishing number pre-detection model' for convenience of explanation) installed in the phishing number pre-detection system (100) is trained to use the pre-processed information as training data and to output a probability value indicating whether voice phishing is possible when phone number and country information are input.

[0041] In an embodiment of the present invention, a random forest model is used as an example of a phishing number pre-detection model installed in a phishing number pre-detection system (100). However, the phishing number pre-detection model is not necessarily limited to use in this way.

[0042] When a caller attempts to make a call to a receiving terminal (20) via an overseas network using a sending terminal (10), the phishing number pre-detection system (100) checks the phone number of the sending terminal and the country of origin in the roaming call.

[0043] The phishing number pre-detection system (100) receives communication service usage history information and roaming service usage history information corresponding to a phone number from a communication service usage history management server (200) and inputs them into a phishing number pre-detection model that has been trained. In an embodiment of the present invention, for convenience of explanation, the communication service usage history information and the roaming service usage history information are referred to as "outgoing terminal activity information." The phishing number pre-detection model outputs a probability value indicating the possibility that a roaming outgoing call is a voice phishing call.

[0044] Then, the phishing number pre-detection system (100) adjusts the voice phishing judgment threshold based on the country-specific risk level of the originating country of the roaming call. Then, the phishing number pre-detection system (100) compares the probability value output from the phishing number pre-detection model with the threshold, and if it is greater than or equal to the threshold, determines that the roaming call may be voice phishing and provides information to the receiving terminal (20) indicating that it is voice phishing.

[0045] At this time, the phishing number pre-detection system (100) performs a check for the possibility of voice phishing of a roaming outgoing call in advance before the call is connected between the outgoing terminal (10) and the receiving terminal (20).

[0046] The structure of such a phishing number pre-detection system (100) is described with reference to FIGS. 2 and FIGS. 3.

[0047] FIGS. 2 and FIGS. 3 are structural diagrams of a phishing number pre-detection system according to an embodiment of the present invention.

[0048] FIG. 2 shows the configuration of a phishing number pre-detection system (100) in the step of training a phishing number pre-detection model in advance, and FIG. 3 shows the configuration of a phishing number pre-detection system (100) that detects voice phishing in advance before a call connection using the trained phishing number pre-detection model.

[0049] As illustrated in FIGS. 2 and 3, the phishing number pre-detection system (100) includes an information collection unit (110), a preprocessing unit (120), a phishing number pre-detection model (130), a threshold optimization unit (140), and a notification delivery unit (150). In an embodiment of the present invention, a random forest model is used as the phishing number pre-detection model (130) for the description.

[0050] First, as illustrated in FIG. 2, the information collection unit (110) links with the communication service usage history management server (200) and the report information management server (300) to collect communication service usage history information, roaming service usage history information, and report information from each server. Since the method by which the information collection unit (110) collects information from each server is diverse, the embodiment of the present invention is not limited to any one method.

[0051] The preprocessing unit (120) preprocesses each piece of information collected by the information collection unit (110) to generate training data.

[0052] That is, the preprocessing unit (120) labels the roaming service usage history information corresponding to the voice phishing phone number of the report information received from the report information management server (300) with voice phishing information indicating that the phone number is a voice phishing number.

[0053] In addition, the preprocessing unit (120) processes the collected communication service usage history information and calculates the daily activity amount and the initial activity amount for each phone number.

[0054] That is, the preprocessing unit (20) calculates the daily activity amount indicating how much communication service was used per day for each phone number, based on the number of outgoing messages, the number of incoming messages, the number of outgoing calls, the number of incoming calls, the outgoing call time, and the incoming call time included in the communication service usage history information for each phone number. Then, the preprocessing unit (120) processes the communication service usage history information to extract the initial activity amount used for a plurality of pre-set periods from the subscription date when the communication service was first subscribed for each phone number.

[0055] In an embodiment of the present invention, activity indicators for using message and call services for one day based on the date the user subscribed to the communication service, activity indicators for three days based on the subscription date, and activity indicators for seven days based on the subscription date are extracted over a plurality of preset periods. To this end, the explanation is given as an example in which subscription information is also included in the communication service usage history information.

[0056] Additionally, the preprocessing unit (120) calculates the number of normal users who used normal call services and the number of abnormal users who attempted phishing for each country where an overseas roaming call occurred, based on voice phishing information labeled in the roaming service usage history information by phone number.

[0057] The preprocessing unit (120) calculates the voice phishing rate by country based on the calculated number of normal users and abnormal users. Then, the preprocessing unit (120) calculates the voice phishing risk level by country based on the calculated voice phishing rate by country. In the embodiment of the present invention, the risk level is calculated in order of countries with high voice phishing rates, and since the preprocessing unit (120) calculates the voice phishing risk level according to the rate through various methods, the embodiment of the present invention is not limited to any one method.

[0058] Additionally, the preprocessing unit (120) processes roaming service usage history information to aggregate the number of countries visited by the user on a daily basis. Then, the preprocessing unit (120) extracts the maximum number of daily visited countries and the average number of daily visited countries as user-specific country change information and generates them as training data.

[0059] In an embodiment of the present invention, the preprocessing unit (120) is described as an example of generating training data including daily activity amount and initial activity amount for each phone number based on communication service usage history information and roaming service usage history information. However, the types of information included in the training data or the collected information processed are not necessarily limited in this way.

[0060] The preprocessing unit (120) transmits the country-specific voice phishing risk level, set according to the country-specific voice phishing rate, to the threshold optimization unit (140). Then, the preprocessing unit (120) inputs the training data into the phishing number pre-detection model (130).

[0061] The phishing number pre-detection model (130) is trained to output a probability value for the possibility of voice phishing when the daily activity and initial activity of a phone number labeled with voice phishing information, and the daily activity and initial activity of a phone number not labeled with voice phishing information are input.

[0062] That is, when daily activity is input, the phishing number pre-detection model (130) analyzes the variability of daily activity patterns for each phone number and identifies the characteristics of daily activity patterns of normal communication service users (hereinafter referred to as 'normal users' for convenience of explanation) and users who attempted voice phishing (hereinafter referred to as 'abnormal users' for convenience of explanation).

[0063] In addition, when the phishing number pre-detection model (130) inputs the initial activity amount, which is training data, it identifies the characteristics of the initial usage patterns of normal users and abnormal users for each phone number.

[0064] Normal users exhibit diverse behavioral patterns when first subscribing to a telecommunications service. In contrast, abnormal users display artificial and patterned behaviors (e.g., changing countries at precise rates or using the service in concentrated countries).

[0065] In addition, the phishing number pre-detection model (130) receives user-specific country change information as input and identifies characteristics of country change patterns such as the maximum number of countries visited per day, the average number of countries visited per day, the number of times countries were changed, and the average daily country change rate.

[0066] In this way, the phishing number pre-detection model (130) is trained by receiving daily activity volume, initial activity volume, and country change information as input, so that even if an abnormal user attempts voice phishing immediately after subscribing to a communication service, the initial phishing attempt can be quickly detected.

[0067] And, as illustrated in FIG. 3, when the preprocessing unit (120) obtains a roaming outgoing call, it extracts a phone number and a country from the roaming outgoing call. The preprocessing unit (120) receives communication service usage history information of the terminal corresponding to the phone number and roaming service usage history information from the call service usage history management server (200).

[0068] And the preprocessing unit (120) calculates the daily activity amount and initial activity amount of the outgoing terminal mapped to the corresponding phone number from the communication service usage history information and the roaming service usage history information.

[0069] When the calculated daily activity amount and initial activity amount are input into the learned phishing number pre-detection model (130), the learned phishing number pre-detection model (130) identifies the caller's communication service usage pattern or roaming service usage pattern based on the input daily activity amount and initial activity amount. Then, based on the identified pattern, the phishing number pre-detection model (130) outputs the probability value that the outgoing roaming call is voice phishing.

[0070] Meanwhile, the threshold optimization unit (140) of FIG. 2 adjusts the threshold based on the country-specific voice phishing risk level and voice phishing label calculated by the preprocessing unit (120), and the predicted probability value. Here, the threshold refers to a reference value for determining the possibility that an outgoing roaming call is voice phishing based on the probability value output from the phishing number pre-detection model (130).

[0071] That is, the threshold optimization unit (140) does not detect voice phishing using a threshold fixed to a specific value as in the conventional method, but rather adjusts the threshold value in various ways according to the probability values ​​output by the phishing number pre-detection model (130) regarding the possibility of voice phishing and the constraints input by the system administrator. In the embodiment of the present invention, the case where a recall target value is input as a constraint is explained as an example.

[0072] To this end, it is assumed that a range of threshold values ​​that can be adjusted by the threshold optimization unit (140) is pre-set. In an embodiment of the present invention, candidate threshold values ​​are set at intervals of 0.05 within a threshold range from 0.1 to 0.95, as an example.

[0073] Then, the threshold optimization unit (140) compares the probability values ​​predicting the possibility of voice phishing calculated based on each candidate threshold value, and calculates performance indicators including recall rate, false positive rate, and F1 score for each candidate threshold value. The threshold optimization unit (140) can calculate the recall rate, false positive rate, and F1 score for each candidate threshold value in various ways, and the embodiment of the present invention is not limited to any one method.

[0074] The threshold optimization unit (140) checks whether there is a candidate threshold that shows a recall rate higher than the recall rate target value entered by the manager for each candidate threshold. If there are multiple candidate thresholds that are higher than the recall rate, the threshold optimization unit (140) determines the candidate threshold with the lowest false positive rate among each candidate threshold as the optimal threshold.

[0075] The threshold optimization unit (140) is described as an example of mapping the determined optimal threshold to all countries. However, the threshold optimization unit (140) may select the top n countries based on the voice phishing risk level by country, and the threshold optimization unit (140) may map different thresholds to each of the top n countries.

[0076] That is, the threshold optimization unit (140) maps thresholds by country to predict the possibility of voice phishing for countries with a high risk of voice phishing by using the optimal threshold described above among the thresholds that satisfy the input recall rate among the candidate thresholds. In the embodiment of the present invention, the explanation is given by citing an example where the thresholds mapped by country have the same value. However, if there are multiple thresholds that satisfy the input recall rate, the threshold optimization unit (140) may map thresholds with a low false positive rate in order to each country with a high risk of voice phishing.

[0077] If there are no candidate threshold values ​​that exhibit a recall rate higher than the recall rate target value, the threshold optimization unit (140) determines the candidate threshold value with the highest F1-score as the optimal threshold value. The optimal threshold value determined in this way is explained using an example where it is mapped identically to all countries.

[0078] The notification delivery unit (150) checks the possibility that a roaming outgoing call is voice phishing based on the probability value of the possibility that the roaming outgoing call is voice phishing in the phishing number pre-detection model (130) as illustrated in FIG. 3 and the threshold value mapped in the threshold optimization unit (140).

[0079] That is, if the probability value is greater than the threshold value, the notification delivery unit (150) transmits information to the phishing notification service provider server (400) indicating that the roaming outgoing call is voice phishing. Through this, the phishing notification service provider server (400) informs the receiving terminal (20) that there is a high possibility that the roaming outgoing call is voice phishing.

[0080] A method for detecting a voice phishing number in advance using such a phishing number pre-detection system (100) before a call is connected between a calling terminal and a receiving terminal attempting voice phishing via a roaming service overseas is explained with reference to FIG. 4.

[0081] FIG. 4 is a flowchart illustrating the operation method of a phishing number pre-detection system according to an embodiment of the present invention.

[0082] As illustrated in FIG. 4, the phishing number pre-detection system (100) obtains communication service usage history information and roaming service usage history information for each phone number from the linked call service usage history management server (200), and obtains report information from the report information management server (300) (S100).

[0083] The phishing number pre-detection system (100) calculates the initial activity amount for each phone number based on the communication service usage history information collected in step S100 (S101). Here, the initial activity amount refers to the frequency and duration of time during multiple pre-set periods (e.g., 1 day, 3 days, 7 days, etc. based on the subscription date) immediately after the user subscribes to the communication service. This is because, compared to normal users, abnormal users attempt messages or calls frequently and for a long time for voice phishing even in the early stages of service subscription.

[0084] Additionally, the phishing number pre-detection system (100) calculates the number of normal users who used normal communication services and the number of abnormal users who attempted phishing in each country based on the roaming service usage history information collected in step S100, and calculates the voice phishing rate by country (S102). Then, the phishing number pre-detection system (100) calculates the voice phishing risk level by country based on the calculated voice phishing rate by country.

[0085] Additionally, the phishing number pre-detection system (100) aggregates the number of countries visited by each user on a daily basis based on roaming service usage history information. Then, the phishing number pre-detection system (100) calculates the maximum number of daily countries visited and the average number of daily countries visited to extract country change information for each user (S103).

[0086] And, the phishing number pre-detection system (100) calculates daily activity volume for each phone number based on communication service usage history information collected in step S100 (S104).

[0087] The phishing number pre-detection system (100) trains a phishing number pre-detection model with training data including an initial activity amount, country change information, and daily activity amount calculated or extracted through steps S101 to S104 (S105).

[0088] That is, the phishing number pre-detection system (100) identifies characteristics of daily activity patterns when daily activity volume is entered for each phone number. In addition, when initial activity volume is entered, the phishing number pre-detection system (100) identifies the usage pattern of the communication service at the beginning of the subscription for each phone number.

[0089] In addition, when user-specific country change information is entered, the phishing number pre-detection system (100) identifies the country change patterns of normal users and abnormal users.

[0090] As such, when training data is input, the phishing number pre-detection model (130) equipped with the phishing number pre-detection system (100) outputs a probability value indicating the possibility of voice phishing by integrating probability values ​​based on daily activity patterns, initial usage patterns, and country change patterns, rather than a binary classification indicating whether a phone number is a phishing number or not.

[0091] When training the phishing number pre-detection model (130) with such training data, the phishing number pre-detection system (100) can adjust the threshold value for determining whether it is voice phishing based on the country-specific voice phishing risk calculated in step S102 and the constraints entered by the manager (S106).

[0092] When a roaming outgoing call is input, the phishing number pre-detection system (100) extracts a phone number and a country from the roaming outgoing call. Then, the phishing number pre-detection system (100) obtains outgoing terminal activity information corresponding to the extracted phone number, namely communication service usage history information and roaming service usage history information (S107).

[0093] Then, the phishing number pre-detection system (100) preprocesses the acquired outgoing terminal activity information to extract daily activity volume, initial activity volume, and country change information, and inputs this into the phishing number pre-detection model trained in step S105 to output the probability value of voice phishing possibility. Then, the phishing number pre-detection system (100) compares the output probability value with the threshold value mapped by country acquired in step S107 to finally predict the probability of voice phishing (S108).

[0094] If it is predicted to be voice phishing in step S108, the phishing number pre-detection system (100) notifies the receiving terminal that it is voice phishing through the phishing notification service providing server (400) (S109).

[0095] Among the operation procedures of such a phishing number pre-detection system (100), the procedure for calculating the voice phishing rate by country in step S102, the procedure for calculating the daily activity amount and initial activity amount by phone number, the procedure for generating country change information by user in step S103, and the procedure for adjusting the threshold value in step S106 will be explained in detail with reference to FIGS. 5 to 8.

[0096] FIG. 5 is a flowchart of a method for a phishing number pre-detection system to calculate a voice phishing rate by country according to an embodiment of the present invention.

[0097] As illustrated in FIG. 5, the phishing number pre-detection system (100) extracts information consisting of a phone number, a roaming country, and a label (S200).

[0098] That is, the phishing number pre-detection system (100) extracts multiple phone numbers (hereinafter referred to as ‘roaming terminal information’ for convenience of explanation) and the country mapped to each phone number from roaming service usage history information. If the roaming terminal information and the phishing terminal information overlap, the phishing number pre-detection system (100) manages them as phishing terminal information.

[0099] Then, multiple phishing phone numbers are extracted from the reported information, and labels included in the roaming service usage history information are extracted. Here, the explanation is based on the example where the labels are distinguished into voice phishing labels and normal usage labels.

[0100] The phishing number pre-detection system (100) calculates the total number of users and the number of abnormal users who used the roaming service in each country based on the extracted information, and calculates the voice phishing rate by country based on this (S201). At this time, the phishing number pre-detection system (100) calculates the fraud rate by country by the value obtained by dividing the number of abnormal users by the total number of users for each country.

[0101] The phishing number pre-detection system (100) calculates the voice phishing rate for each country and then calculates the phishing risk level for each country based on the voice phishing rate. Then, the phishing number pre-detection system (100) filters only the countries calculated to have a voice phishing rate greater than or equal to a preset threshold rate (S202).

[0102] Then, the phishing number pre-detection system (100) selects the filtered countries as countries suspected of voice phishing (S203) and sets an optimal threshold to be used when determining whether a roaming outgoing call originating from those countries is voice phishing. In the embodiment of the present invention, the candidate threshold with the lowest false positive rate among candidate thresholds higher than the recall rate obtained as a constraint is mapped to each country as the optimal threshold, but it may also be mapped to countries with a high risk of voice phishing in order of the threshold with the lowest false positive rate.

[0103] FIG. 6 is a flowchart of a method for a phishing number pre-detection system to calculate activity levels per phone number according to an embodiment of the present invention.

[0104] As illustrated in FIG. 6, the phishing number pre-detection system (100) calculates daily activity volume by phone number based on the number of outgoing messages, the number of incoming messages, the number of outgoing calls, the number of incoming calls, the outgoing call time, and the incoming call time included in the acquired communication service usage history information (S300). Here, the calculated daily activity volume includes the total daily activity volume, standard deviation, maximum value, and average value, and the phishing number pre-detection system (100) calculates the daily activity volume to determine the relative variability of the activity pattern by phone number.

[0105] And, the phishing number pre-detection system (100) calculates the coefficient of variation by dividing the standard deviation of the daily activity amount calculated for each phone number by the average (S301).

[0106] The phishing number pre-detection system (100) identifies a pattern of rapid increase in activity volume for each phone number based on the calculated fluctuation coefficient (S302). In an embodiment of the present invention, the ratio of the maximum value to the average value in the calculated daily activity volume, the change in activity volume between consecutive days, and the maximum daily increase are analyzed as an example of an activity volume rapid increase pattern.

[0107] In addition, the phishing number pre-detection system (100) analyzes the initial usage pattern for each phone number based on the initial activity amount for each phone number calculated in step S101 of FIG. 4 described above (S303). In an embodiment of the present invention, the initial usage pattern is analyzed by summing the message and call usage for 1 day, 3 days, and 7 days based on the date the user subscribed to the communication service.

[0108] At this time, the phishing number pre-detection system (100) can analyze the ratio of the first country where each user first used the roaming service after subscribing to the communication service, and the period from the subscription to the communication service until the first use of a phone or text message via roaming as an initial usage pattern.

[0109] The phishing number pre-detection system (100) generates daily activity information by country by analyzing the daily average call volume, maximum call volume, maximum daily call time, and number of active days by country for each suspected voice phishing country selected in step S203 of FIG. 5 above (S304).

[0110] Based on the activity surge pattern confirmed in the above-described step S302 or the country-specific daily activity information generated in step S304, the phishing number pre-detection system (100) confirms daily activity pattern characteristics for detecting abnormal patterns (S305). Since there are various methods for the phishing number pre-detection system (100) to confirm daily activity pattern characteristics, the embodiment of the present invention is not limited to any one method.

[0111] FIG. 7 is a flowchart illustrating a method for a phishing number pre-detection system to generate user-specific country change information according to an embodiment of the present invention.

[0112] As illustrated in FIG. 7, the phishing number pre-detection system (100) checks the number of countries visited by the user in one day while using the roaming service and the number of times the user changed countries during the entire roaming service usage period from the roaming service usage history (S400, S401). This is intended to be used to detect voice phishing, as users frequently move between countries out of concern that their location may be exposed.

[0113] The phishing number pre-detection system (100) generates country change information such as the maximum number of countries visited per day, the average number of countries visited per day, the total number of country changes per day, the average country change rate per day, the ratio of countries visited suspected of voice phishing, and the average number of days of stay per country, based on the number of countries visited per day and the number of country changes confirmed in steps S400 and S401 (S402).

[0114] FIG. 8 is a flowchart of a method for a phishing number pre-detection system to adjust a threshold value according to an embodiment of the present invention.

[0115] As illustrated in FIG. 8, the phishing number pre-detection system (100) obtains the probability value predicted by the phishing number pre-detection system (100), the country where the roaming outgoing call occurred, and constraints entered by the system administrator using the outgoing terminal activity information labeled with a voice phishing label, namely the daily activity amount and initial activity amount calculated based on the communication service usage history information and the roaming service usage history information (S500). In an embodiment of the present invention, the recall target value among various constraints is obtained as a constraint, and this is explained as an example.

[0116] The phishing number pre-detection system (100) sets a range of threshold values ​​used to determine whether voice phishing is occurring in the phishing number pre-detection system (100) according to the conditions obtained in step S500 (S501). In an embodiment of the present invention, candidate threshold values ​​are set at intervals of 0.05 within a threshold value range from 0.1 to 0.95, as an example.

[0117] The phishing number pre-detection system (100) calculates performance indicators for each candidate threshold (S502). In the embodiment of the present invention, among various performance indicators, recall, precision, F1-score, and false positive rate are calculated as examples, and the method by which the phishing number pre-detection system (100) calculates performance indicators is a known technique and is not limited to any one method in the embodiment of the present invention.

[0118] The phishing number pre-detection system (100) checks if there is a candidate threshold among the performance indicators calculated in step S502 that is greater than or equal to the recall target value, which is a constraint obtained in step S500 (S503).

[0119] If there is a recall rate calculated to be greater than the recall rate target value, the phishing number pre-detection system (100) identifies the candidate threshold with the lowest false positive rate among the candidate thresholds calculated with that recall rate (S504). Then, the phishing number pre-detection system (100) sets the candidate threshold with the lowest false positive rate as the optimal threshold for each country (S505).

[0120] At this time, when the phishing number pre-detection system (100) identifies multiple candidate threshold values ​​that satisfy the conditions, it can map each of the threshold values ​​with a low false positive rate to the optimal threshold values ​​for countries with a high risk of voice phishing by country, and in the embodiment of the present invention, mapping the same optimal threshold value to each country is described as an example.

[0121] On the other hand, if the recall rate on all calculated performance indicators confirmed in step S503 is lower than the recall rate target value, the phishing number pre-detection system (100) identifies the candidate threshold with the highest F1-score among multiple candidate thresholds (S506). Then, the candidate threshold with the highest calculated F1-score is set as the optimal threshold for determining the possibility of voice phishing in each country.

[0122] In this way, the phishing number pre-detection system (100) may dynamically set a threshold value during the process of training the pre-detection model. Additionally, the trained pre-detection model may dynamically set a threshold value when determining whether a voice phishing call is a voice phishing call before the actual roaming outgoing call is connected to the receiving terminal.

[0123] FIG. 9 is a hardware configuration diagram of a computing device according to an embodiment of the present invention.

[0124] As illustrated in FIG. 9, a phishing number pre-detection system (100) operated by at least one processor may be implemented as a computing system (500) and executes a program containing instructions described to execute the operation of the present invention in the computing system (500). The program may be stored in a computer-readable storage medium and may be distributed.

[0125] The hardware of the computing system (500) may include at least one processor (510), memory (520), storage (530), and communication interface (540), and may be connected via a bus. In addition, hardware such as input devices and output devices may be included. The computing system (500) may be equipped with various software, including an operating system capable of running programs.

[0126] The processor (510) is a device that controls the operation of the computing system (500) and may be a various type of processor that processes instructions included in a program, for example, a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), etc.

[0127] The memory (520) loads the corresponding program so that instructions described for executing the operation of the present invention are processed by the processor (510). The memory (520) may be, for example, ROM (read only memory), RAM (random access memory), etc. The storage (530) stores various data, programs, etc. required to execute the operation of the present invention. The communication interface (540) may be a wired / wireless communication module.

[0128] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention.

Claims

Claim 1 A method of operation of a phishing number pre-detection system operating with at least one processor, comprising: a step of training a model that predicts the probability of voice phishing of a roaming outgoing call using terminal activity information including roaming service usage history information by phone number and phone numbers reported for voice phishing; a step of acquiring a specific roaming outgoing call originating from a specific country through a roaming service; a step of acquiring the probability of voice phishing of the specific roaming outgoing call from the terminal activity information of the specific roaming outgoing call using the model; and a step of determining whether the specific roaming outgoing call is a voice phishing call based on the probability of voice phishing acquired from the terminal activity information of the specific roaming outgoing call and the phishing risk level of the country of origin of the specific roaming outgoing call. Claim 2 A method of operation according to claim 1, wherein the learning step comprises: a step of calculating a voice phishing rate by country based on the roaming service usage history information; and a step of setting a threshold value to determine whether a roaming outgoing call originating from each country is a voice phishing call using the voice phishing risk level by country calculated based on the voice phishing rate by country. Claim 3 In paragraph 2, the step of setting the threshold includes: a step of calculating the voice phishing rate by country based on the roaming service usage history information, based on terminal activity information labeled with voice phishing information by country and terminal activity information labeled with normal information by country; a step of setting a plurality of candidate thresholds within a preset threshold range according to acquired standard conditions; and a step of setting one of the plurality of candidate thresholds as the optimal threshold for each country based on performance indicators calculated for each of the plurality of candidate thresholds. Claim 4 In paragraph 3, the step of setting the optimal threshold comprises the step of the performance indicator including a recall and an F1 score calculated for each candidate threshold, and setting the threshold with the lowest false positive rate among candidate thresholds with calculated recalls satisfying the acquired standard conditions as the optimal threshold for each country. Claim 5 A method of operation according to claim 4, wherein if there is no candidate threshold with a calculated recall rate satisfying the above standard conditions, the method of operation includes the step of setting the candidate threshold with the highest calculated F1-score as the optimal threshold for each country. Claim 6 A method of operation according to claim 1, wherein the step of training the model comprises: the terminal activity information further including communication service usage history information; and the step of generating as training data a daily activity amount for communication service usage by phone number and an initial activity amount for communication service usage used for a preset period after subscribing to the communication service by phone number based on the communication service usage history information. Claim 7 In claim 6, the step of training the model comprises, when the training data is input into the model, identifying a daily activity pattern based on the daily activity amount, identifying an initial usage pattern on pre-set dates after subscribing to the communication service based on the initial activity amount, and training the model to output a probability value for the possibility of voice phishing based on the daily activity pattern and the initial usage pattern. Claim 8 A method of operation according to claim 7, wherein the step of training the model further comprises: a step of obtaining country change information by phone number by confirming the number of daily visited countries and the number of times the country is changed from the roaming service usage history; and a step of inputting the country change information by phone number into the model to train it to identify country change patterns. Claim 9 In paragraph 2, the step of determining whether the specific roaming outgoing call is a voice phishing call includes the step of notifying the receiving terminal that the specific roaming outgoing call is a voice phishing call if the voice phishing probability is greater than or equal to a threshold set for the specific country. Claim 10 A phishing number pre-detection system comprising a processor equipped with a model for detecting whether a roaming outgoing call is a voice phishing call, wherein the processor trains a model that predicts the voice phishing probability of a roaming outgoing call using terminal activity information including roaming service usage history information by phone number and phone numbers reported for voice phishing, and when a specific roaming outgoing call originating from a specific country through a roaming service is acquired, the processor uses the model to acquire the voice phishing probability of the specific roaming outgoing call from the terminal activity information of the specific roaming outgoing call, and determines whether the specific roaming outgoing call is a voice phishing call based on the voice phishing probability acquired from the terminal activity information of the specific roaming outgoing call and the phishing risk level of the country of origin of the specific roaming outgoing call. Claim 11 In claim 10, the processor calculates a voice phishing rate by country based on the roaming service usage history information, and sets a threshold value for determining whether a roaming outgoing call originating from each country is a voice phishing call using the voice phishing risk level by country calculated based on the voice phishing rate by country. Claim 12 In claim 11, the processor calculates the voice phishing rate by country based on the roaming service usage history information, based on terminal activity information labeled with voice phishing information by country and terminal activity information labeled with normal information by country, sets a plurality of candidate threshold values ​​within a preset threshold value range according to the acquired standard conditions, and sets one of the plurality of candidate threshold values ​​as the optimal threshold value by country based on the performance indicator calculated for each of the plurality of candidate threshold values. Claim 13 In claim 12, the processor is a phishing number pre-detection system in which the performance indicators include recall and F1 scores calculated for each candidate threshold, and the threshold with the lowest false positive rate among the candidate thresholds with calculated recalls satisfying the acquired criteria conditions is set as the optimal threshold for each country. Claim 14 In paragraph 13, the processor sets the candidate threshold with the highest calculated F1-score as the optimal threshold for each country if there is no candidate threshold with a calculated recall rate satisfying the above criteria conditions, a phishing number pre-detection system. Claim 15 In claim 14, the processor generates as learning data a daily activity amount for communication service usage by phone number and an initial activity amount for communication service usage used for a preset period after subscribing to the communication service by phone number, based on communication service usage history information further included in the terminal activity information. Claim 16 A phishing number pre-detection system according to claim 15, wherein the processor, when the learning data is input into the model, identifies a daily activity pattern based on the daily activity amount, identifies an initial usage pattern on pre-set dates after subscribing to the communication service based on the initial activity amount, and trains the system to output a probability value of voice phishing possibility based on the daily activity pattern and the initial usage pattern. Claim 17 In claim 16, the processor determines the number of daily visited countries and the number of times the country is changed from the roaming service usage history to obtain country change information by phone number, and inputs the country change information by phone number into the model to train it to identify country change patterns, thereby forming a phishing number pre-detection system. Claim 18 In claim 17, the processor detects that the roaming outgoing call is a voice phishing call and notifies the receiving terminal that it is voice phishing if the probability is greater than or equal to a threshold set for the specific country.

Images