Privacy-preserving voice phishing detection system and method using federated learning

Abstract

The present invention relates to a privacy-preserving voice phishing detection system and method using federated learning. According to the present invention, a voice phishing detection method, in a voice phishing detection system including a central server and a plurality of user terminals, comprises the steps in which: the central server distributes an initialized global model to each user terminal; each user terminal individually trains a local model, which is generated on the basis of parameters of the global model, by using a pre-collected training dataset, and then individually transmits the trained result of the local model to the central server; the central server updates the global model by aggregating the trained results of the local models via the respective user terminals, and then transmits the updated global model to each user terminal; and each user terminal updates the local model on the basis of parameters of the received global model, and by using the updated local model, detects, from text data for voice data when receiving a call, whether voice phishing occurs.

Description

Privacy-Protected Voice Phishing Detection System and Method Using Federated Learning

[0001] The present invention relates to a personal information protection type voice phishing detection system and method using federated learning, and more specifically, to a personal information protection type voice phishing detection system and method using federated learning capable of performing voice phishing detection specialized for the protection of users' personal information using federated learning.

[0002] Voice phishing, a recent social issue, is causing harm to many people by utilizing various communication technologies. According to data provided by the Financial Supervisory Service of Korea, the amount of damage from voice phishing reached 196.5 billion won in 2023, an increase of 51.4 billion won (35.4%) compared to the previous year (145.1 billion won in 2022). Although the number of victims decreased compared to the previous year, the scale of damage and the amount of loss per victim increased significantly. Furthermore, it is evident that damage is occurring through various methods, including phishing impersonating institutions, loans, and family members or acquaintances.

[0003] To address this social problem, various organizations are seeking solutions to prevent voice phishing. Currently, many researchers are conducting studies using diverse methods to prevent damage from voice phishing.

[0004] One representative method among these is voice phishing detection technology using artificial intelligence (AI). By utilizing AI-based voice phishing detection technology, it is possible to detect key keywords or patterns used in voice phishing, and to detect and prevent various voice phishing situations through the analysis of call context characteristics.

[0005] However, while it is necessary to collect and train diverse and large amounts of data to showcase such AI technology, there is a problem in that data is frequently leaked during this process, resulting in a failure to protect data privacy.

[0006] Furthermore, there are significant difficulties in securing datasets containing user privacy for AI training, and data providers often feel hesitant to provide phishing data that includes their personal information. Additionally, the fact that not all collected data is labeled poses significant challenges to the training process.

[0007] Therefore, a new type of AI-based voice phishing detection technology is required that can guarantee data privacy and facilitate the acquisition of datasets.

[0008] The technology forming the background of the present invention is disclosed in Korean Published Patent No. 10-2019-0125856 (published Nov. 07, 2019).

[0009] The present invention aims to provide a personal information protection voice phishing detection system and method using federated learning that can perform voice phishing detection specialized for protecting users' personal information using federated learning.

[0010] The present invention relates to a method for detecting voice phishing with privacy protection using federated learning in a voice phishing detection system comprising a central server and a plurality of user terminals, wherein the central server distributes an initialized global model to each user terminal; each user terminal individually learns a local model generated based on the parameters of the global model using a previously collected training dataset and then individually transmits the learning results of the local model to the central server; the central server aggregates the learning results of the local model through each user terminal to update the global model and then transmits them to each user terminal; and each user terminal updates the local model based on the parameters of the received global model and uses the updated local model to detect whether voice phishing has occurred from text data regarding voice data received during a phone call.

[0011] In addition, the step of detecting whether voice phishing is occurring can detect whether voice phishing is occurring by converting voice data into text data based on speech recognition (STT) technology when the user terminal receives a call and applying the converted text data to the local model.

[0012] In addition, the voice phishing detection method may further include the step of the user terminal outputting an alarm when voice phishing is detected.

[0013] In addition, the voice phishing detection method may further include the step of the user terminal auto-labeling the text data based on the voice phishing detection result and storing it in a DB.

[0014] In addition, the step of auto-labeling and storing in the DB may classify the text data into one of non-phishing, pending, or phishing, and among the data labeled as pending, re-label data determined to have suffered phishing damage on the user terminal for a certain period as phishing, and re-label data determined to have suffered no phishing damage as non-phishing.

[0015] In addition, the voice phishing detection method may further include the step of the user terminal retraining and updating a local model through the labeled data and transmitting the training result of the local model to the central server to update the global model.

[0016] Furthermore, the present invention relates to a voice phishing detection system for personal information protection using federated learning, comprising: a central server that initializes a global model and distributes it to each user terminal, aggregates the learning results of local models through each user terminal to update the global model, and then transmits the results to each user terminal; and a plurality of user terminals that individually learn local models generated based on parameters of the global model distributed from the central server using a previously collected training dataset, and individually transmit the learning results of the local models to the central server. Each of the plurality of user terminals, upon receiving the updated global model from the central server, updates the local model based on the parameters of the received global model and uses the updated local model to detect whether voice phishing has occurred from text data regarding voice data received during a phone call.

[0017] According to the present invention, a phishing detection technology is provided that utilizes a federated learning algorithm to prevent voice phishing damage without leaking data containing a user's personal information to the outside, and to easily secure a sensitive dataset containing personal information.

[0018] In addition, unlike centralized model generation technology, the present invention guarantees privacy by using user information exclusively within the user terminal without leaking it externally, and can improve the performance of the phishing detection model and effectively prevent phishing damage by using an auto-labeling technique to automatically collect and learn from updated data and reflect it in the model.

[0019] Figure 1 is a diagram showing the configuration of a personal information protection voice phishing detection system using federated learning according to an embodiment of the present invention.

[0020] FIG. 2 is a diagram illustrating a personal information protection voice phishing detection method using federated learning according to an embodiment of the present invention.

[0021] FIG. 3 is a diagram showing the overall flow of voice phishing detection according to an embodiment of the present invention.

[0022] FIG. 4 is a diagram exemplarily showing a phishing detection process using federated learning and auto-labeling according to an embodiment of the present invention.

[0023] Then, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification have been given similar reference numerals.

[0024] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0025] The present invention relates to a personal information protection-type voice phishing detection system and method, and proposes a system and method capable of performing voice phishing detection specialized for the protection of users' personal information using federated learning.

[0026] The most significant feature of Federated Learning is that, rather than directly transmitting data from each local device to a central server for training, it is a learning method in which data is trained on each local device, only the resulting trained model is sent to the central server, and then the models collected from each local device are aggregated to create a global model.

[0027] Compared to conventional AI training, which transmits all data to a server leading to increased network traffic and storage costs, this federated learning method significantly reduces communication costs while preventing the external leakage of training data. Therefore, federated learning offers two advantages over existing AI training—guaranteeing data privacy and improving communication efficiency—making it capable of playing a crucial role in voice phishing detection, where user data protection is critical and communication efficiency is vital due to the large number of users.

[0028] Based on the above-described points, the present invention proposes a voice phishing detection technique specialized in protecting users' personal information using federated learning.

[0029] Unlike general centralized model generation technology, the federated learning algorithm proposed in this invention ensures privacy by using user information entirely within the user's device without leaking it externally, and simultaneously prevents phishing damage with a high probability by providing a real-time updated model.

[0030] FIG. 1 is a diagram showing a personal information protection voice phishing detection system using federated learning according to an embodiment of the present invention.

[0031] As shown in FIG. 1, a personal information protection voice phishing detection system using federated learning according to an embodiment of the present invention includes a central server (100) and a plurality of user terminals (200). The user terminals (200) correspond to user devices, i.e., clients. The central server (100) and the plurality of user terminals (200) can be connected to each other via wireless, wired, and wired / wireless networks to transmit and receive information.

[0032] The central server (100) can build a global model for voice phishing detection using a federated learning technique that aggregates local knowledge of each user terminal (200). Through this, the central server (100) can acquire global knowledge without sharing original data and create a global model based thereon. The global model can be delivered to all clients and can be utilized and continuously updated after being retrained using local data.

[0033] The following specifically explains a voice phishing detection method using federated learning.

[0034] FIG. 2 is a diagram illustrating a personal information protection voice phishing detection method using federated learning according to an embodiment of the present invention.

[0035] First, the central server (100) initializes a global model implemented as an artificial intelligence model for voice phishing detection (1), and can distribute and share the initialized global model to each user terminal (200) (2). Accordingly, each user terminal (200) can create a local model based on the parameter information of the global model shared from the central server (100). Here, the initialized global model may refer to a model in which all parameters are initialized, or an initial model that has been pre-trained based on a basic dataset (e.g., a public dataset).

[0036] Next, each user terminal (200) individually trains a local model created based on the parameters of a global model using a previously collected training dataset (3), and individually transmits the training results of the local model to a central server (4). During the training process, the parameters of the local model may be adjusted, and the adjusted parameters may be transmitted to the central server (100). Accordingly, the central server (100) can obtain the training results of each local model as local knowledge.

[0037] The central server (100) can aggregate the learning results of the local model through each user terminal to update the global model (5), and then send the updated global model back to each user terminal (200) (6).

[0038] Then, each user terminal (200) can update the parameters of the local model based on the parameters of the global model that received the update (7). Afterwards, the user terminal (200) can use this updated local model to detect whether voice phishing is occurring from the text data of the voice data received during a phone call (8).

[0039] At this time, when a call is received, the user terminal (200) can convert voice data into text data based on Speech To Text (STT) technology embedded in the user terminal (200) and input the converted text data into the corresponding local model to detect whether it is voice phishing. If it is detected as voice phishing, the user terminal (200) can output an alarm to the user.

[0040] Additionally, the user terminal (200) can auto-label the text data based on the result of detecting whether it is voice phishing and store it in the DB (9).

[0041] Afterwards, as in the previous step 3, the user terminal (200) can retrain and update the local model using the auto-labeled text data, and transmit the training results of the local model to the central server (100) to update the global model. Through the training of the local model using the auto-labeled data, parameter adjustment and updates are performed, and the parameters of the model adjusted at each user terminal (200) can be aggregated into the global model and used to update the global model.

[0042] At this time, in the auto-labeling step (9), the user terminal (200) can classify the text data into one of non-phishing, pending, or phishing. In the case of pending, it corresponds to the case where it is classified as pending judgment.

[0043] Subsequently, the user terminal (200) can re-label data that is determined to have suffered phishing damage on the user terminal for a certain period among the data labeled as pending as phishing, and re-label data that is determined not to have suffered phishing damage as non-phishing. The data re-labeled in this way can also be continuously used for retraining the local model and used for updating the global model.

[0044] In the process of automating data labeling, the client can apply the dataset to a local model for voice phishing detection to perform initial labeling. During this process, ambiguous datasets that are definitively phishing or not phishing (not 0 or 1) can be stored separately in the client's internal DB. Data confirmed to have not caused phishing damage for a certain period can be labeled as not phishing (labeled as 0), and data confirmed to have caused phishing damage can be labeled as phishing (labeled as 1).

[0045] More specifically, the client labels the collected data into three categories: Non-phishing (0), Pending, and Phishing (1). Among these, the dataset labeled as Pending is collected and stored separately. Within a certain period, the client determines whether there is phishing damage. If there is phishing damage, the data is re-labeled as Phishing (1), and if there is no damage, it is re-labeled as Non-phishing (0). The determination of whether damage has occurred may be based on data directly entered into the client or information confirmed through an application or app installed on the client.

[0046] In an embodiment of the present invention, Snorkel can be used for data labeling of collected data. Snorkel helps to automatically label data without manually labeling large amounts of data when training a machine learning model. This helps to process tasks such as text classification and data extraction more efficiently, and in the present invention, it can be used in the process of automatically labeling whether new voice phishing data is phishing or not when such data is collected.

[0047] An embodiment of the present invention can automate data labeling by labeling even datasets that are ambiguous as to whether they are phishing with high accuracy, based on the labeling methodology described above.

[0048] Furthermore, by utilizing the present invention, it is possible to provide a phishing blocking app based on text analysis rather than one based on frequently used phone numbers. Unlike existing apps that cannot block unfamiliar phone numbers, this invention analyzes the content of phone calls to detect and block words and nuances suspected of being phishing. Additionally, it is designed to be user-friendly, allowing even the elderly, who are a primary target group for phishing, to use it easily.

[0049] FIG. 3 is a diagram showing the overall flow of voice phishing detection according to an embodiment of the present invention.

[0050] First, the central server (100) can pre-train a global model for performing voice fitting detection using a public dataset collected in advance, and distribute the global model to a user terminal (200) corresponding to the client side.

[0051] The user terminal (200) can train a local model based on a distributed global model using a previously collected dataset and transmit the training results back to the central server (100) to update the global model. The central server (100) can aggregate the training results of the local models received from each client and share the updated global model back to the user terminal (200).

[0052] From then on, when a call is received at the user terminal (200) and new data is collected, the user terminal (200) inputs the data collected upon receiving the call into its local model to classify whether it is phishing, and if it is classified as phishing, it can output an alert to the user. Additionally, the user terminal (200) can auto-label the data based on the phishing classification result. Furthermore, it can continuously update the model by performing retraining based on the auto-labeling result.

[0053] FIG. 4 is a diagram exemplarily showing a phishing detection process using federated learning and auto-labeling according to an embodiment of the present invention.

[0054] First, the central server (100) initializes the global model (①) and selects clients (200) to request model updates along with the delivery of the global model (②). When each client (200) receives a call, it can convert voice data into text data using voice recognition technology (③). The converted text is used to detect whether it is phishing through a phishing detection algorithm of the local model (④), and if it is determined to be phishing, an alarm can be displayed to the user of the client (200) (⑤). Regardless of whether phishing is detected, all data can be automatically labeled as non-phishing, pending, or phishing and stored in the client (200)'s database (⑥). The model processed by the client (200) in this way is then transmitted back to the central server (100) (⑦) and used for global model updates.

[0055] As described above, according to an embodiment of the present invention, a federated learning algorithm can be used to train data containing a user's personal information without sharing it externally, utilizing the data only within the client. Accordingly, within the client, the user's phone call history can be extracted using STT, and the extracted data can be determined whether or not it is voice phishing. Furthermore, the labeled data required for model training can be processed based on the results of a previous determination of whether or not it is phishing. Additionally, data that is ambiguous regarding whether or not it is phishing can be stored within the client, and labeling can be performed based on the facts of the damage after a certain period of time has passed and the fact of phishing damage is confirmed. Finally, using the labeled data, the performance of the phishing detection model is improved in real time through a federated learning algorithm while protecting the user's personal information.

[0056] According to the present invention, learning is performed internally within the client, thereby preventing the client's personal information from being leaked externally and ensuring privacy. The technique of the present invention is suitable for utilizing sensitive datasets containing personal information, and allows for the rapid application of new data through auto-data labeling using Snorkel. Furthermore, by utilizing this for model training, a model incorporating the latest data can be generated more quickly.

[0057] The present invention has been described with reference to embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the technical spirit of the appended claims.

Claims

1. A privacy-protecting voice phishing detection method using federated learning in a voice phishing detection system including a central server and multiple user terminals, The above central server performs the step of distributing the initialized global model to each user terminal; A step in which each user terminal individually trains a local model generated based on the parameters of the global model using a previously collected training dataset, and then individually transmits the training results of the local model to a central server; The above central server aggregates the training results of the local model through each user terminal, updates the global model, and then transmits them to each user terminal; and A voice phishing detection method comprising the step of each user terminal updating a local model based on parameters of a received global model and using the updated local model to detect whether voice phishing is occurring from text data regarding voice data received during a phone call.

2. In Claim 1, The step of detecting whether voice phishing is occurring, above, is A voice phishing detection method in which, upon receiving a phone call, the user terminal converts voice data into text data based on speech recognition (STT) technology and applies the converted text data to the local model to detect whether voice phishing is occurring.

3. In Claim 1, A voice phishing detection method further comprising the step of the user terminal outputting an alarm when the voice phishing is detected.

4. In Claim 1, A voice phishing detection method further comprising the step of the user terminal auto-labeling the text data based on the voice phishing detection result and storing it in a DB.

5. In Claim 4, The step of auto-labeling and storing in the DB above is, Classify the above text data into one of non-phishing, pending, or phishing, and A voice phishing detection method that re-labels data identified as having suffered phishing damage on a user terminal for a certain period among the data labeled as holding as phishing, and re-labels data identified as having suffered no phishing damage as non-phishing.

6. In Claim 4, A voice phishing detection method further comprising the step of the user terminal retraining and updating a local model through the labeled data and transmitting the training result of the local model to the central server to update the global model.

7. In a privacy-protected voice phishing detection system using federated learning, A central server that initializes a global model and distributes it to each user terminal, aggregates the training results of local models through each user terminal, updates the global model, and then transmits it to each user terminal; and It includes a plurality of user terminals that individually train a local model created based on the parameters of a global model distributed from the central server using a previously collected training dataset and individually transmit the training results of the local model to the central server. Each of the above plurality of user terminals is, A voice phishing detection system that, upon receiving the above-mentioned updated global model from a central server, updates the above-mentioned local model based on the parameters of the received global model and uses the updated local model to detect whether voice phishing is occurring from text data regarding voice data received during a phone call.

Images