Chatbot for preventing online fraud

Abstract

In some embodiments, a fraud prevention system uses a chatbot agent to provide input to a threat analyzer. The chatbot agent dialogues with a user to identify a security need, guides the user to provide relevant data (e.g., content of an email, screenshot of a social media conversation), identifies a target object for analysis, and indicates the target object to the threat analyzer. The threat analyzer then applies a series of tests to determine whether the target object indicates an online fraud, such as a phishing attempt. The threat analyzer returns a determination of the analysis to the chatbot agent for communication to the user.

Description

Background Technology

[0001] This invention relates to computer security, and more specifically, to the prevention of online fraud such as phishing.

[0002] Online fraud (especially in the form of phishing and identity theft) poses a growing threat to internet users worldwide. Sensitive personal information (such as usernames, IDs, passwords, social security and medical records, bank and credit card details) obtained through fraud by international criminal networks operating on the internet is used to withdraw private funds and / or further sold to third parties. In addition to direct financial losses to individuals, online fraud causes a range of undesirable side effects, such as increased corporate security costs, higher retail prices and banking fees, decreased stock values, lower wages, and reduced tax revenue.

[0003] The explosive growth of mobile computing and online services has fueled online fraud, with millions of devices, such as smartphones and tablets, constantly connecting to the internet and acting as potential targets. In a typical example of phishing, users receive fraudulent communications disguised as legitimate messages from service providers such as banks, telephone companies, and online retailers. These messages may report fabricated problems with the user's account or recent orders and invite the user to contact the service provider via a link included in the message. This link may lead to a fake interface (e.g., a webpage) used by cybercriminals to steal sensitive data such as login credentials and credit card numbers. Accessing such links can further expose users to the risk of installing malware.

[0004] Various security software programs can be used to detect fraudulent websites and / or phishing messages. However, using such software may require the installation of a local security agent on the user's computing device and may further require a certain level of knowledge about online communications, computer security, and / or certain types of online threats, which is expected to exceed the knowledge level of the average user. Furthermore, the methods used by cybercriminals to trick users into revealing sensitive information are constantly evolving. Therefore, there is ongoing interest in developing robust and user-friendly methods to combat online fraud. Summary of the Invention

[0005] According to one aspect, a computer system includes at least one hardware processor configured to execute a chatbot agent and a threat analyzer coupled to the chatbot agent. The chatbot agent is configured to, in response to receiving a natural language (NL) message from a user, formulate a language model cue based on the NL message and transmit the language model cue to a language model (LM), the LM being configured to determine an LM response including a reply to the NL message. The chatbot agent is further configured to identify a target object for fraud analysis based on the LM response and transmit an indicator of the target object to the threat analyzer. The threat analyzer is configured to perform fraud analysis on the target object to determine whether the target object indicates fraud, and output the result of the fraud analysis to the chatbot agent for transmission to the user.

[0006] According to another aspect, a computer implementation method includes employing at least one hardware processor of a computer system to execute a chatbot agent and a threat analyzer coupled to the chatbot agent. Executing the chatbot agent includes: in response to receiving an NL message from a user, formulating a LM prompt based on the NL message, and transmitting the LM prompt to an LM configured to determine an LM response including a reply to the NL message. Executing the chatbot agent further includes identifying a target object for fraud analysis based on the LM response, and transmitting an indicator of the target object to the threat analyzer. Executing the threat analyzer includes performing fraud analysis on the target object to determine whether the target object indicates fraud, and outputting the result of the fraud analysis to the chatbot agent for transmission to the user.

[0007] According to another aspect, a non-transitory computer-readable media storage instruction, when executed by at least one hardware processor of a computer system, causes the computer system to form a chatbot agent and a threat analyzer coupled to the chatbot agent. The chatbot agent is configured to, in response to receiving an NL message from a user, formulate a LM prompt based on the NL message and transmit the LM prompt to an LM, the LM being configured to determine an LM response including a reply to the NL message. The chatbot agent is further configured to identify a target object for fraud analysis based on the LM response and transmit an indicator of the target object to the threat analyzer. The threat analyzer is configured to perform fraud analysis on the target object to determine whether the target object indicates fraud, and output the result of the fraud analysis to the chatbot agent for transmission to the user. Attached Figure Description

[0008] The foregoing aspects and advantages of the invention will be better understood after reading the following detailed description and referring to the accompanying drawings, wherein:

[0009] Figure 1 Exemplary components of a system for preventing online fraud according to some embodiments of the present invention are shown.

[0010] Figure 2 This describes exemplary dialog interfaces according to some embodiments of the present invention.

[0011] Figure 3-A An exemplary sequence of steps performed by a chatbot agent according to some embodiments of the present invention is shown.

[0012] Figure 3-B Another exemplary sequence of steps performed by a chatbot agent according to some embodiments of the present invention is shown.

[0013] Figure 4 Exemplary language model (LM) prompts are shown according to some embodiments of the present invention.

[0014] Figure 5 Exemplary LM responses are shown according to some embodiments of the present invention.

[0015] Figure 6 Exemplary components of a threat analyzer according to some embodiments of the present invention are described.

[0016] Figure 7 An exemplary sequence of steps performed by a threat analyzer according to some embodiments of the present invention is shown.

[0017] Figure 8 Exemplary hardware configurations of computer systems are shown that are programmed to perform some of the methods described in this article. Detailed Implementation

[0018] In the following description, it should be understood that the connections between all descriptions of structures can be direct operational connections or indirect operational connections through intermediate structures. A group of elements comprises one or more elements. Any description of an element should be understood to refer to at least one element. Multiple elements comprise at least two elements. Any use of 'or' implies non-exclusivity. Unless otherwise required, any described method steps do not necessarily need to be performed in a specific, described order. A first element derived from a second element (e.g., data) encompasses a first element equal to the second element, as well as a first element produced by processing the second element and optionally other data. Making a determination or decision based on parameters encompasses making a determination or decision based on parameters and optionally other data. Unless otherwise specified, some quantity / data indicators may be the quantity / data itself, or may be indicators different from the quantity / data itself. A computer program is a sequence of processor instructions that performs a task. The computer program described in some embodiments of the invention may be a standalone software entity or a sub-entity of another computer program (e.g., a subroutine, a library). A database or knowledge base as used herein refers to any organized, searchable collection of data. Computer-readable media encompasses non-transitory media such as magnetic, optical, and semiconductor storage media (e.g., hard disk drives, optical disks, flash memory, DRAM), as well as communication links such as conductive cables and fiber optic links. According to some embodiments, the present invention particularly provides a computer system comprising hardware (e.g., one or more processors) programmed to perform the methods described herein, and a computer-readable medium encoding instructions to perform the methods described herein.

[0019] The following description illustrates embodiments of the invention by way of example and not necessarily by way of limitation.

[0020] Figure 1 Exemplary components of a fraud prevention system 10 according to some embodiments of the present invention are shown. System 10 includes a chatbot agent 20 and a threat analyzer 30 communicatively coupled to the chatbot agent 20. Components of the fraud prevention system 10 may be embodied as computer programs executing on a set of hardware processors of a computer system, such as computer programs executing on a server computer system that performs fraud prevention transactions with multiple client front-end devices, as detailed below. However, those skilled in the art will appreciate that some or all of the functionality of system 10 may also be implemented in dedicated hardware, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs), or in a combination of hardware and software.

[0021] In some embodiments, chatbot agent 20 includes an artificial intelligence (AI) system configured to conduct conversations (i.e., message exchanges) with the user in a natural language (NL), such as English or Chinese. Agent 20 may be further configured to collaborate with threat analyzer 30 to determine whether the user faces a computer security threat, such as online fraud, malware, etc., based on the content of the corresponding conversation. Figure 2 In the exemplary scenario described, a user can ask chatbot agent 20 whether a recently received message might be fraudulent. Chatbot agent 20 can then request a copy of the message, transmit the suspicious message to threat analyzer 30 for analysis, and relay the analysis results back to the user.

[0022] Some embodiments of the chatbot agent 20 are further configured to provide users with various other information, such as answering general questions and providing advice on various computer security topics such as malware, spam, communication privacy, protecting online payments, parental controls, etc. The chatbot 20 may further recommend that users purchase computer security software, manage users' subscriptions to various computer security services, answer billing questions, or in any other way act as a user-friendly interface between users and computer security service providers.

[0023] To perform natural language dialogue with human users, some embodiments of the chatbot agent 20 rely on a language model (LM) 40 to generate synthetic sentences, questions, and / or answers. LM 40 includes implementations of computational models of natural language, such as a set of artificial neural networks pre-trained on a text corpus formulated in the corresponding natural language. Exemplary LMs include probabilistic n-gram models, language models implemented using recurrent neural networks, and large language models (LLMs) implemented using generative pre-trained transformers (GPTs). In some embodiments, LM 40 is generative because it is configured to take a sequence of words (e.g., a sentence or a question) as input and, in response, produce a plausible continuation of the input sequence of words (e.g., another sentence or a reply). The structural and operational details of LM 40 are beyond the scope of this invention. LM 40 can be implemented using any method known in the field of artificial intelligence. In some embodiments, LM 40 includes, for example, ChatGPT from OpenAI, Inc. and Bard from Google, Inc. The LLM used by public and / or commercial chatbot services, etc. In such embodiments, chatbot agent 20 may access LM 40 via a Hypertext Transfer Protocol (HTTP) request addressing a remote server providing the corresponding language modeling service.

[0024] In some embodiments, threat analyzer 30 includes a set of modules configured to analyze target objects, such as electronic communications (instant messaging, email, etc.), sound files (e.g., recorded voicemail), images (e.g., screenshots), or documents (e.g., web pages), to determine whether the target object indicates a computer security threat. The following description focuses on fraud analysis, which aims to determine whether a target object indicates an online fraud, such as phishing. However, those skilled in the art will recognize that the disclosed systems and methods are suitable for other types of threats, such as malware, intrusions, etc. Threat analyzer 30 may implement a range of analysis methods, detailed below. When performing fraud analysis, analyzer 30 may rely on a repository of computer security knowledge, generally referred to herein as security knowledge base 54. Knowledge base 54 may contain, for example, a blacklist of Internet domains or network addresses involved in online fraud and a list of keywords with characteristics of phishing.

[0025] In some embodiments, the fraud prevention system 10 interacts with human users via a user interface displayed on the front-end device 12, such as... Figure 1 The exemplary front-end device 12 includes personal computers, laptop computers, tablet computers, mobile telecommunications devices (e.g., smartphones), media players, TVs, game consoles, home appliances (e.g., refrigerators, thermostats, smart heating and / or lighting systems), and wearable devices (e.g., smartwatches, sports and fitness equipment), etc. A basic user interface according to some embodiments includes a communication interface that enables a user to interact with the fraud prevention system 10 in natural language, such as by asking questions, transmitting and / or requesting various data, and / or receiving the results of various computer security tasks. The communication interface may be integrated with other computer security functionalities, such as with a dashboard for configuring and displaying various security settings and / or for displaying the current security status of the front-end device 12. In exemplary embodiments, the user interface may display indicators such as whether device 12 contains malware, whether device 12 is currently connected to a Virtual Private Network (VPN), etc. Other exemplary content displayed by the respective user interface may include indicators of the status and / or details of the user account / Service Level Agreement (SLA) / subscription for using the fraud prevention system 10 or other security software. The user interface described herein can be organized in any way known in the field, such as by including various visual elements (e.g., dials, gauges, charts), each indicating the value of the current security setting and / or the value of the monitored quantity. Some visual aspects of the user interface can be customized, such as the color scheme, location, and content of various screen areas.

[0026] In other exemplary embodiments, users can interact with the fraud prevention system 10 via a communication interface of an online messaging application running on front-end device 12. Online messaging as used herein encompasses peer-to-peer messaging as well as messaging via public chat rooms, forums, social media sites, etc. Examples of online messaging include the exchange of Short Message Service (SMS) messages, sequences of email messages, and sequences of messages exchanged via instant messaging applications such as WhatsApp Messenger®, Telegram®, WeChat®, and Facebook® Messenger®. Other exemplary online messaging includes content from Facebook® Walls, chats on online forums such as Reddit® and Discord®, and a set of comments on blog posts. Exemplary online messaging applications according to embodiments of the invention include client-side examples of mobile applications such as WhatsApp®, Facebook®, Instagram®, Snapchat®, etc., and server-side software performing the corresponding messaging operations. Other examples of online messaging applications include examples of email clients and internet browsers.

[0027] For clarity, this description will focus on a communication interface that enables users to perform natural language conversations by typing. In other words, the exchange between the user and the fraud prevention system 10 described herein is primarily in text form. However, those skilled in the art will appreciate that this is not intended to be limiting. The described systems and methods can be adapted to handle any combination of audio messages (spoken conversations), video messages, or carrier media. In such embodiments, the chatbot agent 20 may be configured to directly process the corresponding type of input, or alternatively, to convert the user-provided input of that type into text before applying some of the methods described herein. Furthermore, in some embodiments, the communication interface described herein allows users to attach various types of media files (e.g., images / screenshots, audio files such as recorded voice messages, etc.) to text messages.

[0028] Figure 1The exemplary embodiments described herein employ a messaging system 14 to transmit natural language messages 16 between a front-end device 12 and a fraud prevention system 10. The messaging system 14 generally refers to any messaging and electronic communication functionality beyond the scope of this invention. For example, the messaging system 14 may represent hardware and / or software implementing conventional electronic communication services such as email services, short message services (SMS), and instant messaging services such as WhatsApp®, iMessage®, and Microsoft Teams®. The messaging system 14 may, for example, aggregate messages from multiple front-end devices, route such messages, and / or selectively deliver such messages to their intended destinations. In some exemplary embodiments, the front-end device 12 may invoke a local example of a conventional online messaging application to enable a user to send and / or receive NL messages 16 from device 12. Subsequently, the chatbot agent 20 may transmit and / or receive corresponding messages via application programming interface (API) calls to remote network services exposed by the messaging system 14. The message itself can be routed by the messaging system 14 via a third-party server, while traveling between the front-end device 12 and (a number of) computers implementing the fraud prevention system 10.

[0029] Figure 2 The exemplary communication interface displayed on the front-end device 12, such as that provided by WhatsApp, is shown. The interface exposed by a typical online messaging application, such as a client. The illustrated communication interface shows a conversation 18 comprising a sequence of natural language (NL) messages 16a to 16b exchanged between a user and chatbot agent 20. Messages are typically ordered according to transmission time. In some embodiments, a single chatbot agent 20 may perform conversations with multiple users. Conversation 18 may contain messages exchanged between more than two users / parties, as in the case of a group chat. Figure 2 Further, an exemplary dialogue context 17 is shown, which is defined herein as a set of messages preceding and / or following the selected message (e.g., message 16a in the illustrated example). In some embodiments, context 17 is intentionally configured to contain only messages from the selected party / user.

[0030] NL messages 16a to 16b may vary in format depending on the corresponding messaging platform, protocol, and / or application, but generally, messages 16a to 16b may include encoding of text and / or encoding of media files (e.g., images, movies, sound, etc.). The text portion may include text written in natural language, as well as other alphanumeric and / or special characters (e.g., emojis). The encoding of messages 16a to 16b may further include identifiers of the sender and receiver of the corresponding message and a timestamp indicating the transmission time of the corresponding message. This metadata enables the chatbot agent 20 to associate each message with an ongoing conversation and maintain the conversational context of each conversation, for example, by sequentially arranging messages according to their corresponding timestamps.

[0031] Some embodiments of agent 20 can maintain multiple concurrent conversations with various users on various topics. Internally, agent 20 can represent each conversation as a separate data structure (e.g., an object with multiple data fields) identified by a unique conversation ID. The conversation object can be defined according to any data standard known in the field and may contain a user_ID identifying an individual user of front-end device 12 and / or the corresponding device. The conversation object may further contain multiple message indicators, each corresponding to an individual message exchanged within the corresponding conversation. Each individual message indicator may further contain an identifier of the sender and / or receiver, the text content of the corresponding message, and a timestamp indicating the time when the corresponding message was sent and / or received. In alternative embodiments, the conversation object may include a concatenation of the text content of all messages in the corresponding conversation, with individual messages arranged in transmission order according to their respective timestamps. The message indicators may further contain a set of media indicators, such as a copy of an image / video / audio file appended to the corresponding message, or the network address / URL to which the corresponding media file is located. Some embodiments keep the conversation active as long as the message count of the conversation does not exceed a predetermined value, as long as the time elapsed since the first message of the conversation does not exceed a predetermined time threshold, and / or as long as the time elapsed since the latest message of the conversation does not exceed another predetermined time threshold.

[0032] Figure 3-A Step B illustrates an exemplary sequence of steps performed by chatbot agent 20 according to some embodiments of the present invention. In the sequence of steps 202 to 204, agent 20 may listen for input. Figure 1 The description states that agent 20 can receive input from any of the user, LM model 40, and threat analyzer 30, each such input being associated with a corresponding dialogue. Therefore, step 206 can identify the corresponding dialogue based on the corresponding input, for example, based on metadata such as the user's identifier, timestamp, or dialogue ID.

[0033] If the received input includes a message from the user (step 208 returns yes), then in step 210, some embodiments run a set of checks to determine whether the received message indicates a malicious attempt to manipulate or otherwise disrupt the fraud prevention process. Such checks can be configured to detect various types of attacks commonly referred to in the art as adversarial attacks against generative AI systems, including carefully crafted inputs to a language model to intentionally cause the model to malfunction, such as producing no output or producing malformed, incorrect, or anomalous output. Exemplary adversarial attacks include cue bypass, where input to the chatbot is designed to oppose or cancel previous input (e.g., instructing the chatbot to ignore all previous instructions). Other exemplary adversarial attacks design input to include various special characters, misspelled words, or snippets of computer code, etc., in an effort to disable the corresponding language model or chatbot. Step 208 may include any attack detection methods known in the art.

[0034] If an attack is detected (step 210 returns "Yes"), then in step 222, some embodiments may terminate the corresponding dialogue. Some embodiments may additionally transmit a warning to the user against such behavior and mark the corresponding user as a potential attacker. Subsequent dialogue with the corresponding user may be restricted or may use a special protocol.

[0035] Conversely, when no indication of an attack is detected, step 212 may update the history / context of the corresponding dialogue, for example, by updating the content of the dialogue object specifying the corresponding dialogue to include an indicator of the current user message. In a further step 214, the chatbot agent 20 may formulate an LM prompt 24 based on the current user message. Figure 1 In this paper, LM hints represent at least a portion of the input to the natural language model. Figure 4 Exemplary LM prompts 24 are shown according to some embodiments of the present invention. Prompt 24 may contain a set of LM instructions 24a, 24c specifying how the LM model 40 should process the content of the dialogue and formulate questions and / or answers for the user. For example, such instructions may set various parameters of the LM 40, such as the style of the dialogue, the personality / avatar / name of the corresponding chatbot, the format of dates and times, etc. Instructions 24a, 24c may further indicate the preferred output format of the LM 40, such as as text formulated in natural language, as a set of attribute-value pairs, as a data object with a specific set of fields, etc.

[0036] Those skilled in the field will know, Figure 4The content, format, and encoding of LM prompt 24 described herein are not intended to be restrictive. Exemplary instruction sections 24a and 24c include text written in natural language. Depending on the actual implementation of LM 40, such instructions may alternatively or additionally include computer code, bytecode, etc., or may be available in, for example, a version of Extensible Markup Language (XML) or JavaScript. The markup language used is JSON (Simplified Chinese Object Notation). The fact that directive sections 24a and 24c are scattered among other types of content is not intended to be restrictive.

[0037] LM hint 24 may further include context segment 24b, which includes, for example, references to the above regarding... Figure 2 The encoding of at least a portion of the dialogue in the described dialogue context 17. Figure 4 In this instance, context section 24b is encoded as a set of attribute-value pairs specified in a certain version of JSON, but such implementation details are not intended to be restrictive.

[0038] In some embodiments, specify LM prompt 24 ( Figure 3-A Step 214) includes, for example, determining the dialogue context of the current dialogue based on the content of the dialogue object associated with the dialogue identified in step 206. The dialogue context may consist of the content of a set of recent messages exchanged between the chatbot agent 20 and (a number of) corresponding users. Developing LM prompts 24 may further include appending the latest message received from the user to the identified dialogue context. Developing LM prompts 24 may further include constructing a context segment, for example, by concatenating messages of the dialogue context, adding various message metadata / attributes (e.g., source, timestamp, etc. for each message), and adding a set of LM instructions (see [link to relevant documentation]). Figure 4 (Item 24b in the table). In a further step 216, the chatbot agent 20 may transmit the LM prompt 24 to the language model 40. When the LM 40 is implemented as an online service, step 216 may include issuing a set of API calls to a remote server providing the corresponding LM service, for example, in the form of an HTTP request, based on the LM prompt 24. The chatbot agent 20 may then return to listening for input (step 202).

[0039] In some embodiments, LM model 40 is configured to return LM response 26 in response to LM prompt 24. Figure 1In an instance of a generative language model, LM response 26 may include a reasonable continuation of the corresponding dialogue, such as a response to a user-submitted question. Some embodiments of LM model 40 may be further configured to perform various predefined functions, such as summarizing the dialogue, classifying the dialogue into one of several predetermined categories, determining the mood of the dialogue, searching for various keywords or lexical terms in the dialogue, converting a portion of the dialogue into a database query (e.g., Structured Query Language - SQL), and extracting structured data from the dialogue, etc. In some embodiments, each such function may be invoked by including a specific command / set of instructions in LM prompt 24.

[0040] In some embodiments, the LM model 40 is configured to identify target objects for fraud analysis based on the content of the conversation. For example, the chatbot agent 20 may intentionally craft LM prompts 24 to cause LM 40 to return indicators for target objects for fraud analysis. Exemplary target objects include, in particular, a piece of text, media files (images, audio, video, etc.), a piece of computer code (e.g., executable files, scripts, etc.), and indicators of the location of Internet resources (e.g., Uniform Resource Identifier-URI, Uniform Resource Locator-URL, etc.). In an exemplary use case scenario, the chatbot agent 20 may collaborate with the LM model 40 to request WhatsApp messages that the user has indicated as suspicious. A screenshot of the message exchange. The LM model 40 can then automatically identify the corresponding image file as a target object to be analyzed against fraud indicators. Some embodiments may combine content from multiple messages into a single aggregate target object. In one such example, the target object may include a concatenation of multiple text messages. In response to identifying the target object, the LM model 40 may include an indicator of the corresponding target object in the LM response. Exemplary target indicators include the target object itself (e.g., target text, target image) and an indicator of the target object's location (e.g., file path, network address, URL).

[0041] Figure 5 An exemplary LM response 26 is shown according to some embodiments of the invention. LM response 26 includes a target indicator 28 that indicates the target text that includes the content of a message received from a user. In the illustrated example, LM response 26 includes a set of attribute-value pairs specified in a certain version of JSON, wherein the target indicator 28 includes the value of a “parameter” attribute. However, those skilled in the art will understand that many alternative ways of indicating a target object may exist, and this aspect is not intended to be limiting.

[0042] If the input detected by chatbot agent 20 includes a reply from LM 40 ( Figure 3-A If step 220 returns "yes", then agent 20 can proceed to step 230. Figure 3-B This includes validating the corresponding LM response. This validation can be performed to ensure, for example, that LM response 26 has correct or expected syntax, format, etc. In one example, step 230 may check whether LM response 26 includes a valid JSON expression. If it does not, some embodiments may attempt to force LM 40 to produce valid output. To do this, some embodiments may formulate a new LM hint (see step 214 above), which may contain the currently invalid response and may contain new formatting instructions, and feed the new LM hint to LM model 40.

[0043] When LM response 26 is valid (step 232 returns yes), in step 234, some embodiments may determine whether LM response 26 contains an indicator of an action to be performed by chatbot agent 20. Depending on the implementation, chatbot agent 20 may assist the user in performing various tasks other than fraud prevention, such as account management, billing inquiries, license acquisition, etc. Some embodiments may rely on LM model 40 to identify the topic of the conversation and / or the user's needs or demands based on the corresponding dialogue. In one instance, LM model 40 may be trained to categorize the current dialogue into multiple predetermined categories based on its content. Exemplary categories may include "scan," "information," "sales," and "billing," etc. Each such category may be associated with an action to be performed by agent 20. For example, a "scan" category assignment may cause chatbot agent 20 to initiate fraud detection analysis. Meanwhile, an "information" category assignment may cause agent 20 to provide advice on fraud prevention, describe popular tactics used by online fraudsters, etc. The category assignment of the dialogue may be communicated to chatbot agent 20 via the category and / or action indicators included in LM response 26. Figure 5 In one such instance described herein, the action indicator is provided in the form of an attribute value “scan.” Those skilled in the art will understand that many alternative ways exist to include category / action indicators within LM Reply 26, and the illustrated instance is not intended to be limiting.

[0044] If step 234 determines that LM reply 26 does not contain an action indicator (e.g., when LM reply 26 only includes a message for the user), then in step 242, agent 20 may update the dialogue context of the current dialogue, for example, by adding the current reply to the dialogue object associated with the corresponding dialogue. A further sequence of steps 244 to 246 may formulate an NL message based on LM reply 26 and transmit the corresponding message to messaging system 14 for delivery to the corresponding user. Chatbot agent 20 may then return to listening for input.

[0045] If step 234 returns yes, then a further step 236 determines whether the action indicated by LM response 26 includes threat analysis, and if so, then in step 238, agent 20 may identify a target object based on LM response 26. In some embodiments, the target object is identified by a target indicator included in LM response 26 (see, for example...). Figure 5 The target indicator 28 in the context indicates the target text to be analyzed against indicators for online fraud. In an alternative embodiment, the LM response 26 contains a summary / digest of the current conversation, and the agent 20 is configured to determine the target indicator 28 based on the corresponding summary. For example, the conversation summary / digest may indicate whether the user has submitted any images, and if so, the chatbot agent 20 may identify the corresponding image as a target and formulate the target indicator 28 accordingly. A further step 238 transmits the target indicator 28 to the threat analyzer 30 for analysis. The operation of the threat analyzer is detailed below.

[0046] If step 236 returns no, i.e., when the action indicated by LM response 26 is different from fraud analysis, then in the sequence of steps 248 to 250, chatbot agent 20 may perform (a number of) corresponding actions and determine the result of (a number of) corresponding actions. For example, LM response 26 may indicate that the user wants to renew his / her subscription to the fraud prevention service. Then, steps 248 to 250 may include searching the database of clients and / or subscriptions, retrieving the entry corresponding to the user, and identifying the offer applicable to the corresponding user. In another instance where the user is looking for best practices or advice on protecting himself / herself from online fraud, steps 248 to 250 may include directing the user to a web-based anti-fraud information resource. To perform such actions, agent 20 may use any methods known in the art, such as formulating a database query and submitting it to an appropriate server, parsing the response to extract various data, etc. Such details are beyond the scope of the invention.

[0047] In the above examples, the action results may include the amount payable to renew the current subscription and the URL of the anti-fraud webpage. Some embodiments may employ LM model 40 to formulate and provide answers to the user within the context of the corresponding dialogue. For example, some embodiments may formulate new LM prompts based on the results of step 248 and submit the corresponding prompts to LM model 40 (see step 214 above).

[0048] Figure 6 Exemplary components of a threat analyzer module according to some embodiments of the present invention are shown. Threat analyzer 30 is configured to receive a target indicator 22 from chatbot agent 20, determine whether the target object identified by the target indicator 22 indicates online fraud, and output the determined results to chatbot agent 20 in the form of an analysis report 52 (see [link to analysis report]). Figure 1 Analysis report 52 may include a determination of whether the corresponding target indicates online fraud and / or indicators of the fraud type (e.g., phishing, fraudulent investment, unexpected prize winnings, etc.). Some embodiments further include a summary of the findings of the fraud analysis and / or a set of explanations, recommendations, or suggestions regarding handling the corresponding incident or, more generally, handling the corresponding type of online fraud.

[0049] In some embodiments, the threat analyzer 30 includes a target parser 32 communicatively coupled to the detector 36. The parser 32 is configured to extract a set of features 33 characterizing a target object and transmit the features 33 to the detector 36. Features 33 generally represent any characteristic or attribute used to determine whether a given object indicates a target object for online fraud. Exemplary features 33 include text (e.g., the text content of an SMS message received by a user), indicators of whether the target text contains specific keywords, indicators of whether the message contains hyperlinks, and layout indicators that quantify the visual organization of a webpage, etc.

[0050] Parser 32 may further include a set of media converters configured to extract text features from various media files, such as image and audio files. Such converters may use any methods known in the art, such as optical character recognition (OCR) and speech recognition techniques. In one such instance, where a user provides a screenshot of a message exchange, parser 32 may apply OCR to the corresponding image file to extract the text content of the corresponding message. Recorded audio and / or video messages may similarly be converted to text using speech recognition. In yet another instance, where the target object includes a matrix barcode or QR code, parser 32 may employ a decoder to extract the text content of the corresponding barcode / QR code. Parser 32 may then extract target features 33 from the corresponding text.

[0051] Detector 36 can implement a series of methods to determine whether a target object characterized by feature 33 indicates online fraud. Exemplary fraud detection methods based on text analysis include keyword detection (the presence of certain keywords can indicate certain types of fraud). Other detection methods can analyze URIs / URLs, such as hyperlinks contained in text messages. For example, some embodiments can check a target domain against a blacklist of domains known to be associated with online fraud. Other exemplary techniques include performing a WHOIS lookup to determine domain registration data associated with the target Internet domain and checking for fraud indication patterns in the corresponding registration data. Still other exemplary methods determine whether a webpage located at a target URL is fraudulent based on the visual layout of the corresponding webpage. For example, some embodiments can check whether the corresponding page displays a login form or various other features typically encountered in online banking or e-commerce interfaces.

[0052] In some embodiments, detector 36 includes multiple filters, each implementing a different fraud detection method or criterion. For example, when analyzing a target webpage, one filter might look for specific keywords, while another might analyze the visual layout. Each such filter might determine an individual fraud indication score. These individual scores can then be aggregated into a combined score based on the observation that some target features themselves do not indicate fraud but do when they co-occur with other features. The contribution of each individual score to the combined score can vary depending on the reliability of the respective filter in detecting fraud. Detector 36 can then determine whether the target object indicates fraud by comparing the combined score to a predetermined threshold.

[0053] In another exemplary embodiment, detector 36 may represent each target object as a vector in a multidimensional abstract space, where each coordinate is determined based on the output of an individual filter and / or based on an individual target feature 33. Detector 36 may then determine whether a given target object indicates fraud based on the position of the corresponding vector in the abstract space. In some embodiments, selected areas of this abstract representation space are associated with various types of online fraud.

[0054] In yet another exemplary embodiment, detector 36 includes an artificial intelligence system (e.g., a set of artificial neural networks pre-trained on a corpus of legitimate objects and fraud indication objects) configured to receive an input vector of target features 33 and assign a corresponding target object to one of a plurality of object classes / categories based on the corresponding feature vector. For example, detector 36 may determine a category indicator 35 that identifies a selected category of an object. Object categories may include legitimate categories and a set of fraud indication categories, each of which corresponds to a different type of fraud. Exemplary fraud categories may include frauds related to payments, express delivery, telecommunications, travel, stocks, cryptocurrencies, real estate, jobs, etc.

[0055] When performing fraud detection operations, detector 36 may rely on a security knowledge base 54b that stores configuration parameter values ​​for various filters, value ranges representing each object category, thresholds, and score weights. Figure 6 ).

[0056] In some embodiments, the threat analyzer 30 includes a dedicated natural language processing (NLP) module 34 configured to analyze text objects to determine summaries, intents, sentiments, or any other textual features for detecting online fraud. The NLP module 34 may include an AI system, such as a set of neural networks. Exemplary neural architectures include convolutional neural networks, recurrent neural networks, and transformer neural networks implementing generative language models as described above. Structural and functional details of the NLP module 34 are beyond the scope of this invention. Embodiments may use any architecture and training strategy known in the fields of machine learning and NLP. Instead of having a native NLP module as described, alternative embodiments may employ an LM 40 (… Figure 1 To achieve similar functionality, NLP module 34 can interface with security knowledge base 54a, which can store various configuration parameter values ​​for NLP 34, as well as other parameters used in online fraud detection. For example, knowledge base 54a can store a table mapping text summaries to fraud categories, enabling NLP 34 to determine whether a text message indicates fraud based on the summary of the corresponding message. This table / mapping can be determined a priori based on a reference corpus including both legitimate and fraudulent messages belonging to various categories. Knowledge base 54a can further store tables mapping fraud categories to category-specific explanations, recommendations, and / or suggestions displayed to the user.

[0057] Figure 7 An exemplary sequence of steps performed by a threat analyzer 30 in some embodiments of the present invention is shown. In step 260, the analyzer 30 receives a target indicator 22 from a chatbot agent 20. The target indicator 22 includes an indicator of a target object, such as a segment of text or image that will be evaluated against fraudulent indicators. Step 262 then identifies and retrieves the corresponding target object. In some embodiments, the indicator 22 includes the target object itself (e.g., a text message). However, when the target object includes a webpage, document, or media file, the indicator 22 may simply refer to the corresponding target object via an address or URL. In such cases, step 262 may include retrieving the target object from an indicated location.

[0058] In step 264, parser 34 may extract a set of target features 33, which may depend on the type of object (e.g., text message and web page). When the target object contains media files, step 264 may further include applying a media converter to attempt to extract the text content of the corresponding target object.

[0059] When the target object includes natural language text, in step 266, analyzer 30 may execute NLP module 34 to determine a summary of the corresponding text. In an alternative embodiment, step 266 may include developing a LM prompt based on the corresponding target text and submitting the LM prompt to LM 40. The LM prompt may be intentionally developed so that LM 40 outputs a summary of the target text. Several NLP methods for summarizing text are known in the art; details of such methods are beyond the scope of this invention.

[0060] In some embodiments, the target summary is represented as a vector with multiple elements, each element including the value of a different attribute of the target text. Therefore, the summary vector produces a multifaceted summary of the target text. Text attributes are selected to be relevant to fraud analysis; in this sense, the selected combination of attribute values ​​indicates online fraud. Exemplary attributes include whether the target text includes a question, whether the target text invites the user to access a remote resource / URL, whether the target text includes an advertisement / offer, and the topic of the target text (e.g., investment, video games, subscriptions, etc.). Feature values ​​may include numbers, tags, or the entire NL statement, depending on the implementation. (Based on WhatsApp) The exemplary summary vector for the sequence of messages can take the values ​​{3, 1, 34, ...}, or equivalently, {“is a quote”, “contains a hyperlink”, “is about online streaming”, ...}.

[0061] Step 268 may execute detector 36 and / or NLP module 34 to determine target category indicator 35 based on target features 33 and / or target summaries determined in step 266. Step 268 may combine text features with other features of the target object. For example, when the target summary indicates that the target object contains hyperlinks, a dedicated filter of detector 36 may analyze the corresponding hyperlinks, for example, by searching for the domain name of the corresponding hyperlink in a blacklist of known fraudulent domains, determining whether the corresponding hyperlink includes a randomly generated portion, and / or analyzing domain registration data. In another instance, detector 36 may augment the summary vector by extracting other features of the corresponding target text (e.g., whether it contains fraudulent indicator keywords).

[0062] Step 268 may further include mapping a set of feature values ​​characterizing the target object (e.g., summary vector elements and other values ​​determined by detector 36) to object categories. This mapping may be performed by a classifier including a pre-trained neural network or by any other method known in the field of data mining. Object categories may indicate whether the target object is legitimate or fraudulent, and may further identify categories of online scams, such as phishing, investment fraud, cryptocurrency scams, etc.

[0063] In response to determining category indicator 35, step 270 may construct analysis report 52 based on indicator 35. Exemplary analysis report 52 may include category indicator 35 and a target summary determined in step 266. In some embodiments, step 270 includes developing indicator 35 and / or target summary in natural language and including it in NL form in analysis report 52.

[0064] Step 272 may further expand the analysis report 52 with supplements including explanations, recommendations, and / or suggestions tailored to the user. These supplements may include NL text and may be specific to the object category identified in step 268. In other words, the explanations and / or suggestions may be tailored to the relevant category of the target (e.g., fraud vs. legitimate) and / or a specific threat type (phishing vs. investment fraud, etc.). An exemplary supplement of this kind may include a definition of the relevant category of fraud, examples of attacks, a description of typical consequences for the user, and recommended actions (e.g., "Don't open attachments," "Don't click links," etc.). A further step 274 may transmit report 52 to chatbot agent 20.

[0065] When agent 20 receives analysis report 52 from analyzer 30 ( Figure 3-A If step 218 returns yes, some embodiments may use LM model 40 to repackage and / or reformulate the content of analysis report 52 in a user-friendly dialog format. To achieve this, agent 20 may formulate LM prompt 24 based on report 52 and transmit prompt 24 to LM 40. Formulating prompt 24 may include adding the content of report 52 to the context / history of the current dialog and includes a set of instructions that cause LM 40 to interpret report 52 and format LM response 26 into a response to the user (see, for example...). Figure 2 (Message 16b).

[0066] Figure 8 An exemplary hardware configuration of a computer system 80 is shown, which is programmed to perform some of the methods described herein. The computer system 80 generally represents... Figure 1 The front-end device 12 and the server computer system that implements the fraud prevention system 10 are examples of this. The device described is a personal computer; other devices (such as servers, mobile phones, tablet computers, and wearable computing devices) may have slightly different configurations.

[0067] (Several) processors 82 include physical means (e.g., microprocessors, multi-core integrated circuits formed on semiconductor substrates) configured to perform computational and / or logical operations with a set of signals and / or data. Such signals or data may be encoded and delivered to (several) processors 82 in the form of processor instructions (e.g., machine code).

[0068] Memory unit 84 may include volatile computer-readable media (e.g., dynamic random access memory - DRAM) that stores data / signals / instructions accessed or generated by processor(s) 82 during operation. Input device 86 may include a computer keyboard, mouse, and microphone, etc., and includes corresponding hardware interfaces and / or adapters that allow users to introduce data and / or instructions into computer system 80. Output device 88 may include display devices, such as monitors and speakers, and hardware interfaces / adapters, such as graphics cards, that enable corresponding computing devices to transmit data to users. In some embodiments, input and output devices 86 to 88 share common hardware (e.g., a touchscreen). Storage device 92 includes computer-readable media that implements non-volatile storage, retrieval, and writing of software instructions and / or data. Exemplary storage devices include magnetic disks and optical disks and flash memory devices, as well as removable media, such as CDs and / or DVDs and drives. Network adapter(s) 94 includes dedicated hardware that enables computer system 80 to connect to electronic communication networks and / or to other devices / computer systems for data transmission and reception.

[0069] A controller hub 90 generally represents multiple systems, peripherals, and / or chip bus, and / or all other circuitry that enables communication between the processor(s) 82 and the remaining hardware components of the computer system 80. For example, a controller hub 90 may include a memory controller, an input / output (I / O) controller, and an interrupt controller. Depending on the hardware manufacturer, some of these controllers may be incorporated into a single integrated circuit and / or integrated with the processor(s) 82. In another example, a controller hub 90 may include a northbridge connecting the processor 82 to memory 84, and / or a southbridge connecting the processor 82 to devices 86, 88, 92, and 94.

[0070] The exemplary systems and methods described above provide efficient and user-friendly protection against online scams such as phishing.

[0071] Conventional anti-fraud systems typically require the installation of software agents on the user's device, which may take the form of plug-ins or add-ons to electronic communication applications (e.g., email programs, social networking applications, etc.). These agents analyze incoming communications to determine if they indicate online fraud and can flag or suppress suspicious messages. Several such fraud detection methods are described in the relevant field. However, because modern users often use multiple devices and software applications for online communication, local protection solutions often require the concurrent installation and operation of multiple anti-fraud agents, which can be cumbersome, expensive, and cluttered for the user, consuming significant computational resources. The latter is particularly problematic on resource-scarce mobile devices.

[0072] Alternatives to conventional protection solutions execute "in the cloud," thus avoiding some of the drawbacks of local software agents. In typical use cases, users can, for example, send data for analysis to a remote server via an online interface and receive a determination indicating whether the submitted data indicates a threat. However, such solutions may require a certain level of technical skill and / or understanding of computer security threats, which is expected to exceed the technical skill level and understanding of computer security threats of the average user. For example, users may need to know what kind of information to submit for analysis.

[0073] In contrast to conventional anti-fraud solutions, some embodiments of the present invention employ chatbot agents to interact with users in a friendly, conversational manner. Chatbot agents can assist users in performing various tasks, such as determining whether a user is under online threat, such as phishing, providing advice on computer security issues, and answering questions about subscriptions, accounts, billing, etc. In some embodiments, the chatbot agent acts as a popular messaging or social media platform (e.g., Facebook). Or WhatsApp Messenger Users can access this information through the corresponding application's user interface. In other words, users do not need to install or learn any new software to perform fraud analysis. Furthermore, users can submit questions and data related to any communication application or platform through a single chatbot interface. For example, users can use WhatsApp Messenger. Examples of applications to use anti-fraud chatbots related to other communication applications (e.g., Facebook) The chatbot interacts with messages received via email clients, SMS, etc. It automatically identifies user needs, guides users to provide relevant data for analysis, and conveys the analysis results along with explanations, recommendations, and suggestions to protect users from online scams.

[0074] Chatbots implementing Large Language Models (LLMs) are rapidly becoming a popular technology solution for interacting with users in a variety of situations and applications. Advantages include extending the reach of the target technology to users lacking technical or computational backgrounds, and reducing operational costs by replacing human customer service operators with AI agents. Some advanced chatbots (such as ChatGPT from OpenAI Inc.) These chatbots can answer computer security questions and analyze data to determine if a user is a target of a computer security threat. However, research shows that such chatbots sometimes provide incorrect or misleading answers, or answers that heavily depend on how the question is phrased. More importantly, their mastery of computer security questions depends solely on the training corpus they ingest. In other words, if the training corpus does not contain training instances relevant to a specific question or situation, the corresponding chatbot may not return the correct answer or assessment. This problem is particularly acute in the field of computer security, where the methods employed by malware and online scammers are constantly evolving. Therefore, generalized training corpora and methodologies are relatively unlikely to keep pace with the threat landscape.

[0075] In principle, pre-trained LLMs can be further trained to specifically address computer security issues, for example, using specially constructed and maintained text corpora containing instances of online fraud, such as known phishing attempts delivered via email messages, SMS, and social media platforms. However, even if this additional training improves the performance of the corresponding LLM in detecting online fraud, it does not solve the fundamental problem that LLMs are extremely complex systems, typically with billions of adjustable parameters, and whose behavior is essentially opaque and unpredictable.

[0076] Current LLM-based chatbots have also been shown to be vulnerable to malicious manipulation, often referred to in the field as adversarial attacks. Typical examples involve carefully crafting LLM inputs to disable them, such as producing erroneous output, unexpected output (also known as hallucinations), or no output at all.

[0077] In light of the above observations, some embodiments of the present invention pair a general-purpose chatbot with a separate, specially designed threat analyzer. Some embodiments then utilize the chatbot's natural language proficiency to interact with the user in a user-friendly conversational manner, while the threat analyzer performs fraud detection data analysis. In some embodiments, the chatbot identifies targets for fraud analysis based on its conversation with the user and transmits the identified target indicators to the threat analyzer. The threat analyzer may run a series of tests to determine whether the target indicates a fraud and return the analysis's determination to the chatbot. The chatbot can then communicate the determination to the user in a user-friendly conversational manner.

[0078] Separating threat analysis from user interaction provides a significant advantage for some embodiments of the present invention in ensuring complete control, transparency, and predictability of threat analysis. Furthermore, fraud detection / threat analysis methods can be developed and maintained independently of the chatbot. Some embodiments can even utilize, for example, ChatGPT from OpenAI Inc. Or from Google Inc.'s Bard Commercially and / or publicly available implementations of LLM chatbots can significantly simplify maintenance and shorten time-to-market for fraud prevention systems. Meanwhile, analytical judgments and any user suggestions / recommendations are strictly controlled by the threat analyzer's developers.

[0079] Another advantage is that, compared to anti-fraud solutions based solely on LLM, embodiments of the present invention are significantly less susceptible to adversarial attacks. While chatbot components can still be attacked, such attacks are less likely to affect the decision-making or threat analyzer itself compared to conventional LLM chatbots; such attacks are also easier to detect and / or their impact is more easily contained.

[0080] Those skilled in the art will understand that the above embodiments can be modified in many ways without departing from the scope of the invention. Therefore, the scope of the invention should be determined by the appended claims and their legal equivalents.

Claims

1. A computer system comprising at least one hardware processor configured to execute a chatbot agent and a threat analyzer coupled to the chatbot agent, wherein: The chatbot agent is configured to: In response to receiving a Natural Language (NL) message from the user, a language model prompt is formulated based on the NL message. The language model prompt is transmitted to the language model (LM), which is configured to determine an LM response that includes a response to the NL message. The target object used for fraud analysis is identified by the LM response identifier, and The indicator of the target object is transmitted to the threat analyzer; and The threat analyzer is configured to: Perform fraud analysis on the target object to determine whether the target object is indicative of fraud, and The results of the fraud analysis are output to the chatbot agent for transmission to the user.

2. The computer system of claim 1, wherein the chatbot agent is configured to formulate the LM prompt to include a set of instructions that cause the LM to identify the target object based on the NL message.

3. The computer system according to claim 2, wherein: The chatbot agent is configured to further formulate the LM prompt based on another NL message received from the user; and The instruction causes the LM to further identify the target object based on another NL message.

4. The computer system according to claim 1, wherein the chatbot agent is configured to: The LM prompt is configured to include a conversation summary in the LM response, the conversation summary including a summary of the conversation between the chatbot agent and the user, the conversation including the NL message; and The target object is identified based on the summary of the dialogue.

5. The computer system of claim 1, wherein the target object comprises items selected from a group consisting of screenshots received from the user and Uniform Resource Identifiers (URIs) of Internet resources, the URIs being included in the NL message.

6. The computer system of claim 1, wherein the target object comprises text determined based on a dialogue between the chatbot and the user, the dialogue comprising the NL message.

7. The computer system of claim 6, wherein the fraud analysis of the target object comprises: Determine a summary of the text; and Determine whether the text indicates fraud based on the summary of the text.

8. The computer system according to claim 1, wherein: The fraud analysis of the target object includes classifying the target object into selected categories from a plurality of categories, each of the plurality of categories indicating a different type of online fraud; and The results of the fraud analysis include indicators for the selected categories.

9. The computer system of claim 8, wherein the result of the fraud analysis further includes fraud protection recommendations based on the selected category.

10. A computer implementation method comprising using at least one hardware processor of a computer system to execute a chatbot agent and a threat analyzer coupled to the chatbot agent, wherein: Executing the chatbot agent includes: In response to receiving a Natural Language (NL) message from the user, a language model prompt is formulated based on the NL message. The language model prompt is transmitted to the language model (LM), which is configured to determine an LM response that includes a response to the NL message. The target object used for fraud analysis is identified by the LM response identifier, and The indicator of the target object is transmitted to the threat analyzer; and Executing the threat analyzer includes: Perform fraud analysis on the target object to determine whether the target object is indicative of fraud, and The results of the fraud analysis are output to the chatbot agent for transmission to the user.

11. The method of claim 10, wherein the chatbot agent is configured to formulate the LM prompt to include a set of instructions that cause the LM to identify the target object based on the NL message.

12. The method according to claim 11, wherein: The chatbot agent is configured to further formulate the LM prompt based on another NL message received from the user; and The instruction causes the LM to further identify the target object based on another NL message.

13. The method of claim 10, wherein the chatbot agent is configured to: The LM prompt is configured to include a conversation summary in the LM response, the conversation summary including a summary of the conversation between the chatbot agent and the user, the conversation including the NL message; and The target object is identified based on the summary of the dialogue.

14. The method of claim 10, wherein the target object comprises items selected from a group consisting of screenshots received from the user and Uniform Resource Identifiers (URIs) of Internet resources, the URIs being included in the NL message.

15. The method of claim 10, wherein the target object comprises text determined based on a conversation between the chatbot and the user, the conversation comprising the NL message.

16. The method of claim 15, wherein the fraud analysis of the target object comprises: Determine a summary of the text; and Determine whether the text indicates fraud based on the summary of the text.

17. The method of claim 10, wherein: The fraud analysis of the target object includes classifying the target object into selected categories from a plurality of categories, each of the plurality of categories indicating a different type of online fraud; and The results of the fraud analysis include indicators for the selected categories.

18. The method of claim 17, wherein the result of the fraud analysis further includes fraud protection recommendations based on the selected category.

19. A non-transitory computer-readable medium storing instructions that, when executed by at least one hardware processor of a computer system, cause the computer system to form a chatbot agent and a threat analyzer coupled to the chatbot agent, wherein: The chatbot agent is configured to: In response to receiving a Natural Language (NL) message from the user, a language model prompt is formulated based on the NL message. The language model prompt is transmitted to the language model (LM), which is configured to determine an LM response that includes a response to the NL message. The target object used for fraud analysis is identified by the LM response identifier, and The indicator of the target object is transmitted to the threat analyzer; and The threat analyzer is configured to: Perform fraud analysis on the target object to determine whether the target object is indicative of fraud, and The results of the fraud analysis are output to the chatbot agent for transmission to the user.

Images