Scam detection in a text-based conversational application

A generative model-based system in text-based conversational applications monitors and adapts to detect scams by analyzing conversation content, enhancing real-time scam detection and user safety.

WO2026234079A1PCT designated stage Publication Date: 2026-11-12F SECURE CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2026/063077
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-08
Filing Date
2026-05-06
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing text-based conversational applications lack effective methods to detect scams, particularly in real-time, as users are often unaware of malicious activities and scammers continually evolve their strategies, making it difficult to distinguish normal exchanges from potential threats.

Method used

Implement a generative model-based scam detection system that monitors and analyzes ongoing text-based conversations, deriving specific content for outbound messages to facilitate detection and respond to potential scams by adapting to the conversation dynamics.

Benefits of technology

Enhances the ability to detect scams early in conversations, improving user safety by identifying malicious intent and reducing the risk of information exposure or financial loss.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2026063077_12112026_PF_FP_ABST
    Figure EP2026063077_12112026_PF_FP_ABST
Patent Text Reader

Abstract

According to an embodiment, a method (100) for scam detection in a sustained text-based conversation in an online chat application is provided, the method (100) comprising running a screening procedure that comprises: monitoring (104) a first portion of said text-based conversation that comprises a first plurality of outbound messages sent from the first party to the second party and a first plurality of inbound messages sent from the second party to the first party; and applying (106) a generative model to derive (106a), based at least in part on respective content of one or more inbound and / or outbound messages of said first portion, specific content for use in one or more subsequent outbound messages of said first portion, where the specific content is tailored to cause the second party to respond with respective content that facilitates the scam detection, and carry out (106b) the scam detection based at least in part on respective content of the first plurality of inbound and / or outbound messages.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Scam detection in a text-based conversational application TECHNICAL FIELD

[0002] The present invention relates to scam detection in a text-based conversation in an online chat application.

[0003] BACKGROUND

[0004] Various communication services and social media platforms that enable online communication between two or multiple parties provide a convenient means of exchanging and sharing information, both in business contexts and in recreational contexts. However, at the same time such services provide a tempting platform for various malicious activities, such as ‘phishing’ in order to obtain passwords or access credentials of other kind that enable access to sensitive personal information of a user to obtain financial gain, distribution of misinformation for political or ideological gain, online bullying, online sexual harassment, including manipulation for predatory sexual gain, etc. Such malicious activities may be commonly referred to as online threats or as online scam and they may occur in the framework of any service or platform that enables communication between parties, e.g. in phone calls, in email, in text messages, etc.

[0005] One of the most efficient methods to fight online threats is user-awareness: one should avoid disclosing any sensitive or unnecessary information as part of information exchange over a digital service in case any doubts of the identity and / or intention of the other party or parties of the information exchange and / or one should exit discussions that seem to be heading towards suspicious or uncomfortable topics. However, users who are new to the services or who are otherwise not well prepared for online threats may be especially prone to online threats of various kinds, while on the other hand the parties behind such malicious activities are continuously developing their strategies, which makes it harder even for experienced and alert users to always tell a normal information exchange from an online threat, e.g. from an attempted scam.If considering scams in general and ones that aim at getting access to sensitive personal information in particular, recent development of artificial intelligence (Al) has given a significant boost to scammers in developing improved strategies for obtaining access credentials and / or other sensitive information they seek. In contrast, there are digital tools that are able to detect and react to attempted scams e.g. in context of phone calls and email threads, where the scam detection may be based on a communication context, on an identity of a party of a communication and / or on content of the communication. Especially the content-based approaches to scam detection have been likewise improved due to the fast development of Al, while one the other hand there is continuous need for improved solutions for fighting scam in various communication scenarios.

[0006] SUMMARY

[0007] It is an object of the present invention to provide an improved content-based scam detection technique for sustained text-based conversational applications such as online chat.

[0008] According to an embodiment, a method for scam detection in a sustained textbased conversation in an online chat application is provided, the method comprising running a screening procedure that comprises: monitoring a first portion of said text-based conversation that comprises a first plurality of outbound messages sent from the first party to the second party and a first plurality of inbound messages sent from the second party to the first party; and applying a generative model to derive, based at least in part on respective content of one or more inbound and / or outbound messages of said first portion, specific content for use in one or more subsequent outbound messages of said first portion, where the specific content is tailored to cause the second party to respond with respective content that facilitates the scam detection, and carry out the scam detection based at least in part on respective content of the first plurality of inbound and / or outbound messages.According to another embodiment, an apparatus for scam detection in an online chat application is provided, the apparatus comprising the apparatus at least one processor and at least one memory including computer program code for one or more programs, which computer program code, when executed by the at least one processor, causes the apparatus to perform at least the following: monitor a first portion of a sustained text-based conversation between a first party and a second party via said online chat application, where the first portion comprises a first plurality of outbound messages sent from the first party to the second party and a first plurality of inbound messages sent from the second party to the first party; and apply a generative model to: derive, based at least in part on respective content of one or more inbound and / or outbound messages of said first portion, specific content for use in one or more subsequent outbound messages of said first portion, where the specific content is tailored to cause the second party to respond with respective content that facilitates the scam detection, and carry out the scam detection based at least in part on a respective content of the first plurality of inbound and / or outbound messages.

[0009] According to an example, a computer program for scam detection in a sustained text-based conversation in an online chat application is provided, the computer program comprising computer readable program code configured to cause performing at least the method according to the example embodiment described in the foregoing when said program code is executed on a computer apparatus.

[0010] The computer program according to the above-described example may be embodied on a volatile or a non-volatile computer-readable record medium, for example as a computer program product comprising at least one computer readable non-transitory medium having the program code stored thereon, which, when executed by a computer apparatus, causes the computer apparatus at least to perform the method according to the example embodiment described in the foregoing.The exemplifying embodiments of the invention described in the present disclosure and variations thereof are not to be interpreted to pose limitations to the applicability of the appended claims. Some features of the invention are set forth in the appended claims. Aspects of the invention, however, both as to its construction and its method of operation, together with additional objects and advantages thereof, will be best understood from the following description of example embodiments when read in connection with the accompanying drawings.

[0011] Moreover, the verb “to comprise” and its derivatives are used in the present disclosure to indicate an open limitation that does not exclude the existence of also unrecited features.

[0012] BRIEF DESCRIPTION OF FIGURES

[0013] The embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, where

[0014] Figure 1 illustrates a method according to an example;

[0015] Figure 2A illustrates a block diagram that represents a system according to an example;

[0016] Figure 2B illustrates a block diagram that represents a system according to an example;

[0017] Figure 2C illustrates a block diagram that represents a system according to an example;

[0018] Figure 2D illustrates a block diagram that represents a system according to an example;

[0019] Figure 3 illustrates a method according to an example; and

[0020] Figure 4 illustrates a block diagram of some components of an apparatus according to an example.DESCRIPTION OF SOME EMBODIMENTS

[0021] Figure 1 illustrates a method 100, which is applicable for scam detection in a sustained text-based conversion in a chat application between a first party and a second party. The method 100 implements a screening procedure that comprises the following steps:

[0022] - Monitor a first portion of a text-based conversation in an online chat application, where the conversation comprises a first plurality of outbound messages sent from the first party to the second party and a first plurality of inbound message sent from the second part to the first party (block 104).

[0023] - Apply a generative model for scam detection (block 106), where the generative model is applied to provide the following:

[0024] o derive, based at least in part on respective content of one or more inbound and / or outbound messages of said first portion, specific content for use in one or more subsequent outbound messages of said first portion, where the specific content is tailored to cause the second party to respond with respective content that facilitates the scam detection (block 106a), and o carry out the scam detection based at least in part on respective content of the first plurality of inbound and / or outbound messages (block 106b).

[0025] The operations described above with references to blocks 104 and 106 may be varied or complemented in a number of ways without departing from the scope of the method 100, e.g. in accordance with the examples described in the following with a (direct) reference to operations carried out in relation to respective one of the blocks 104 and / or 106 or with a reference to an aspect of operation of an apparatus or a system that makes use of the method 100. Alternatively or additionally, the method 100 as described above may be complemented with one or more further method steps. As an example in this regard, the screening procedure according to the claim 100 may be, optionally, preceded by one or more operations that constitute a pre-screening procedurethat is carried out before proceeding to operations described with references to blocks 104 and 106. The optional pre-screening procedure is described later in this text.

[0026] Herein, the expression ‘sustained text-based conversation’ refers to online conversation that is currently ongoing and that involves a substantially constant flow of outbound and inbound messages without a significant delay between the messages, so the first party is typically engaged throughout the conversation, unlike for example in an email thread, where the first party typically leaves and re-joins the on-going communication and hence has substantial time for absorbing, processing and acting on information received in the inbound messages. Such a sustained text-based conversation may be referred to as a chat and it basically corresponds to ‘live’ conversation between the parties with the exception that it is carried out via the online chat application. While the present disclosure predominantly refers to the conversation between the first party and the second party, this does not exclude participation of further parties in the conversation.

[0027] Figure 2A illustrates a block diagram of some elements of a communication system 10 within which the method 100 may be applied. The communication system 10 comprises a first apparatus 11 , a second apparatus 12, and a chat server arrangement 15. Each of the first apparatus 11 and the second apparatus 12 is able to communicate with the chat server arrangement 15 via a communication network. The chat server arrangement 15 hosts the online chat application that enables the sustained text-based conversation between the first party and the second party. In this regard, the chat server arrangement 15 may execute a computer program that implements the online chat application. Each of the first apparatus 11 and the second apparatus 12 may execute a respective client application (e.g. a browser or a dedicated client application) to send (outbound) messages to the other one of the apparatuses 11, 12 and to receive (inbound) messages from the other one of the apparatuses 11, 12 via the chat server arrangement 15.The first party involves a first user and the first apparatus 11 , where the first user participates in the conversation using the first apparatus 11. The first user may be a human user and the first apparatus 11 may typically be a user apparatus, such as mobile communication device or a personal computer, operated by the first user. Alternatively, the first party may be a computer program executed by the first apparatus 11, where the first party may be referred as an agentic Al. In this second alternative, the first apparatus 11 may be a user apparatus or a server apparatus, which executes the agentic Al that is arranged to participate in the conversation. In other words, in this second alternative, the agentic Al may participate in the conversation instead of the human user. In one scenario, the second party involves a second user and the second apparatus 12, where the second party is likewise a human user and the second apparatus 12 is a user apparatus, such as mobile communication device or a personal computer, operated by the second user. In another scenario, the second party is a computer program executed by the second apparatus 12, where the second party may be referred to as a chatbot. In this latter scenario, the second apparatus 12 may be a user apparatus or a server apparatus, which executes the chatbot that is arranged to participate in the conversation. In an example, the chatbot is arranged to openly declare itself as a non-human user, whereas in another example the chatbot is arranged to hide its identity as a non-human user.

[0028] In context of the present disclosure, the agentic Al refers to an Al system configured to pursue one or more objectives by determining actions to be taken, generating and executing multi-step plans, monitoring outcomes, and iteratively adjusting its behavior based on feedback, with limited or no continuous human intervention. The agentic Al acting as the first party may be a generative Al-based agentic Al that is able to adapt to the conversation and mimic natural human communication. The agentic Al may for example comprise at least one large language model (LLM) and is also able to process data other than natural language such as interpreting images and processing numerical information derived from text. The agentic Al may perform reasoningtasks as a part of the decision making that governs or influences the content of the outbound messages that are produced as a part of the conversation.

[0029] In the context of the present disclosure, the chatbot possibly acting as the second party may be e.g. a deterministic chatbot that derives the content for transmitted to the first party in the inbound messages based on a set of rules or a generative Al-based chatbot that is able to adapt to the conversation and mimic natural human communication. Quite apparently, the generative Al-based chatbots applied for scamming purposes pose a bigger threat to the first party than the deterministic ones, since the first user may not be able to readily identify the second party as a generative Al-based chatbot based on the content of the inbound messages received therefrom.

[0030] Figure 2A further shows a screening entity 14 and a generative model 18 operating in the first apparatus 11 , where the screening entity 14 is arranged to apply the method 100 to carry out the scam detection on behalf of the first party and to apply the generative model 18 in the course of the method 100. Figure 2B illustrates a block diagram of some elements of the communication system 10 in an alternative setting, which includes the screening entity 14 and the generative model 18 operating in the chat server arrangement 15. Figure 2C illustrates a block diagram of some elements of the communication system 10 in another alternative setting, which further includes a third apparatus 13 that is able to communicate with the first apparatus 11 and / or with the chat server arrangement 15 via the communication network and where the screening entity 14 and the generative model 18 are operating in the third apparatus 13. Figure 2D illustrates a block diagram of some elements of the communication system 10 in a further setting, which is similar to the example of Figure 2A apart from inclusion of a model server arrangement 16 that is able to communicate with the first apparatus 11 and / or with the chat server arrangement 15 via the communication network and where the generative model 18 is operating in the model server arrangement 16. In further examples, similar modification that introduces the model server arrangement 16 thatoperates the generative model 18 may be also provided based on the respective examples of Figures 2B and 2C as well, mutatis mutandis.

[0031] The respective examples of Figures 2A, 2B, 2C and 2D are respective nonlimiting examples concerning a relationship between the screening entity 14, the generative model 18 and the apparatuses 11 , 12, 15, 16 involved in the online chat. In the following, reference number 14 is applied to refer to the screening entity in general, where the screening entity 14 may be provided by any apparatus or by any arrangement of two or more apparatuses that is provided with access to respective content of the inbound and outbound message of the text-based conversation between the first and second parties and with access to the generative model 18 to derive the specific content and to carry out the scam detection. Along similar lines, the reference number 18 is applied to refer to the generative model in general, where the generative model 18 may be provided by any apparatus or by any arrangement of two or more apparatuses that is accessible by the screening entity 14. Moreover, in the following, various aspects that pertain to operation of the screening procedure according to the method 100 are predominantly described with references to the screening entity 14 carrying out the method 100 to implement the screening procedure.

[0032] In a viewpoint of the first party, the text-based conversation involves the client application executing in the first apparatus 11 repeatedly carrying out the following:

[0033] - receiving text content from the user via a user interface (III) of the first apparatus 11 and sending the text content received via the III through the server arrangement 15 to the second party, and

[0034] - receiving an inbound message from the second party through the server arrangement 15 and displaying the text content of the inbound message via the III of the first apparatus 11 for viewing by its user.

[0035] In the course of the conversation, the client application executing in the first apparatus 11 typically displays the respective content of the outbound andinbound messages exchanged so far via its III such that messages are shown in the order of their transmission I reception with each message shown with an implicit or explicit indication of its origin (e.g. as an outbound message originating from the first party or as an inbound message originating from the second party). Moreover, the arrangement of the outbound and inbound messages displayed via the III is continuously updated with each new outbound message sent and with each new inbound message received.

[0036] The aspect of monitoring the first portion of the text-based conversation (cf. block 104) may comprise the screening entity 14 receiving the first plurality of outbound messages and the first plurality of inbound messages as the conversation progresses. In this regard, the screening entity 14 may receive each outbound message as it sent from the first entity, receive each inbound message as it is received at the first entity, and store a history of the outbound and inbound messages exchanged between the first and second parties to enable an analysis of their content by the generative model 18. In other words, the monitoring takes place while the conversation between the first and second parties is going on, which enables the screening entity 14 to take actions that facilitate detecting possible scam as early as possible in the course of the conversation, thereby improving safety of the conversation and facilitating safe use of the first apparatus 11 by the first user.

[0037] The aspect of applying the generative model to derive the specific content for use in one or more outbound messages of the first portion of the conversation (cf. block 106) may be provided, for example, in one of the following ways:

[0038] - the screening entity 14 applies the generative model to derive the specific content and sends the specific content in one or more outbound messages on behalf of the first party as part of the first portion of the conversation;

[0039] - the screening entity 14 applies the generative model to derive the specific content and presents the specific content, e.g. via the III of the first apparatus 11 , to the first party to assist the first party in preparationof respective content for one or more outbound messages sent as part of the first portion of the conversation.

[0040] Along the lines described in the foregoing, for each occasion of the screening entity 14 applying the generative model to derive the specific content, the derivation is based at least in part on respective content of one or more inbound and / or outbound messages derived earlier during the first portion of the conversation. In this regard, application of the generative model to derive the specific content may be responsive to a new inbound message such that the generative model is applied to derive new specific content in response to each new inbound message received from the second party, thereby ensuring that derivation of the specific content accounts for all information derivable from the content of the inbound messages received from the second party.

[0041] As an example of using the first option above with regards to making use of the specific content, the screening entity 14 completely takes over the conversation on behalf of the first party by deriving the respective specific content and sending it in one or more outbound messages to the second part on behalf of the first party after each received inbound message. In another example, the screening entity 14 only partially takes over the conversation on behalf of the first party, e.g. such that some of the outbound messages convey text content received from the user via the III of the first apparatus 11 while some of the outbound messages convey the specific content derived via operation of the generative model and they are sent automatically via operation of the screening entity 14.

[0042] As an example of using the second option above with regards to making of use of the specific content, the client application executing in the first apparatus 11 displays the specific content derived via operation of the generative model via the III of the first apparatus 11 , whereas the user has the possibility of making use of the specific content displayed via the III in formulating the text content he / she inputs via the III for transmission in one or more outbound messages to the second party.Operation of the generative model 18 for determining the specific content and for carrying out the scam detection may be partially defined, for example, based on one or more of the following aspects:

[0043] - scam type to be detected,

[0044] - scam detection task to be carried out,

[0045] - scam detection strategy to be applied via the specific content of the outbound messages.

[0046] In various examples, the scam type to be detected by operation of the method 100 via application of the generative model 18 may be any malicious online activity enabled by the text-based conversation. Non-limiting examples in this regard include the scam types outlined in the foregoing, e.g. one of ‘phishing’ in order to obtain access credentials and / or sensitive personal information of the first user for monetary gain and / or to facilitate an identity theft, distribution of misinformation for political or ideological gain, online bullying, online sexual harassment, including manipulation for predatory sexual gain (e.g. ‘grooming’), etc. In a particular example, the scam type to be detected comprises ‘phishing’ in order to obtain a password or access credentials of other kind to obtain an access to the apparatus 11 or another computer system or service (e.g. to an online banking system) e.g. via persuading the first user into disclosing the access credentials in the conversation (in the outbound messages) or via persuading the first user into accessing a website that leads into the first user entering the password or access credentials of other kind. In another example, the ‘phishing’ may involve manipulation of the first user to install malware to the apparatus 11 or to another apparatus e.g. persuading the first user into accessing a website that leads into installation of the malware to the apparatus 11 or to another apparatus.

[0047] In various examples, the scam detection task to be carried out in the course of the method 100 via application of the generative model 18 comprises one or both of the following:detect whether the first plurality of inbound messages received from the second party represent scam,

[0048] determine whether the second party is a chatbot.

[0049] Both these exemplifying scam detection tasks come in many different variations. In non-limiting examples, the first scam detection task above may be applied to designate the information received in the first plurality of inbound messages as one of scam or legitimate, to designate the first plurality of inbound messages as one of scam, legitimate or undecided (in terms of the inbound messages being scam or legitimate), or to determine a likelihood (e.g. a percentage or a score of other kind) of the first plurality of inbound messages representing scam. Alternatively or additionally, in non-limiting examples, the second scam detection task above may be applied to designate the second party as one of chatbot or a human user, to designate the second party as one of chatbot, a human user or undecided (in terms of its identity as a chatbot or a human user), or to determine a likelihood (e.g. a percentage or a score of other kind) of the first plurality of inbound messages originating from a chatbot.

[0050] In various examples, the scam detection strategy to be applied via the specific content generated for the outbound messages may comprise a predefined scam detection strategy or a user-selected one of a plurality of predefined scam detection strategies.

[0051] The applied scam detection strategy, together with the scam type to be detected and / or the scam detection task to be carried out serves to steer the determination of the specific content via application of the generative model 18. Non-limiting examples of applicable scam detection strategies include the following:

[0052] - ‘speculative’, which may result in applying the generative model 18 to determine the specific content as one that includes leading questions or otherwise manipulative content in order to cause the second party to respond with respective content that facilitates carrying out the scam detection via operation of the generative model 18;- ‘naive’, which may result in applying the generative model 18 to determine the specific content as one that gives an impression of the first party playing along with the second party in order to cause the second party to respond with respective content that involves one or more known indicia of scam that hence facilitate carrying out the scam detection via operation of the generative model 18;

[0053] - ‘technical’, which may result in applying the generative model 18 to determine the specific content as one that includes question addressed to (possibly) technical implementation of a computer program serving as the second party in order to cause the second party to respond with (a) respective content that suggest identity of the second party as a nonhuman in scenarios where this identity was originally hidden and / or (b) respective content that reveals malicious nature of the programmed instructions that control the program serving as the second party in scenarios where this identity may not be originally hidden, thereby facilitating the scam detection to be carried out via operation of the generative model 18.

[0054] In cases where the first party is agentic Al, the knowledge of the agentic nature of the first party may be utilized when generating the content for the outbound messages in order to elicit evidence of malicious intent by the second party. The utilization of the knowledge of the agentic nature of the first party may for example result in revealing some tactics applied by the second party that are known to exploit vulnerabilities of agentic Al, for example the ways in which agentic Al may be deceived or accept changes to its initial goals.

[0055] The generative model 18 applied in the screening procedure may comprise a generative Al model. Regardless of the type of the generative model 18 applied, the generative model may be based on a large foundation model (LFM). In some examples, the LFM is provided as a large language model (LLM), whereas in other examples the LFM is provided as a multimodal model that includes an LLM. In particular, in some examples the scam detection is solely based on text content of the inbound and / or outbound messages viaapplication of the generative model 18 that is based on the LLM only, whereas in other examples the scam detection also considers e.g. images carried in the inbound and / or outbound messages via application of the generative model 18 that is based on a multimodal LFM that includes the LLM.

[0056] According to an example, the LFM applied by the generative model 18 comprises a specific LFM that has been derived from a generic LFM via a posttraining procedure to facilitate determination of the specific content and / or carrying out the scam detection based on respective content of one or more inbound and / or outbound messages of a text-based conversation. In this regard, the post-training may be based on a history of scam data obtained from various sources, e.g. from text-based conversations carried out via online chat applications, from email threads, from online discussion forums, from literature, from news sources, etc.

[0057] According to another example, the LFM applied by generative model 18 comprises a generic LFM and the generative model 18 is configured by one or more predefined prompts to facilitate determination of the specific content and / or carrying out the scam detection based on respective content of one or more inbound and / or outbound messages of a text-based conversation. The one or more predefined prompts may be determined via an experimental procure carried out as part of the design of the method 100 and / or operation of the screening entity 14 in implementing the screening procedure according to the method 100. Such an experimental procedure may be referred to as prompt engineering that aims at finding, or even optimizing, the one or more predefined prompts such that they serve the purpose of deriving the specific content and carrying out the scam detection in accordance with operations described with references to blocks 106a and 106b of the method 100.

[0058] The one or more predefined prompts may comprise one or more of the following:

[0059] a definition of a scam detection framework,

[0060] a definition of the scam type under consideration,- a definition of the scam detection task to be carried out,

[0061] - a definition of the scam detection strategy that specifies the type of content to be determined for the outbound messages to be sent to the second party.

[0062] Herein, the definition of the scam detection framework may comprise an overall description of the scam detection context to be applied by the screening entity 14, whereas the respective definitions of the scam type under consideration the scam detection task to be carried out, and the scam detection strategy to be applied may define one of the respective examples described in the foregoing.

[0063] As an example concerning determination of the one or more predefined prompts, a single prompt engineering procedure may be carried out to define respective one or more prompts for controlling derivation of the specific content (cf. block 106a) and for carrying out the scam detection (cf. block 106b) in view of the scam detection task to be carried out (e.g. reaching a verdict whether the conversation involves scam), whereas in other example separate prompt engineering procedures may be carried out to define respective one or more prompts for controlling derivation of the specific content (cf. block 106a) and for carrying out the scam detection (cf. block 106b). Further in this regard, the examples provided in the present disclosure predominantly refer to the generative model 18 as one that both determines the specific content and carries out the scam detection (to make a decision in view of the scam detection task at hand), whereas in other examples separate generative models may be used for these two functionalities, e.g. such that the generative model 18 is applied to derive the specific content (cf. block 106a) and another generative model is applied for carrying out the scam detection (cf. block 106b). In either case, whether the generative model applied for carrying out the scam detection is the same that determines the specific content during conversation or a separate generative model, that model carries out the scam detection according to additional prompt engineering, whereby scam detection takes the form of a verdict as to the likelihood of malicious intent on the part ofthe second party along with reasoning, both provided in textual form to the first party.

[0064] Along the lines described in the foregoing via a number of examples, the application of the generative model 18 (cf. block 106) comprises the ‘sub-steps’ of deriving (106a) the specific content and carrying out (106b) the scam detection. Even though as described herein as respective separate tasks provided via application of the generative model 18, the generative model 18 may implement the aspects of deriving the specific content and carrying out the scam detection as two outcomes of a single process, as respective outcomes of two separate processes, or as respective outcomes of two processes where part of the processing involved is shared with the two processes. In terms of the timeline of deriving the specific content and carrying out the scam detection, e.g. one of the following approaches may be applied:

[0065] - Multiple occasions of deriving respective specific content for one or more outbound messages is applied before proceeding into carrying out the scam detection based on the respective content of the inbound messages received in response to the specific content applied so far. In other words, in this approach the generative model 18 is applied to collect multiple inbound messages that are received in response to the (respective) outbound messages that convey the (respective) specific content before proceeding into carrying out the (attempted) scam detection.

[0066] - Scam detection is carried out after each inbound message that is received in response to the most recently sent outbound message that convey the (respective) specific content. In other words, this approach involves an attempt for detecting scam based on the respective content of the inbound messages received in response to the specific content applied so far after reception of each inbound message.

[0067] The screening procedure is terminated with a decision that serves as an outcome of the screening procedure. In various examples, the decision may be the outcome of the scam detection step (cf. block 106b) obtained viaapplication of the generative model 18 (or another generative model) in accordance with the examples provided in the foregoing in view of the scam detection task carried out via application of the generative model 18. In particular, the decision may concern at least one of the following:

[0068] - determination whether the first plurality of inbound message represents scam,

[0069] - determination whether the second party is a chatbot.

[0070] In various examples, the screening procedure comes into a conclusion, i.e. the screening procedure is terminated with a decision, in response to having received, in the first plurality of inbound messages, respective content that is sufficient for the generative model 18 to make the decision or in response to having received a predefined number of inbound messages that are respective responses to respective outbound messages that are based on the specific content.

[0071] After having terminated the screening procedure described in the foregoing with a decision, in an example, the screening entity 14 may report the outcome of the scam detection (i.e. the decision) to the first party e.g. via displaying the decision via the III of the first apparatus 11 regardless of the outcome. According to another example, the screening entity 14 may report the outcome only in those cases where the decision indicates that there is at least potential scam involved, e.g. when the first plurality of inbound messages is found to represent scam and / or when the second party is found to be a chatbot.

[0072] In various examples, the screening entity 14 may further proceed into taking a predefined action in response to the decision taken in the screening procedure indicates that there is at least potential scam involved. In this regard, the predefined action may comprise e.g. one of the following:

[0073] - issuing an alert to the first party, where the alert may further involve instructions regarding one or more actions to be taken by the first party, where the one or more actions may include, but are not restricted to, the first party taking back control of subsequent outbound messageswith a new stance based on advice from the screening entity 14 or, in the case where the first party is agentic Al, the first party relinquishing control of subsequent outbound messages to the human user,

[0074] - blocking subsequent transmission of outbound messages to the second party and / or blocking subsequent reception of inbound messages from the second party,

[0075] - applying the generative model to determine further content for use in subsequent outbound messages to the second party, where the further content is tailored to prolong the conversation between the first party and the second party, frustrating the malicious efforts of the second party and consuming its resources to reduce its potential for harm.

[0076] The examples provided in the foregoing with references to the method 100 and / or its implementation via operation of the screening entity involve the respective operations that relate to monitoring of the first portion of the textbased conversation (cf. block 104) and application of the generative model to derive the specific content and to carry out the scam detection (cf. blocks 106, 106a, 106b), whereas in various examples the screening procedure may be initiated automatically upon the first party engaging into the conversation with the second party or in response to activation by the first party (e.g. based on user input received via the III of the first apparatus 11 ).

[0077] In a further example, the screening procedure according to the method 100 may be initiated via operation of a pre-screening procedure, which may be likewise carried out via operation of the screening entity 14. The pre-screening procedure may be initiated e.g. automatically upon the first party engaging into the conversation with the second party or in response to the fist party activating the pre-screening procedure (e.g. based on user input received via the III of the first apparatus 11). As an example of making use of the pre-screening procedure, the method 100 described in the foregoing may preceded by carrying out a method 100’ illustrated in Figure 3. The method 100’ implements the pre-screening procedure and it includes the following steps:- Monitor a second portion of the text-based conversation, where the second portion precedes the first portion in time and comprises a second plurality of outbound messages sent from the first party to the second party and a second plurality of inbound messages sent from the second party to the first party (block 101 ).

[0078] - Initiate the screening procedure (according to the method 100) based on an analysis of respective content of the second plurality of inbound and / or outbound messages (block 102).

[0079] It is worth noting that the second portion of the text-based conversation is carried out between the first party and the second party without involvement of the generative model 18 described in the foregoing. It is likewise worth noting that the specifiers ‘first’ and ‘second’ applied herein in relation to the portions of the text-based conversation under consideration and in relation to the respective pluralities of outbound and inbound messages therein do not represent a temporal or sequential order in this regard but they are applied as labels that distinguish the second portion of the conversation from the first portion of the conversation and their respective outbound and inbound messages from each other, where the second portion occurs before the first portion (as already defined above in context of operations that pertain to block 101). In this regard, the specifier ‘first’ is applied to refer to the later-occurring portion of the conversation (and its respective outbound and inbound messages) simply because this latter portion of the conversation is rather thoroughly described before proceeding into description that (also) involves the ‘second’ portion, thereby aiming at avoiding confusion that could arise from starting the description from the ‘second’ portion and only later proceeding into description of the ‘first’ portion.

[0080] The aspect of monitoring the second portion of the text-based conversation (cf. block 101) may comprise the screening entity 14 receiving the second plurality of outbound messages and the second plurality of inbound messages as the conversation progresses. In this regard, the screening entity 14 may receive each outbound message as it sent from the first entity, receive eachinbound message as it is received at the first entity, and store a history of the outbound and inbound messages exchanged between the first and second parties to enable their subsequent analysis in order to determine the likelihood of scam. In other words, also in the case of the pre-screening procedure the monitoring takes place while the conversation between the first and second parties is going on, which enables the screening entity 14 to activate the actual screening procedure to facilitate detecting possible scam as early as possible in the course of the conversation, thereby improving safety of the conversation and facilitating safe use of the first apparatus 11 by its user.

[0081] The aspect of initiating the screening procedure based on the analysis of the respective content of the second plurality of inbound and / or outbound messages (cf. block 102) may comprise, for example, the following:

[0082] - determining, based on the respective content of the second plurality of inbound and / or outbound messages, a likelihood of the second portion of the text-based conversation representing scam, and

[0083] - initiating the screening procedure in response to said likelihood exceeding a predefined threshold.

[0084] In this regard, the analysis may involve applying a pretrained model that is arranged to estimate the likelihood based at least in part on respective content of the second plurality of inbound and / or outbound messages. In this regard, the pretrained model may comprise a machine-learned model that is based on Natural Language Processing (NLP) techniques applied to the respective content of said second plurality of inbound and / or outbound messages. The NLP techniques may include, but are not limited to conversational dynamics, observing the dynamic behaviours of numerical representations of sentiment and / or semantics, and static NLP features calculated over the whole or part of a conversation that quantify linguistic style and structure.

[0085] In the course of the above-described example of initiating the screening procedure based on the analysis of the respective content of the second plurality of inbound and / or outbound messages, the likelihood of the secondportion of the conversation representing scam may be determined, for example, as a scam likelihood score in a predefined scale, e.g. as a percentage. The scam likelihood score is determined in view of the scam type under consideration (e.g. an online fraud) and in view of the scam detection task at hand (e.g. detecting whether the inbound messages represent scam or detecting whether the second party is a chatbot) and, consequently, it serves to indicate the likelihood of this specific scam type in consideration of the scam detection task at hand. The predefined threshold applied in this example may be set in view of the type of the applied scam likelihood score. Assuming that the scam likelihood score is a percentage (implicitly in a range from 0 to 100), the threshold may be chosen e.g. from a range from 20% to 50% to provide a desired sensitivity of initiating the screening procedure based on a threat level detected based on the second plurality of inbound messages.

[0086] In another example, the aspect of initiating the screening procedure based on the analysis of the respective content of the second plurality of inbound and / or outbound messages (cf. block 102) may comprise using a generative Al model to make a decision on possible initiation of the screening procedure according to the method 100 based on the respective content of the second plurality of inbound and / or outbound messages of the second portion of the conversation. Hence, in this approach the Al model (at least conceptually) makes a decision whether the respective content of the second plurality of inbound and / or outbound messages likely represents scam or not, thereby substantially deriving a binary scam detection score as part of its operation. The generative Al model applied for the analysis in this approach may be configured based on one or more prompts found via a prompt engineering procedure carried out e.g. in consideration of the scam type under consideration.

[0087] A variation of the pre-screening procedure according to the method 100’ omits the step of (automatically) initiating the screening procedure based on the outcome of the analysis carried out based on respective content of the second plurality of inbound and / or outbound messages (cf. block 102) and, instead, proceeds into displaying the determined likelihood (e.g. the scam likelihoodscore) to the first party e.g. via the III of the first apparatus 11 , thereby leaving the choice of initiating the screening procedure to the first user. Consequently, the screening procedure may be initiated based on user input received via the III of the first apparatus 11, where the user input is a response to the determined likelihood displayed to the first user.

[0088] As described above, the first party may be agentic Al. The model (either the pretrained machine-learned model or the generative Al model) used in the analysis of the pre-screening procedure may be adapted (e.g. trained or defined) specifically for cases where the first party is agentic Al. For example, in adapting the model specifically for cases where the first party is agentic Al it may be taken into account that conversations where the first party is the agentic Al bare different linguistic traits and conversational dynamics to those where the first party is human. Alternatively or in addition, it may be taken into account that certain types of scams may deliberately target the cases, where the first party is an agentic Al, exploiting weaknesses or predictabilities in behavior that would not be typical of human first parties. Knowledge of these agent-targeting tactics may help make a more successful, specialized model for use in cases where the first party is agentic Al.

[0089] Figure 4 illustrates a block diagram of some components of an apparatus 200 that may be employed to implement at least some of the operations described in the foregoing with references to any of the methods 100, 100’. The apparatus 200 comprises a processor 210 and a memory 220. The memory 220 may store data and computer program code 225. The apparatus 200 may further comprise communication means 230 for wired and / or wireless communication with other apparatuses and / or user I / O (input / output) components 240 that may be arranged, together with the processor 210 and a portion of the computer program code 225, to provide the user interface for receiving input from a user and / or providing output to the user. The user I / O components may include user input means, such as one or more keys or buttons, a keyboard, a touchscreen or a touchpad, etc. The user I / O components may include output means, such as a display or a touchscreen.The components of the apparatus 200 are communicatively coupled to each other via a bus 250 that enables transfer of data and control information between the components.

[0090] The memory 220 and a portion of the computer program code 225 stored therein may be further arranged, with the processor 210, to cause the apparatus 200 to perform at least some aspects of the method 100 and / or the method 100’. The processor 210 is configured to read from and write to the memory 220. Although the processor 210 is depicted as a respective single component, it may be implemented as respective one or more separate processing components. Similarly, although the memory 220 is depicted as a respective single component, it may be implemented as respective one or more separate components, some or all of which may be integrated / removable and / or may provide permanent I semi-permanent / dynamic / cached storage.

[0091] The computer program code 225 may comprise computer-executable instructions that implement at least some aspects of the method 100 and / or the method 100’. As an example, the computer program code 225 may include a computer program consisting of one or more sequences of one or more instructions. The processor 210 is able to load and execute the computer program by reading the one or more sequences of one or more instructions included therein from the memory 220. The one or more sequences of one or more instructions may be configured to, when executed by the processor 210, cause the apparatus 200 to perform at least some aspects of the method 100 and / or the method 100’. Hence, the apparatus 200 may comprise at least one processor 210 and at least one memory 220 including the computer program code 225 for one or more programs, the at least one memory 220 and the computer program code 225 configured to, with the at least one processor 210, cause the apparatus 200 to perform at least some aspects of the method 100 and / or the method 100’.

[0092] The computer program code 225 may be provided e.g. a computer program product comprising at least one computer-readable non-transitory medium having the computer program code 225 stored thereon, which computerprogram code 225, when executed by the processor 210 causes the apparatus 200 to perform at least some aspects of the method 100 and / or the method 100’. The computer-readable non-transitory medium may comprise a memory device, a record medium or another article of manufacture that tangibly embodies the computer program. As another example, the computer program may be provided as a signal configured to reliably transfer the computer program.

[0093] Reference(s) to a processor herein should be construed broadly, encompassing both programmable processors and dedicated circuits such FPGAs, ASICs, etc.

Claims

26Claims1. A method (100) for scam detection in a sustained text-based conversation in an online chat application between a first party and a second party, the method (100) comprising running a screening procedure that comprises:monitoring (104) a first portion of said text-based conversation that comprises a first plurality of outbound messages sent from the first party to the second party and a first plurality of inbound messages sent from the second party to the first party; andapplying (106) a generative model to:derive (106a), based at least in part on respective content of one or more inbound and / or outbound messages of said first portion, specific content for use in one or more subsequent outbound messages of said first portion, where the specific content is tailored to cause the second party to respond with respective content that facilitates the scam detection, andcarry out (106b) the scam detection based at least in part on respective content of the first plurality of inbound and / or outbound messages.

2. A method (100) according to claim 1, wherein the screening procedure comprises applying (106) the generative model to derive (106a) said specific content and sending said specific content in one or more outbound messages of said first portion on behalf of the first party.

3. A method (100) according to claim 1, wherein the screening procedure comprises applying (106) the generative model to derive (106a) said specific content and presenting said specific content to the first party to assist the first party in preparation of respective content of one or more outbound messages of said first portion.

4. A method (100) according to any of claims 1 to 3, wherein at least one of the following applies:the first party comprises one of the following: a first apparatus (11) operated by a first user, a first apparatus (11) running software that implements an agentic Al;the second party comprises one of the following: a second apparatus (12) operated by a second user, a second apparatus (12) running software that implements a chatbot.

5. A method (100) according to any of claims 1 to 4, wherein the generative model comprises a generative artificial intelligence, Al, model that is based on a large foundation model, LFM.

6. A method (100) according to any of claims 1 to 5, wherein at least one of the following applies:the generative model is based on a specific LFM that is post-trained based on a generic LFM to facilitate determination of said specific content and / or carrying out the scam detection based on respective content of one or more inbound and / or outbound messages;the generative model is based on a generic LFM and is configured by one or more predefined prompts to facilitate determination of said specific content and / or carrying out the scam detection based on respective content of one or more inbound and / or outbound messages.

7. A method (100) according to claim 6, wherein the one or more predefined prompts comprise one or more of the following:a definition of a scam detection framework,a definition of a scam type under consideration,a definition of a scam detection task to be carried out,a definition of a scam detection strategy that specifies a type of content to be determined for sending to the second party.

8. A method (100) according to any of claims 1 to 7, wherein one of the following applies:the generative model is applied to terminate the screening procedure with a decision in response to having received, in the first plurality of inbound messages, respective content that is sufficient for making the decision; the generative model is applied to terminate the screening procedure with a decision in response to having received a predefined number of inbound messages that are responses to respective outbound messages that are based on said specific content.

9. A method (100) according to any of claims 1 to 8, wherein at least one of the following applies:carrying out the scam detection comprises determining whether the first plurality of inbound messages represents scam;carrying out the scam detection comprises determining whether the second party is a chatbot.

10. A method (100) according to claim 9, further comprising one of the following in response to detecting that the first plurality of inbound messages represents scam and / or detecting that the second party is a chatbot:issuing an alert to the first party,blocking subsequent transmission of outbound messages to and / or reception of inbound messages from the second party,29applying the generative model to determine further content for use in subsequent outbound messages to the second party, where the further content is tailored to prolong said the text-based conversation.

11. A method (100, 100’) according to any of claims 1 to 10, comprising running a pre-screening procedure, where the pre-screening procedure comprises:monitoring (101) a second portion of said text-based conversation, said second portion preceding the first portion in time and comprising a second plurality of outbound messages sent from the first party to the second party and a second plurality of inbound messages sent from the second party to the first party;initiating (102) the screening procedure based on an analysis of respective content of the second plurality of inbound and / or outbound messages.

12. A method (100, 100’) according to claim 11, wherein initiating (102) the screening procedure comprises:determining, based on said analysis, a likelihood of the second portion representing scam; andinitiating the screening procedure in response to said likelihood exceeding a predefined threshold.

13. A method (100, 100’) according to claim 12, wherein said analysis comprise using a pretrained machine-learned model that is based on natural language processing, NLP, techniques applied to the respective content to the second plurality of the inbound and / or outbound messages, where the NLP techniques comprises one or more of the following:30conversational dynamics observed in said second plurality of inbound and / or outbound messages,numerical representations of sentiment and / or semantic in said second plurality of inbound and / or outbound messages,static NLF features that quantify linguistic style and structure captured in said second plurality of inbound and / or outbound messages.

14. A method (100, 100’) according to claim 11, wherein initiating (102) the screening procedure comprises using a generative artificial intelligence, Al, model to make a decision on initiation of the screening procedure, wherein the Al model is configured using one or more predefined prompts defined via a prompt engineering procedure.

15. An apparatus (11, 14, 15, 200) for scam detection in an online chat application, the apparatus (11, 14, 15, 200) comprising at least one processor (210) and at least one memory (220) including computer program code (225) for one or more programs, which computer program code (225), when executed by the at least one processor (210), causes the apparatus (11, 14, 15, 200) to perform at least the following: monitor a first portion of a sustained text-based conversation between a first party and a second party via said online chat application, where the first portion comprises a first plurality of outbound messages sent from the first party to the second party and a first plurality of inbound messages sent from the second party to the first party; andapply a generative model to:derive, based at least in part on respective content of one or more inbound and / or outbound messages of said first portion, specific content for use in one or more subsequent outbound messages of said first portion, where the specific content is tailored to cause the second31party to respond with respective content that facilitates the scam detection, andcarry out the scam detection based at least in part on a respective content of the first plurality of inbound and / or outbound messages.