System

Abstract

A system is provided.SOLUTION: A system comprising: means for forwarding a call received over a landline to a dedicated calling application; voice recognition means for converting received call data to text data; natural language processing means for detecting patterns of fraud; means for terminating a call if fraud is likely; and means for notifying a user of a fraud warning message.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, fraudulent phone calls have been increasing rapidly, with many victims falling victim to them, particularly the elderly. Current countermeasures are limited and have not provided a fundamental solution. Fixed-line calls are difficult to monitor, and it is also difficult to respond quickly to ever-evolving fraud methods. Therefore, there is a need for a comprehensive system that analyzes the content of calls in real time, detects potential fraud, and issues appropriate warnings. The purpose of this invention is to solve these problems and ensure user safety. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by using the following means.

[0006] A fixed line forwarding means is provided to forward calls received via fixed line lines to a dedicated calling application. A voice recognition means is provided to convert received call data into text data, and a natural language processing means is constructed to use the converted text data to detect fraud patterns. A call termination means is provided to automatically end the call if there is a high possibility of fraud, and a warning notification means is provided to notify the user of a fraud warning message. In addition, an analysis means is provided to score the possibility of fraud, and by using a generative AI model that is regularly updated, it is possible to quickly respond to the latest fraud techniques. This makes it possible to detect fraudulent calls in real time and ensure user safety.

[0007] A "fixed communication line" is a fixed communication infrastructure that transmits voice and data using physical cables.

[0008] A "dedicated calling application" is calling software with a specific purpose or function that enables specific call management and analysis.

[0009] The "fixed communication line forwarding means" is a function or device that forwards a call received via a fixed communication line to another call application.

[0010] "Received call data" refers to all voice data of calls obtained from outside, and is subject to analysis and conversion.

[0011] "Speech recognition means" refers to technology or devices for converting voice data into text data, automatically analyzing the voice and outputting it as text.

[0012] "Text data" refers to character information converted by a voice recognition means.

[0013] "Natural language processing means" refers to technology or devices that analyze text data and understand its meaning and context.

[0014] "Fraud patterns" refer to a sequence of phrases or actions characteristic of fraud, and are a criterion for identifying fraudulent activity by detecting them.

[0015] A "call termination means" is a function or device that automatically terminates a call when it is determined that there is a high possibility of fraud.

[0016] A "warning notification means" is a function or device for sending a warning message to a user in the event of a possible fraud.

[0017] "Analytical tools" are techniques or devices used to evaluate data and derive a specific result (e.g., likelihood of fraud).

[0018] A "generative AI model" refers to an artificial intelligence model that uses machine learning to perform certain generative processes, and is used for real-time data analysis and pattern recognition. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] MODE FOR CARRYING OUT THE INVENTION

[0041] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system transfers calls received via fixed communication lines to a dedicated calling application and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[0042] 1. Transferring calls from a landline to a dedicated calling app

[0043] server

[0044] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[0045] Terminal

[0046] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[0047] user

[0048] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[0049] 2. Real-time conversion of call content

[0050] server

[0051] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[0052] Terminal

[0053] The terminal continues to send call data to the server.

[0054] 3. Fraud Pattern Detection

[0055] server

[0056] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[0057] Terminal

[0058] The terminal receives the analysis results from the server in real time.

[0059] 4. Call End and Warning Notifications

[0060] server

[0061] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[0062] Terminal

[0063] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0064] user

[0065] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[0066] 5. Call history storage and AI model updates

[0067] server

[0068] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[0069] user

[0070] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0071] Specific examples

[0072] Next, we will explain how this system works in practice using a specific scenario.

[0073] Scenario: The process of detecting fraudulent calls

[0074] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0075] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0076] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0077] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0078] 5. The user receives a warning message and considers appropriate action.

[0079] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[0080] The processing flow will be explained below.

[0081] Step 1:

[0082] The user receives a call on a landline, which is then connected to the server when the call is initiated.

[0083] Step 2:

[0084] The server detects incoming calls via fixed line and, once it has confirmed that a call has been initiated, prepares to transfer the call data to a dedicated calling application.

[0085] Step 3:

[0086] The server transfers the call data to a dedicated call application, which then sends the call data from the server to the device via the application.

[0087] Step 4:

[0088] The terminal starts the dedicated call application and receives the call data transferred from the server, so that the call continues on the terminal.

[0089] Step 5:

[0090] The device sends call data to the server in real time. Through this collaboration, the contents of the call are delivered to the server in real time.

[0091] Step 6:

[0092] The server uses a voice recognition means to convert the received call data into text data in real time.

[0093] Step 7:

[0094] The server then analyzes the converted text data using natural language processing technology, which detects phrases and patterns characteristic of fraud contained in the text data.

[0095] Step 8:

[0096] The server scores the likelihood of fraud and assesses the risk of fraud based on the results. If fraud is deemed likely, the next step is taken.

[0097] Step 9:

[0098] If the server determines that there is a high possibility of fraud, it sends an instruction to the terminal to end the call.

[0099] Step 10:

[0100] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0101] Step 11:

[0102] The user receives a warning message and is alerted to the fraud and considers appropriate action, such as contacting a family member or reporting the incident to the police.

[0103] Step 12:

[0104] The server stores the analysis results and related data, which allows the call history to be stored in a database for future analysis and trend analysis.

[0105] Step 13:

[0106] The server periodically updates the generated AI model so that it can respond to new fraud methods. To maintain analysis accuracy, the model learns the latest fraud information.

[0107] Step 14:

[0108] Users will receive notifications as the system updates, and can review and change their call settings as needed.

[0109] Step 15:

[0110] Servers and devices continue to monitor and analyze to continuously detect fraudulent calls and protect users.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] In recent years, the number of victims of fraudulent phone calls has been increasing, making it difficult for users to take preventative measures. Elderly people and those who are not familiar with technology in particular have difficulty identifying fraudulent calls, which tends to increase the damage they cause. To address this issue, a system is needed that analyzes the content of calls in real time, quickly detects potentially fraudulent calls, and protects users.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes a means for transferring calls received via a fixed communication line to dedicated call software, voice recognition software for converting received call data into text data, a natural language processing program for detecting fraud patterns, a means for terminating calls when there is a high possibility of fraud, a means for notifying users of a fraud warning message, and a means for analyzing fraud patterns using a generative AI model. This makes it possible to detect fraudulent calls in real time and quickly and reliably protect users.

[0116] A "fixed line" is a communications infrastructure that transmits data and voice over physical wiring.

[0117] "Purpose-built Call Software" means a call management application that is optimized for a particular use or environment.

[0118] "Speech recognition software" refers to software technology that converts voice data into text data.

[0119] A "natural language processing program" is software that has the ability to understand and analyze human language.

[0120] A "generative AI model" is an artificial intelligence model generated using machine learning and data science techniques.

[0121] "Fraud patterns" are characteristics that identify fraudulent activity based on specific phrases or patterns of behavior.

[0122] "Analytical methods" refer to techniques and methods used to analyze data and draw conclusions.

[0123] A "warning message" is a message that notifies users of risks or dangers.

[0124] The present invention is a system that provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed communication lines to dedicated calling software and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[0125] Transfer from a fixed line to dedicated calling software

[0126] server

[0127] The server monitors the fixed line and detects when a call is made. Based on this detection, the call data is transferred to the dedicated call software. The hardware used is preferably a telephone device for fixed lines (such as a general IP phone). When a call is detected, the call data is sent to the receiving port of the dedicated call software via the TCP / IP protocol.

[0128] Terminal

[0129] The terminal starts the dedicated call software and receives the call data transferred from the server. The call continues on the terminal while the call data continues to be sent to the server in real time. The dedicated call software has an interface for the call and a data transmission function.

[0130] user

[0131] Users receive calls through dedicated calling software, which reduces the risk of fraudulent calls.

[0132] Real-time conversion of call content

[0133] server

[0134] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition software (e.g., Google Cloud Speech-to-Text API). The voice data is sent to the API, which returns the converted text data.

[0135] Fraud Pattern Detection

[0136] server

[0137] The server analyzes the text data using generative AI models (such as BERT), a natural language processing program, to identify phrases and context characteristic of fraud and score the likelihood of fraud.

[0138] Terminal

[0139] The device receives the analysis results from the server in real time, and the analysis result data is sent to the application, which prepares the display and audio notification.

[0140] Call End and Warning Notifications

[0141] server

[0142] If the server determines that there is a high possibility of fraud, it will send an instruction to the device to end the call, along with information to send a warning message to the user.

[0143] Terminal

[0144] The device receives the call termination instruction from the server and automatically terminates the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please be on alert immediately."

[0145] user

[0146] The call will be automatically terminated and the user will receive a warning message, informing them of appropriate action (e.g., contacting family or the police) if necessary.

[0147] Call history storage and AI model updates

[0148] server

[0149] The server stores and manages the call analysis results and related data. The database management system used is a standard relational database (e.g., MySQL). Based on the results, the call history is stored in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud techniques.

[0150] user

[0151] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0152] Specific examples

[0153] Next, we will explain how this system works in practice using a specific scenario.

[0154] Scenario: The process of detecting fraudulent calls

[0155] 1. The user receives a call on a landline, which is then forwarded by the server to dedicated calling software.

[0156] 2. Dedicated calling software on the device receives the call and sends the call data to the server.

[0157] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0158] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0159] 5. The user receives a warning message and considers appropriate action.

[0160] Prompt Sentence Examples

[0161] "To conduct analysis on fraud call detection, please convert the following phone call into text and detect fraud patterns. The call goes like this: 'Hello, this is a bank representative. There has been fraud on your account. Please provide your card details for verification.'"

[0162] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[0163] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0164] Step 1: Call detection and routing

[0165] server

[0166] The server monitors the fixed communication line and detects when a call is made.

[0167] Input: Speech signal

[0168] Specific operation: A server connected to a telephone device for fixed communication lines (a typical IP phone) detects the start of a call.

[0169] Output: Call data (audio data)

[0170] Step 2: Transferring call data to dedicated calling software

[0171] server

[0172] The server transfers the detected call data to dedicated call software.

[0173] Input: Call data

[0174] Specific operation: Call data is sent via the TCP / IP protocol to the receiving port of the dedicated call software.

[0175] Output: Transferred call data

[0176] Step 3: Receiving and sending call data

[0177] Terminal

[0178] The terminal starts dedicated call software and receives the call data transferred from the server.

[0179] Input: Transferred call data

[0180] Specific operation: The dedicated call software starts up and captures the call data from the receiving port.

[0181] Output: Received call data

[0182] The terminal continues to send the received call data to the server in real time.

[0183] Input: Received call data

[0184] Specific operation: The data transmission module transmits the voice data to the server in real time.

[0185] Output: Call data sent

[0186] Step 4: Convert audio data to text

[0187] server

[0188] The server converts the received call data into text data using voice recognition software.

[0189] Input: Call data sent

[0190] Specific operation: The call data is sent to the Google Cloud Speech-to-Text API, and the audio data is converted into text data.

[0191] Output: Converted text data

[0192] Step 5: Fraud pattern analysis of text data

[0193] server

[0194] The server analyzes the text data using a generative AI model.

[0195] Input: Converted text data

[0196] How it works: A natural language processing program using the BERT model analyzes text data and detects fraudulent patterns.

[0197] Output: Analysis results (fraud score)

[0198] Step 6: Notification of analysis results

[0199] server

[0200] If the server determines that there is a high possibility of fraud, it sends a call termination instruction and a warning message to the terminal.

[0201] Input: Analysis result (fraud score)

[0202] Specific operation: If the fraud score exceeds a certain threshold, generate call termination instructions and warning message data and send them to the terminal.

[0203] Output: End call prompt and warning message

[0204] Step 7: Ending the call and warning notifications

[0205] Terminal

[0206] The terminal receives a call end instruction from the server and automatically ends the call.

[0207] Input: End call instruction

[0208] Specific action: The dedicated call software ends the call.

[0209] Output: Call ended

[0210] It then plays a warning message to the user.

[0211] Input: Warning message information

[0212] What happens: A warning message is played.

[0213] Output: The warning message played

[0214] Step 8: Consider the best course of action

[0215] user

[0216] The user can review the automatically ended call and the warning message that is played and consider appropriate action (e.g., contacting family or reporting to the police).

[0217] Input: The warning message played

[0218] Specific Action: The user hears the warning message and decides what to do next.

[0219] Output: Considered solutions

[0220] Step 9: Save call history and update the model

[0221] server

[0222] The server stores the call analysis results and related data for future analysis, and regularly updates the generative AI model to adapt to new fraud techniques.

[0223] Input: Call analysis results and related data

[0224] How it works: A database management system stores the analysis results, and the AI ​​model is periodically retrained.

[0225] Output: Saved data and updated AI model

[0226] Step 10: Update Notification

[0227] user

[0228] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0229] Input: Update notification

[0230] Specific behavior: The user checks the notification and reviews the settings.

[0231] Output: Updated system settings

[0232] (Application example 1)

[0233] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0234] With traditional calls made over fixed-line communications, it was difficult to detect the risk of fraudulent calls in real time, putting users at high risk of falling victim to fraud. Similar issues existed with mobile communications and voice over IP (VoIP), and a comprehensive solution was needed.

[0235] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0236] In this invention, the server includes means for transferring calls received via fixed communication lines to a dedicated call application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating calls when there is a high possibility of fraud, means for notifying the user of a fraud warning message, server communication means for performing speech recognition on the voice of the call content in real time on the smartphone and detecting fraud patterns, and means for issuing a warning to the user's smartphone based on the results of scoring the possibility of fraud. This makes it possible to detect fraud risks not only on fixed communication lines but also on mobile lines and internet calls in real time and protect users.

[0237] A "fixed line" is a communications infrastructure for transmitting voice data over physical telephone lines or optical fiber.

[0238] A "dedicated calling application" is software designed for a specific purpose, and is an application for managing and analyzing call data.

[0239] "Speech recognition means" refers to technology or equipment for converting voice data into text data, and generally includes speech recognition software and hardware devices.

[0240] "Natural language processing means" refers to technologies and devices for analyzing text data and detecting fraudulent patterns, and generally includes natural language processing software and algorithms.

[0241] "Call termination measures" refers to technology or devices that automatically disconnect a call if it is determined that fraud is likely.

[0242] "Means for notifying users of fraud warning messages" refers to technologies or devices used to warn users of potentially fraudulent calls, and generally includes notification and alert systems.

[0243] "Voice recognition means" refers to technology or devices that use a smartphone's microphone to capture voice data and convert it into text.

[0244] "Server communication means" refers to the technology or device used to send voice data or text data to the server and receive the analysis results.

[0245] "Means for detecting fraud patterns" refers to technologies and devices that identify fraud patterns in real time based on analyzed text data.

[0246] "Fraud likelihood scoring result" refers to the evaluation result that quantifies the risk of fraud based on the analysis results.

[0247] "Warning Means" means any technology or device that delivers a warning message to a user in audio, text, or visual form.

[0248] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the calls in real time. Specific embodiments of this system are described in detail below.

[0249] 1. Transferring calls from a landline to a dedicated calling app

[0250] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[0251] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[0252] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[0253] 2. Real-time conversion of call content

[0254] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition methods, specifically the speech_recognition library and the Google Speech Recognition API.

[0255] The terminal continues to send call data to the server.

[0256] 3. Fraud Pattern Detection

[0257] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[0258] The device receives analysis results in real time from the server, which uses a generative AI model for analysis and updates it regularly.

[0259] 4. Call End and Warning Notifications

[0260] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[0261] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0262] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[0263] 5. Call history storage and AI model updates

[0264] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[0265] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0266] Specific examples

[0267] Next, we will explain how this system works in practice using a specific scenario.

[0268] Scenario: The process of detecting fraudulent calls

[0269] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0270] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0271] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0272] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0273] 5. The user receives a warning message and considers appropriate action.

[0274] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[0275] Example prompt using a generative AI model:

[0276] "Call text: {text of the call}

[0277] Is this call a fraud risk? Generate an appropriate fraud risk score."

[0278] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0279] Step 1:

[0280] When a call is made via a fixed line, the server detects the call and transfers it to a dedicated calling application. The input is the call signal from the fixed line, and the output is the call data transferred to the dedicated calling application. The server monitors the call content and converts the call data into an appropriate format using a dedicated protocol before sending it.

[0281] Step 2:

[0282] A terminal running a dedicated calling application receives call data transferred from the server. The input is the call data sent from the server, and the output is the call content played on the terminal. The terminal analyzes the received data in real time and continues the call.

[0283] Step 3:

[0284] The device continuously transmits the received call data to the server in real time. The input is the voice data during the call, and the output is the voice data transmitted to the server. The device uses a high-performance voice recognition engine to convert the voice into digital data and transmits it to the server.

[0285] Step 4:

[0286] The server converts the voice data received from the device into text data using a voice recognition method. The input is the voice data from the device, and the output is text data. The server analyzes the voice using a voice recognition library (e.g., speech_recognition library) and an external API (e.g., Google Speech Recognition API) and converts it into text.

[0287] Step 5:

[0288] The server uses natural language processing (NLP) technology on the text data to detect fraud patterns. The input is the text data, and the output is a fraud likelihood score. The server uses a generative AI model to analyze characteristic phrases and context contained in the text data and quantify the likelihood of fraud.

[0289] Step 6:

[0290] If the server determines that there is a high possibility of fraud based on the fraud probability score, it sends an instruction to end the call to the terminal. The input is the fraud probability score, and the output is an instruction to end the call. The server checks the fraud judgment criteria, and if the fraud risk is high, it immediately issues an instruction to end the call.

[0291] Step 7:

[0292] The terminal receives a call termination instruction from the server and automatically terminates the call. The input is the call termination instruction, and the output is disconnecting the call and playing a warning message. The terminal terminates the call and displays or plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0293] Step 8:

[0294] The user receives a warning message and considers the appropriate response. The input is the warning message, and the output is the user's action. The user checks the warning message and, if necessary, contacts a family member or calls the police.

[0295] Step 9:

[0296] The server stores and manages the analysis results and related data for calls. The input is the analysis results and call data, and the output is historical data stored in a database. The server uses this data for future analysis and to respond to new fraud methods.

[0297] Step 10:

[0298] The generative AI model is regularly updated to improve its accuracy in order to respond to the latest fraud techniques. The input is new fraud patterns and analytical data, and the output is an updated generative AI model. The server reflects the new data and improves the model's performance.

[0299] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0300] MODE FOR CARRYING OUT THE INVENTION

[0301] The present invention provides an AI calling service that combines an emotion engine to prevent damage caused by fraudulent phone calls. This system transfers calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the call in real time. It also analyzes the user's emotional state and responds accordingly. Detailed modes for implementing the present invention are described below.

[0302] 1. Transferring calls from a landline to a dedicated calling app

[0303] server

[0304] The server monitors the fixed communication line and detects when a call is made. Based on this detection, it prepares to transfer the call data to the dedicated calling application. The call data is transferred from the server to the dedicated calling application and then sent to the terminal via the application.

[0305] Terminal

[0306] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[0307] user

[0308] By receiving calls through a dedicated calling application, users can limit their use of free calling apps and reduce the risk of fraudulent calls.

[0309] 2. Real-time conversion of call content

[0310] server

[0311] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[0312] Terminal

[0313] The terminal continues to send call data to the server.

[0314] 3. Fraud Pattern Detection

[0315] server

[0316] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts of fraud, and scoring the likelihood of fraud.

[0317] Terminal

[0318] The terminal receives the analysis results from the server in real time.

[0319] 4. Call End and Warning Notifications

[0320] server

[0321] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[0322] Terminal

[0323] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0324] user

[0325] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[0326] 5. Emotion analysis using an emotion engine

[0327] server

[0328] The server is equipped with an emotion engine that analyzes not only the content of the call but also the user's emotional state. This emotion engine analyzes the tone, speed, emphasis, etc. of the user's voice during the call to assess the user's emotional state in real time.

[0329] Terminal

[0330] The device sends emotion data along with the call data to the server, which uses this data to perform emotion analysis.

[0331] 6. Emotion-based warnings and instructions

[0332] server

[0333] The server adjusts the warning message based on the user's emotional state, for example, playing a more detailed and reassuring message if the user is showing signs of impatience or anxiety.

[0334] Terminal

[0335] The terminal plays an appropriate warning message to the user based on instructions from the server.

[0336] 7. Call history storage and AI model updates

[0337] server

[0338] The server stores and manages the analysis results and related data of calls. This allows the call history to be stored in a database for future analysis and trend analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraudulent techniques.

[0339] user

[0340] Users will receive notifications as their system is updated and can review and change their settings as needed.

[0341] Specific examples

[0342] Next, we will explain how this system works in practice using a specific scenario.

[0343] Scenario: Detecting fraudulent calls and taking emotions into account

[0344] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0345] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0346] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0347] 4. The server uses the emotion engine to evaluate the user's emotional state in real time.

[0348] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0349] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[0350] 7. The server stores the call analysis results and related data and updates the generative AI model.

[0351] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[0352] The processing flow will be explained below.

[0353] Step 1:

[0354] A user receives a call on a landline. When the call begins, an initial signal is sent from the landline to the server.

[0355] Step 2:

[0356] The server detects when a call is made over a fixed line and, based on this detection, instructs the server to transfer the call data to a dedicated calling application.

[0357] Step 3:

[0358] The server forwards the call data to the dedicated calling application. In this process, the call data is sent from the server to the application.

[0359] Step 4:

[0360] The device starts a dedicated call application and receives the call data transferred from the server, and the device makes the received call available to the user.

[0361] Step 5:

[0362] The device continuously transmits call data to the server. The voice data of the call is sent to the server in real time.

[0363] Step 6:

[0364] The server converts the call data into text data using a speech recognition means, using a high-performance speech recognition engine for this purpose.

[0365] Step 7:

[0366] The server analyzes the text data using natural language processing technology to detect fraudulent patterns and characteristic phrases within the text data.

[0367] Step 8:

[0368] The server scores the likelihood of fraud and assesses the risk of the call based on the result. If it determines that fraud is likely, it proceeds to the next step.

[0369] Step 9:

[0370] The server activates an emotion engine to recognize the user's emotions, which analyzes the user's voice tone, speed, emphasis, etc. in real time.

[0371] Step 10:

[0372] The server evaluates the user's emotional state based on the analysis results of the emotion engine. If the user shows signs of impatience or anxiety, it is determined that there is a high possibility of fraud.

[0373] Step 11:

[0374] If the server determines that there is a high possibility of fraud, it will send an instruction to end the call to the terminal, which will automatically end the call.

[0375] Step 12:

[0376] The device receives the server's instructions, automatically ends the call, and then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0377] Step 13:

[0378] The user receives a warning message, and upon receiving the warning, the user considers appropriate action, such as contacting a family member or reporting the incident to the police.

[0379] Step 14:

[0380] The server stores the call data, analysis results, and emotion data, which can be used for future analysis and data analysis.

[0381] Step 15:

[0382] The server regularly updates the emotion engine and generative AI model to adapt to new scams and emotional patterns.

[0383] Step 16:

[0384] Users will receive notifications about system updates and can check and change their settings as needed, ensuring they always have the latest fraud prevention system.

[0385] Example 2

[0386] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0387] Fraudulent phone calls are a malicious method used to illegally obtain personal information and cause financial damage. The elderly are particularly targeted, and the damage caused by these calls is becoming widespread. However, existing technologies lack effective means for detecting and preventing fraudulent calls. In particular, it has been difficult for existing technologies to detect fraud in real time and take appropriate action while taking into account the emotional state of the user during the call.

[0388] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0389] In this invention, the server includes means for transferring calls received via a fixed communication line to a dedicated calling application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating the call if there is a high possibility of fraud, means for notifying the user of a fraud warning message, emotion analysis means for evaluating the user's emotional state during the call in real time, means for adjusting the warning message based on the user's emotional state, and means for saving and managing the call analysis results and related data. This reduces the risk of fraudulent calls and makes it possible to effectively prevent fraud while providing users with a sense of security.

[0390] "Fixed line" refers to infrastructure for wired communications, including the Internet and the public switched telephone network.

[0391] A "dedicated calling application" is application software designed to process specific calling data.

[0392] A "speech recognition means" is a computer program or algorithm for converting speech data into text.

[0393] "Natural language processing means" is a technology for analyzing text data and understanding its meaning.

[0394] A "pattern of fraud" is a characteristic phrase or context that suggests fraudulent activity.

[0395] A "high likelihood of fraud" means that the content and context of the call indicate a high risk of fraud.

[0396] A "warning message" is a notification intended to alert the user.

[0397] "Emotion analysis means" is a technique for assessing a user's emotional state.

[0398] A "generative AI model" is a model created based on artificial intelligence techniques and used to perform specific tasks.

[0399] "Text data" is data expressed as character information.

[0400] "Analysis results" are information or conclusions obtained based on the analysis of data.

[0401] "Related data" is additional information or metadata related to the analysis results.

[0402] "Preservation" is the act of keeping data in digital form.

[0403] "Management" is the act of keeping stored data available efficiently and effectively.

[0404] MODE FOR CARRYING OUT THE INVENTION

[0405] The present invention provides an AI call service system that combines an emotion analysis engine to prevent damage caused by fraudulent phone calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the call content and the user's emotional state in real time. Detailed embodiments of this system are described below.

[0406] Hardware and Software Configuration

[0407] server

[0408] The server monitors the fixed line and detects when a call is made. When a call is detected, the server performs the following actions:

[0409] 1. Transfer of call data: The server captures call data from fixed communication lines and transfers it to a dedicated call application in real time, buffering the data and converting its format.

[0410] 2. Speech recognition: The server converts the received call data into text using a high-performance speech recognition system (e.g., Google Cloud Speech-to-Text API), which allows the voice content to be analyzed as text information.

[0411] 3. Natural Language Processing (NLP): The server analyzes the text data using natural language processing techniques (e.g., Python's SpaCy or NLTK libraries) to detect fraud patterns, identify phrases and contexts characteristic of specific frauds, and assign a score to the likelihood of fraud.

[0412] 4. Emotion Analysis: The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to assess the user's emotional state. It analyzes acoustic features such as tone, rate, and emphasis of the user's voice to estimate the user's emotional state in real time.

[0413] 5. Warning message generation: If a fraudulent activity is deemed likely, the server generates a warning message for the user. The warning message is tailored based on the user's emotional state to provide a sense of security.

[0414] Terminal

[0415] The device runs a dedicated call application and receives and processes the call data sent from the server. The process on the device is as follows:

[0416] 1. Continuing the call: The dedicated calling application transfers the call data received from the server to the audio output device, allowing the user to continue the call.

[0417] 2. Data transmission: The device continuously transmits voice data and emotional characteristic data to the server in real time during the call. This transmitted data is important for analysis.

[0418] 3. Warning Implementation: If the server determines that there is a high possibility of fraud, the terminal will automatically terminate the call and play a warning message to the user. The warning message will be played through the audio output device.

[0419] user

[0420] Users interact with the system as follows:

[0421] 1. Receiving calls: Receive calls via landline and continue the call through a dedicated calling application, limiting the use of free calling apps and reducing the risk of fraudulent calls.

[0422] 2. Warning Response: If the system detects possible fraud, the user will receive a warning message and consider taking appropriate action (e.g., contacting family, reporting to the police).

[0423] Specific examples

[0424] Next, we will explain how this system works in practice using a specific scenario.

[0425] Scenario: Detecting fraudulent calls and taking emotions into account

[0426] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0427] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0428] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0429] 4. The server uses an emotion analysis engine to assess the user's emotional state in real time.

[0430] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0431] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[0432] 7. The server stores the call analysis results and related data and updates the generative AI model.

[0433] Prompt Sentence Examples

[0434] By inputting the following prompt sentences into the generative AI model, we can get answers about the detailed operation of this system.

[0435] "Please explain the role of the server and terminal in the fraudulent call detection system."

[0436] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[0437] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0438] Step 1:

[0439] The server monitors the fixed line and detects when a call is made. When a call is started, the server prepares to acquire the call data. Specific operations include capturing voice data and allocating a buffer. The input of this step is the call signal from the fixed line, and the output is the acquisition of the ready call data.

[0440] Step 2:

[0441] The server captures call data from fixed communication lines and transfers it to the dedicated calling application in real time. The call data is buffered and format converted. The input of this step is the captured call data, and the output is the data transferred to the dedicated calling application.

[0442] Step 3:

[0443] The terminal starts a dedicated call application and receives the call data sent from the server. The received call data is sent to the audio output device, allowing the user to continue the call. The input of this step is the call data from the server, and the output is data sent to the audio output device.

[0444] Step 4:

[0445] The terminal continues to send voice data during the call to the server in real time. The server converts the received call data into text using a high-performance speech recognition system. The input of this step is the call data and the speech recognition system, and the output is text data.

[0446] Step 5:

[0447] The server then analyzes the converted text data using natural language processing techniques, specifically analyzing phrases and context to detect fraud patterns. The input for this step is the text data, and the output is a scoring result indicating the likelihood of fraud.

[0448] Step 6:

[0449] The server uses an emotion analysis engine to evaluate the user's emotional state in real time. For this analysis, acoustic features such as tone, rate, and emphasis of the voice data are analyzed. The input of this step is the call data and acoustic features, and the output is the evaluation result of the emotional state.

[0450] Step 7:

[0451] If the server determines that there is a high possibility of fraud, it sends a call termination instruction to the terminal. The instruction includes a warning message for the user. The input of this step is the fraud scoring result and the emotion evaluation result, and the output is the call termination instruction and the warning message.

[0452] Step 8:

[0453] The terminal receives the instruction from the server and automatically ends the call. After the call ends, the terminal plays a warning message to notify the user. The input of this step is the instruction to end the call, and the output is the end of the call and the playing of the warning message.

[0454] Step 9:

[0455] The user receives an automatically terminated call and a warning message, and decides what action to take based on that information. The input to this step is the warning message, and the output is the user's action.

[0456] Step 10:

[0457] The server stores and manages the call analysis results and related data, and periodically updates the generative AI model, allowing it to adapt to new fraud techniques. The inputs to this step are the analysis results and call data, and the outputs are the stored data and an updated AI model.

[0458] (Application example 2)

[0459] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0460] Conventional fraud prevention systems analyze the content of calls and issue a warning if they determine there is a possibility of fraud, but they are unable to take the user's emotional state into account. This makes it difficult to respond when the user is feeling anxious or anxious, and in some cases, they are unable to provide effective protection. Furthermore, as fraud methods evolve daily, delays in system updates have led to the problem of being unable to respond to new fraud patterns.

[0461] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0462] In this invention, the server includes a means for forwarding calls received via a fixed-line communication line to a dedicated calling application, a speech recognition means for converting received call data into text data, a natural language processing means for detecting fraud patterns, a means for terminating calls when a fraudulent call is highly likely, a means for notifying the user of a fraud warning message, an emotion engine for analyzing the user's emotional state, and a means for adjusting the warning message based on the user's emotional state. This not only enables the detection of fraudulent calls but also enables appropriate responses based on the user's emotional state. Furthermore, by using a generative AI model to analyze call content in real time and periodically updating it, the system can quickly respond to new fraud patterns.

[0463] A "fixed communication line" is a communication line that transmits data using a physical cable, and includes telephone lines and internet lines.

[0464] A "dedicated calling application" is an application designed specifically for a specific purpose (in this case, fraudulent call detection) that is installed on a smartphone or other device.

[0465] "Speech recognition means" refers to technology or devices for converting voice data into text data, including high-performance speech recognition APIs.

[0466] "Natural language processing means" refers to technologies and algorithms for analyzing text data and extracting specific patterns and meanings, including NLP libraries.

[0467] "Call termination measures" are features or technologies that automatically disconnect calls when fraud is deemed likely.

[0468] "Means for notifying warning messages" means any technology or device for sending a message to a user indicating a potential fraud, including audio or text notifications.

[0469] The "emotion engine" is a technology that analyzes the tone, speed, emphasis, etc. of a user's voice during a call to assess their emotional state.

[0470] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and make predictions or judgments about new data, including, for example, machine learning algorithms.

[0471] "Analytical means" refers to techniques and devices for processing data and extracting specific patterns or meanings, including data analysis tools and algorithms.

[0472] The present invention relates to a system that redirects calls received via fixed-line communication lines to a dedicated calling application and analyzes the received call data in real time to prevent fraudulent calls. This system can analyze not only the content of the call but also the user's emotional state and adjust warning messages accordingly.

[0473] System configuration

[0474] The system includes fixed-line communications, servers, a dedicated calling application, voice recognition technology, natural language processing technology, an emotion engine, and a generative AI model.

[0475] server

[0476] The server monitors fixed-line communications lines and detects incoming calls. Based on this detection, it prepares to transfer the call data to a dedicated calling application. The server then uses high-performance speech recognition technology, such as the Google Cloud Speech-to-Text API, to convert the call data from voice to text in real time. The converted text data is then analyzed using natural language processing (NLP) technologies, such as SpaCy and NLTK, to detect fraud patterns.

[0477] The server also uses an emotion engine (e.g., Nuance emotion analysis API) to analyze the user's emotional state in real time. If it determines that there is a high possibility of fraud, the server sends an instruction to the terminal to end the call and simultaneously notifies the terminal of a warning message. The warning message is adjusted based on the user's emotional state.

[0478] Terminal

[0479] The device launches a dedicated calling application and receives call data transferred from the server. The device continues to send the received call data to the server and receives analysis results and instructions from the server in real time. If the server determines that there is a high possibility of fraud, the device automatically terminates the call and plays a warning message to the user.

[0480] user

[0481] When a user receives a call on their landline, the call is forwarded to a dedicated calling application. If a potential scam is detected during the call, the call is automatically terminated and a warning message such as "High probability of scam. Please proceed with caution" is played. Depending on the user's emotional state, a more detailed message may be provided.

[0482] For example, if the user is feeling anxious or worried, the message could include specific instructions such as, "This may be a scam. Your call has been automatically terminated due to anxiety in your voice. Please contact a family member and report this to the police."

[0483] This system not only detects fraudulent calls, but also responds appropriately based on the user's emotional state. It also uses a generative AI model to analyze call content in real time and provides regular updates, allowing it to quickly respond to new fraud patterns.

[0484] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0485] Step 1:

[0486] The server monitors the fixed communication line and detects when a call occurs. The input is the call signal from the fixed communication line, and the output is the call data. Specifically, the server detects the call signal, captures the signal, and stores it as digital data.

[0487] Step 2:

[0488] The server prepares to transfer the detected call data to the dedicated call application. The input is the call data obtained in step 1, and the output is a flag indicating that the data is ready for transfer. Specifically, the server converts and compresses the data, and then prepares it for transfer to the dedicated call application.

[0489] Step 3:

[0490] The terminal starts a dedicated calling application and receives call data transferred from the server. The input is the call data from the server, and the output is the received call data. Specifically, the terminal starts the application and receives data from the server using the set communication protocol.

[0491] Step 4:

[0492] The server converts the received call data from voice to text in real time. The input is the call data sent from the device, and the output is the converted text data. Specifically, the server calls the Google Cloud Speech-to-Text API to convert the call voice to text.

[0493] Step 5:

[0494] The server analyzes the converted text data using natural language processing techniques to detect fraud patterns. The input is the text data obtained in step 4, and the output is a score indicating the likelihood of fraud. Specifically, the server uses NLP libraries such as SpaCy and NLTK to analyze suspicious phrases and contexts in the text data.

[0495] Step 6:

[0496] The server uses an emotion engine to analyze the user's emotional state in real time. The input is call data from the device, and the output is an indicator of the emotional state. Specifically, the server uses the Nuance emotion analysis API to analyze the tone, speed, and emphasis of the call audio to evaluate the user's emotional state.

[0497] Step 7:

[0498] The server determines that there is a high possibility of fraud and sends an instruction to terminate the call to the terminal. The input is the score and emotional state obtained in steps 5 and 6, and the output is an instruction to terminate the call. Specifically, the server evaluates the score and emotional state, and if it determines that there is a high possibility of fraud, it sends an instruction to terminate the call to the terminal.

[0499] Step 8:

[0500] The terminal automatically terminates a call based on an instruction received from a server. The input is a call termination instruction from the server, and the output is a call termination action. Specifically, the terminal receives the instruction from the server and terminates the current call through the calling application.

[0501] Step 9:

[0502] The server notifies the user of a fraud warning message. The input is the judgment result obtained in step 7, and the output is a warning message. Specifically, the server generates a message such as "There is a high possibility of fraud. Please proceed with caution" and sends it to the terminal.

[0503] Step 10:

[0504] The terminal plays back to the user the warning message received from the server. The input is the warning message from the server, and the output is the notification to the user. Specifically, the terminal reads out the message received from the server or displays it as text.

[0505] Step 11:

[0506] The server stores all analysis results and related data and periodically updates the generative AI model. The input is the data based on the analysis, and the output is an updated generative AI model. Specifically, the server periodically updates the generative AI model to reflect new fraud patterns.

[0507] These processing steps enable the system to detect fraudulent calls and provide appropriate responses based on the user's emotional state, while regular analysis using generative AI models allows the system to quickly respond to new fraud patterns.

[0508] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0509] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0510] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0511] [Second embodiment]

[0512] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0513] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0514] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0515] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0516] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0517] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0518] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0519] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0520] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0521] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0522] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0523] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0524] MODE FOR CARRYING OUT THE INVENTION

[0525] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system transfers calls received via fixed communication lines to a dedicated calling application and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[0526] 1. Transferring calls from a landline to a dedicated calling app

[0527] server

[0528] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[0529] Terminal

[0530] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[0531] user

[0532] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[0533] 2. Real-time conversion of call content

[0534] server

[0535] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[0536] Terminal

[0537] The terminal continues to send call data to the server.

[0538] 3. Fraud Pattern Detection

[0539] server

[0540] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[0541] Terminal

[0542] The terminal receives the analysis results from the server in real time.

[0543] 4. Call End and Warning Notifications

[0544] server

[0545] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[0546] Terminal

[0547] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0548] user

[0549] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[0550] 5. Call history storage and AI model updates

[0551] server

[0552] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[0553] user

[0554] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0555] Specific examples

[0556] Next, we will explain how this system works in practice using a specific scenario.

[0557] Scenario: The process of detecting fraudulent calls

[0558] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0559] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0560] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0561] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0562] 5. The user receives a warning message and considers appropriate action.

[0563] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[0564] The processing flow will be explained below.

[0565] Step 1:

[0566] The user receives a call on a landline, which is then connected to the server when the call is initiated.

[0567] Step 2:

[0568] The server detects incoming calls via fixed line and, once it has confirmed that a call has been initiated, prepares to transfer the call data to a dedicated calling application.

[0569] Step 3:

[0570] The server transfers the call data to a dedicated call application, which then sends the call data from the server to the device via the application.

[0571] Step 4:

[0572] The terminal starts the dedicated call application and receives the call data transferred from the server, so that the call continues on the terminal.

[0573] Step 5:

[0574] The device sends call data to the server in real time. Through this collaboration, the contents of the call are delivered to the server in real time.

[0575] Step 6:

[0576] The server uses a voice recognition means to convert the received call data into text data in real time.

[0577] Step 7:

[0578] The server then analyzes the converted text data using natural language processing technology, which detects phrases and patterns characteristic of fraud contained in the text data.

[0579] Step 8:

[0580] The server scores the likelihood of fraud and assesses the risk of fraud based on the results. If fraud is deemed likely, the next step is taken.

[0581] Step 9:

[0582] If the server determines that there is a high possibility of fraud, it sends an instruction to the terminal to end the call.

[0583] Step 10:

[0584] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0585] Step 11:

[0586] The user receives a warning message and is alerted to the fraud and considers appropriate action, such as contacting a family member or reporting the incident to the police.

[0587] Step 12:

[0588] The server stores the analysis results and related data, which allows the call history to be stored in a database for future analysis and trend analysis.

[0589] Step 13:

[0590] The server periodically updates the generated AI model so that it can respond to new fraud methods. To maintain analysis accuracy, the model learns the latest fraud information.

[0591] Step 14:

[0592] Users will receive notifications as the system updates, and can review and change their call settings as needed.

[0593] Step 15:

[0594] Servers and devices continue to monitor and analyze to continuously detect fraudulent calls and protect users.

[0595] Example 1

[0596] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0597] In recent years, the number of victims of fraudulent phone calls has been increasing, making it difficult for users to take preventative measures. Elderly people and those who are not familiar with technology in particular have difficulty identifying fraudulent calls, which tends to increase the damage they cause. To address this issue, a system is needed that analyzes the content of calls in real time, quickly detects potentially fraudulent calls, and protects users.

[0598] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0599] In this invention, the server includes a means for transferring calls received via a fixed communication line to dedicated call software, voice recognition software for converting received call data into text data, a natural language processing program for detecting fraud patterns, a means for terminating calls when there is a high possibility of fraud, a means for notifying users of a fraud warning message, and a means for analyzing fraud patterns using a generative AI model. This makes it possible to detect fraudulent calls in real time and quickly and reliably protect users.

[0600] A "fixed line" is a communications infrastructure that transmits data and voice over physical wiring.

[0601] "Purpose-built Call Software" means a call management application that is optimized for a particular use or environment.

[0602] "Speech recognition software" refers to software technology that converts voice data into text data.

[0603] A "natural language processing program" is software that has the ability to understand and analyze human language.

[0604] A "generative AI model" is an artificial intelligence model generated using machine learning and data science techniques.

[0605] "Fraud patterns" are characteristics that identify fraudulent activity based on specific phrases or patterns of behavior.

[0606] "Analytical methods" refer to techniques and methods used to analyze data and draw conclusions.

[0607] A "warning message" is a message that notifies users of risks or dangers.

[0608] The present invention is a system that provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed communication lines to dedicated calling software and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[0609] Transfer from a fixed line to dedicated calling software

[0610] server

[0611] The server monitors the fixed line and detects when a call is made. Based on this detection, the call data is transferred to the dedicated call software. The hardware used is preferably a telephone device for fixed lines (such as a general IP phone). When a call is detected, the call data is sent to the receiving port of the dedicated call software via the TCP / IP protocol.

[0612] Terminal

[0613] The terminal starts the dedicated call software and receives the call data transferred from the server. The call continues on the terminal while the call data continues to be sent to the server in real time. The dedicated call software has an interface for the call and a data transmission function.

[0614] user

[0615] Users receive calls through dedicated calling software, which reduces the risk of fraudulent calls.

[0616] Real-time conversion of call content

[0617] server

[0618] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition software (e.g., Google Cloud Speech-to-Text API). The voice data is sent to the API, which returns the converted text data.

[0619] Fraud Pattern Detection

[0620] server

[0621] The server analyzes the text data using generative AI models (such as BERT), a natural language processing program, to identify phrases and context characteristic of fraud and score the likelihood of fraud.

[0622] Terminal

[0623] The device receives the analysis results from the server in real time, and the analysis result data is sent to the application, which prepares the display and audio notification.

[0624] Call End and Warning Notifications

[0625] server

[0626] If the server determines that there is a high possibility of fraud, it will send an instruction to the device to end the call, along with information to send a warning message to the user.

[0627] Terminal

[0628] The device receives the call termination instruction from the server and automatically terminates the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please be on alert immediately."

[0629] user

[0630] The call will be automatically terminated and the user will receive a warning message, informing them of appropriate action (e.g., contacting family or the police) if necessary.

[0631] Call history storage and AI model updates

[0632] server

[0633] The server stores and manages the call analysis results and related data. The database management system used is a standard relational database (e.g., MySQL). Based on the results, the call history is stored in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud techniques.

[0634] user

[0635] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0636] Specific examples

[0637] Next, we will explain how this system works in practice using a specific scenario.

[0638] Scenario: The process of detecting fraudulent calls

[0639] 1. The user receives a call on a landline, which is then forwarded by the server to dedicated calling software.

[0640] 2. Dedicated calling software on the device receives the call and sends the call data to the server.

[0641] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0642] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0643] 5. The user receives a warning message and considers appropriate action.

[0644] Prompt Sentence Examples

[0645] "To conduct analysis on fraud call detection, please convert the following phone call into text and detect fraud patterns. The call goes like this: 'Hello, this is a bank representative. There has been fraud on your account. Please provide your card details for verification.'"

[0646] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[0647] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0648] Step 1: Call detection and routing

[0649] server

[0650] The server monitors the fixed communication line and detects when a call is made.

[0651] Input: Speech signal

[0652] Specific operation: A server connected to a telephone device for fixed communication lines (a typical IP phone) detects the start of a call.

[0653] Output: Call data (audio data)

[0654] Step 2: Transferring call data to dedicated calling software

[0655] server

[0656] The server transfers the detected call data to dedicated call software.

[0657] Input: Call data

[0658] Specific operation: Call data is sent via the TCP / IP protocol to the receiving port of the dedicated call software.

[0659] Output: Transferred call data

[0660] Step 3: Receiving and sending call data

[0661] Terminal

[0662] The terminal starts dedicated call software and receives the call data transferred from the server.

[0663] Input: Transferred call data

[0664] Specific operation: The dedicated call software starts up and captures the call data from the receiving port.

[0665] Output: Received call data

[0666] The terminal continues to send the received call data to the server in real time.

[0667] Input: Received call data

[0668] Specific operation: The data transmission module transmits the voice data to the server in real time.

[0669] Output: Call data sent

[0670] Step 4: Convert audio data to text

[0671] server

[0672] The server converts the received call data into text data using voice recognition software.

[0673] Input: Call data sent

[0674] Specific operation: The call data is sent to the Google Cloud Speech-to-Text API, and the audio data is converted into text data.

[0675] Output: Converted text data

[0676] Step 5: Fraud pattern analysis of text data

[0677] server

[0678] The server analyzes the text data using a generative AI model.

[0679] Input: Converted text data

[0680] How it works: A natural language processing program using the BERT model analyzes text data and detects fraudulent patterns.

[0681] Output: Analysis results (fraud score)

[0682] Step 6: Notification of analysis results

[0683] server

[0684] If the server determines that there is a high possibility of fraud, it sends a call termination instruction and a warning message to the terminal.

[0685] Input: Analysis result (fraud score)

[0686] Specific operation: If the fraud score exceeds a certain threshold, generate call termination instructions and warning message data and send them to the terminal.

[0687] Output: End call prompt and warning message

[0688] Step 7: Ending the call and warning notifications

[0689] Terminal

[0690] The terminal receives a call end instruction from the server and automatically ends the call.

[0691] Input: End call instruction

[0692] Specific action: The dedicated call software ends the call.

[0693] Output: Call ended

[0694] It then plays a warning message to the user.

[0695] Input: Warning message information

[0696] What happens: A warning message is played.

[0697] Output: The warning message played

[0698] Step 8: Consider the best course of action

[0699] user

[0700] The user can review the automatically ended call and the warning message that is played and consider appropriate action (e.g., contacting family or reporting to the police).

[0701] Input: The warning message played

[0702] Specific Action: The user hears the warning message and decides what to do next.

[0703] Output: Considered solutions

[0704] Step 9: Save call history and update the model

[0705] server

[0706] The server stores the call analysis results and related data for future analysis, and regularly updates the generative AI model to adapt to new fraud techniques.

[0707] Input: Call analysis results and related data

[0708] How it works: A database management system stores the analysis results, and the AI ​​model is periodically retrained.

[0709] Output: Saved data and updated AI model

[0710] Step 10: Update Notification

[0711] user

[0712] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0713] Input: Update notification

[0714] Specific behavior: The user checks the notification and reviews the settings.

[0715] Output: Updated system settings

[0716] (Application example 1)

[0717] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0718] With traditional calls made over fixed-line communications, it was difficult to detect the risk of fraudulent calls in real time, putting users at high risk of falling victim to fraud. Similar issues existed with mobile communications and voice over IP (VoIP), and a comprehensive solution was needed.

[0719] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0720] In this invention, the server includes means for transferring calls received via fixed communication lines to a dedicated call application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating calls when there is a high possibility of fraud, means for notifying the user of a fraud warning message, server communication means for performing speech recognition on the voice of the call content in real time on the smartphone and detecting fraud patterns, and means for issuing a warning to the user's smartphone based on the results of scoring the possibility of fraud. This makes it possible to detect fraud risks not only on fixed communication lines but also on mobile lines and internet calls in real time and protect users.

[0721] A "fixed line" is a communications infrastructure for transmitting voice data over physical telephone lines or optical fiber.

[0722] A "dedicated calling application" is software designed for a specific purpose, and is an application for managing and analyzing call data.

[0723] "Speech recognition means" refers to technology or equipment for converting voice data into text data, and generally includes speech recognition software and hardware devices.

[0724] "Natural language processing means" refers to technologies and devices for analyzing text data and detecting fraudulent patterns, and generally includes natural language processing software and algorithms.

[0725] "Call termination measures" refers to technology or devices that automatically disconnect a call if it is determined that fraud is likely.

[0726] "Means for notifying users of fraud warning messages" refers to technologies or devices used to warn users of potentially fraudulent calls, and generally includes notification and alert systems.

[0727] "Voice recognition means" refers to technology or devices that use a smartphone's microphone to capture voice data and convert it into text.

[0728] "Server communication means" refers to the technology or device used to send voice data or text data to the server and receive the analysis results.

[0729] "Means for detecting fraud patterns" refers to technologies and devices that identify fraud patterns in real time based on analyzed text data.

[0730] "Fraud likelihood scoring result" refers to the evaluation result that quantifies the risk of fraud based on the analysis results.

[0731] "Warning Means" means any technology or device that delivers a warning message to a user in audio, text, or visual form.

[0732] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the calls in real time. Specific embodiments of this system are described in detail below.

[0733] 1. Transferring calls from a landline to a dedicated calling app

[0734] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[0735] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[0736] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[0737] 2. Real-time conversion of call content

[0738] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition methods, specifically the speech_recognition library and the Google Speech Recognition API.

[0739] The terminal continues to send call data to the server.

[0740] 3. Fraud Pattern Detection

[0741] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[0742] The device receives analysis results in real time from the server, which uses a generative AI model for analysis and updates it regularly.

[0743] 4. Call End and Warning Notifications

[0744] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[0745] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0746] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[0747] 5. Call history storage and AI model updates

[0748] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[0749] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[0750] Specific examples

[0751] Next, we will explain how this system works in practice using a specific scenario.

[0752] Scenario: The process of detecting fraudulent calls

[0753] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0754] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0755] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0756] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0757] 5. The user receives a warning message and considers appropriate action.

[0758] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[0759] Example prompt using a generative AI model:

[0760] "Call text: {text of the call}

[0761] Is this call a fraud risk? Generate an appropriate fraud risk score."

[0762] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0763] Step 1:

[0764] When a call is made via a fixed line, the server detects the call and transfers it to a dedicated calling application. The input is the call signal from the fixed line, and the output is the call data transferred to the dedicated calling application. The server monitors the call content and converts the call data into an appropriate format using a dedicated protocol before sending it.

[0765] Step 2:

[0766] A terminal running a dedicated calling application receives call data transferred from the server. The input is the call data sent from the server, and the output is the call content played on the terminal. The terminal analyzes the received data in real time and continues the call.

[0767] Step 3:

[0768] The device continuously transmits the received call data to the server in real time. The input is the voice data during the call, and the output is the voice data transmitted to the server. The device uses a high-performance voice recognition engine to convert the voice into digital data and transmits it to the server.

[0769] Step 4:

[0770] The server converts the voice data received from the device into text data using a voice recognition method. The input is the voice data from the device, and the output is text data. The server analyzes the voice using a voice recognition library (e.g., speech_recognition library) and an external API (e.g., Google Speech Recognition API) and converts it into text.

[0771] Step 5:

[0772] The server uses natural language processing (NLP) technology on the text data to detect fraud patterns. The input is the text data, and the output is a fraud likelihood score. The server uses a generative AI model to analyze characteristic phrases and context contained in the text data and quantify the likelihood of fraud.

[0773] Step 6:

[0774] If the server determines that there is a high possibility of fraud based on the fraud probability score, it sends an instruction to end the call to the terminal. The input is the fraud probability score, and the output is an instruction to end the call. The server checks the fraud judgment criteria, and if the fraud risk is high, it immediately issues an instruction to end the call.

[0775] Step 7:

[0776] The terminal receives a call termination instruction from the server and automatically terminates the call. The input is the call termination instruction, and the output is disconnecting the call and playing a warning message. The terminal terminates the call and displays or plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0777] Step 8:

[0778] The user receives a warning message and considers the appropriate response. The input is the warning message, and the output is the user's action. The user checks the warning message and, if necessary, contacts a family member or calls the police.

[0779] Step 9:

[0780] The server stores and manages the analysis results and related data for calls. The input is the analysis results and call data, and the output is historical data stored in a database. The server uses this data for future analysis and to respond to new fraud methods.

[0781] Step 10:

[0782] The generative AI model is regularly updated to improve its accuracy in order to respond to the latest fraud techniques. The input is new fraud patterns and analytical data, and the output is an updated generative AI model. The server reflects the new data and improves the model's performance.

[0783] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0784] MODE FOR CARRYING OUT THE INVENTION

[0785] The present invention provides an AI calling service that combines an emotion engine to prevent damage caused by fraudulent phone calls. This system transfers calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the call in real time. It also analyzes the user's emotional state and responds accordingly. Detailed modes for implementing the present invention are described below.

[0786] 1. Transferring calls from a landline to a dedicated calling app

[0787] server

[0788] The server monitors the fixed communication line and detects when a call is made. Based on this detection, it prepares to transfer the call data to the dedicated calling application. The call data is transferred from the server to the dedicated calling application and then sent to the terminal via the application.

[0789] Terminal

[0790] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[0791] user

[0792] By receiving calls through a dedicated calling application, users can limit their use of free calling apps and reduce the risk of fraudulent calls.

[0793] 2. Real-time conversion of call content

[0794] server

[0795] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[0796] Terminal

[0797] The terminal continues to send call data to the server.

[0798] 3. Fraud Pattern Detection

[0799] server

[0800] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts of fraud, and scoring the likelihood of fraud.

[0801] Terminal

[0802] The terminal receives the analysis results from the server in real time.

[0803] 4. Call End and Warning Notifications

[0804] server

[0805] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[0806] Terminal

[0807] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0808] user

[0809] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[0810] 5. Emotion analysis using an emotion engine

[0811] server

[0812] The server is equipped with an emotion engine that analyzes not only the content of the call but also the user's emotional state. This emotion engine analyzes the tone, speed, emphasis, etc. of the user's voice during the call to assess the user's emotional state in real time.

[0813] Terminal

[0814] The device sends emotion data along with the call data to the server, which uses this data to perform emotion analysis.

[0815] 6. Emotion-based warnings and instructions

[0816] server

[0817] The server adjusts the warning message based on the user's emotional state, for example, playing a more detailed and reassuring message if the user is showing signs of impatience or anxiety.

[0818] Terminal

[0819] The terminal plays an appropriate warning message to the user based on instructions from the server.

[0820] 7. Call history storage and AI model updates

[0821] server

[0822] The server stores and manages the analysis results and related data of calls. This allows the call history to be stored in a database for future analysis and trend analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraudulent techniques.

[0823] user

[0824] Users will receive notifications as their system is updated and can review and change their settings as needed.

[0825] Specific examples

[0826] Next, we will explain how this system works in practice using a specific scenario.

[0827] Scenario: Detecting fraudulent calls and taking emotions into account

[0828] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0829] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0830] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0831] 4. The server uses the emotion engine to evaluate the user's emotional state in real time.

[0832] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0833] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[0834] 7. The server stores the call analysis results and related data and updates the generative AI model.

[0835] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[0836] The processing flow will be explained below.

[0837] Step 1:

[0838] A user receives a call on a landline. When the call begins, an initial signal is sent from the landline to the server.

[0839] Step 2:

[0840] The server detects when a call is made over a fixed line and, based on this detection, instructs the server to transfer the call data to a dedicated calling application.

[0841] Step 3:

[0842] The server forwards the call data to the dedicated calling application. In this process, the call data is sent from the server to the application.

[0843] Step 4:

[0844] The device starts a dedicated call application and receives the call data transferred from the server, and the device makes the received call available to the user.

[0845] Step 5:

[0846] The device continuously transmits call data to the server. The voice data of the call is sent to the server in real time.

[0847] Step 6:

[0848] The server converts the call data into text data using a speech recognition means, using a high-performance speech recognition engine for this purpose.

[0849] Step 7:

[0850] The server analyzes the text data using natural language processing technology to detect fraudulent patterns and characteristic phrases within the text data.

[0851] Step 8:

[0852] The server scores the likelihood of fraud and assesses the risk of the call based on the result. If it determines that fraud is likely, it proceeds to the next step.

[0853] Step 9:

[0854] The server activates an emotion engine to recognize the user's emotions, which analyzes the user's voice tone, speed, emphasis, etc. in real time.

[0855] Step 10:

[0856] The server evaluates the user's emotional state based on the analysis results of the emotion engine. If the user shows signs of impatience or anxiety, it is determined that there is a high possibility of fraud.

[0857] Step 11:

[0858] If the server determines that there is a high possibility of fraud, it will send an instruction to end the call to the terminal, which will automatically end the call.

[0859] Step 12:

[0860] The device receives the server's instructions, automatically ends the call, and then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[0861] Step 13:

[0862] The user receives a warning message, and upon receiving the warning, the user considers appropriate action, such as contacting a family member or reporting the incident to the police.

[0863] Step 14:

[0864] The server stores the call data, analysis results, and emotion data, which can be used for future analysis and data analysis.

[0865] Step 15:

[0866] The server regularly updates the emotion engine and generative AI model to adapt to new scams and emotional patterns.

[0867] Step 16:

[0868] Users will receive notifications about system updates and can check and change their settings as needed, ensuring they always have the latest fraud prevention system.

[0869] Example 2

[0870] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0871] Fraudulent phone calls are a malicious method used to illegally obtain personal information and cause financial damage. The elderly are particularly targeted, and the damage caused by these calls is becoming widespread. However, existing technologies lack effective means for detecting and preventing fraudulent calls. In particular, it has been difficult for existing technologies to detect fraud in real time and take appropriate action while taking into account the emotional state of the user during the call.

[0872] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0873] In this invention, the server includes means for transferring calls received via a fixed communication line to a dedicated calling application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating the call if there is a high possibility of fraud, means for notifying the user of a fraud warning message, emotion analysis means for evaluating the user's emotional state during the call in real time, means for adjusting the warning message based on the user's emotional state, and means for saving and managing the call analysis results and related data. This reduces the risk of fraudulent calls and makes it possible to effectively prevent fraud while providing users with a sense of security.

[0874] "Fixed line" refers to infrastructure for wired communications, including the Internet and the public switched telephone network.

[0875] A "dedicated calling application" is application software designed to process specific calling data.

[0876] A "speech recognition means" is a computer program or algorithm for converting speech data into text.

[0877] "Natural language processing means" is a technology for analyzing text data and understanding its meaning.

[0878] A "pattern of fraud" is a characteristic phrase or context that suggests fraudulent activity.

[0879] A "high likelihood of fraud" means that the content and context of the call indicate a high risk of fraud.

[0880] A "warning message" is a notification intended to alert the user.

[0881] "Emotion analysis means" is a technique for assessing a user's emotional state.

[0882] A "generative AI model" is a model created based on artificial intelligence techniques and used to perform specific tasks.

[0883] "Text data" is data expressed as character information.

[0884] "Analysis results" are information or conclusions obtained based on the analysis of data.

[0885] "Related data" is additional information or metadata related to the analysis results.

[0886] "Preservation" is the act of keeping data in digital form.

[0887] "Management" is the act of keeping stored data available efficiently and effectively.

[0888] MODE FOR CARRYING OUT THE INVENTION

[0889] The present invention provides an AI call service system that combines an emotion analysis engine to prevent damage caused by fraudulent phone calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the call content and the user's emotional state in real time. Detailed embodiments of this system are described below.

[0890] Hardware and Software Configuration

[0891] server

[0892] The server monitors the fixed line and detects when a call is made. When a call is detected, the server performs the following actions:

[0893] 1. Transfer of call data: The server captures call data from fixed communication lines and transfers it to a dedicated call application in real time, buffering the data and converting its format.

[0894] 2. Speech recognition: The server converts the received call data into text using a high-performance speech recognition system (e.g., Google Cloud Speech-to-Text API), which allows the voice content to be analyzed as text information.

[0895] 3. Natural Language Processing (NLP): The server analyzes the text data using natural language processing techniques (e.g., Python's SpaCy or NLTK libraries) to detect fraud patterns, identify phrases and contexts characteristic of specific frauds, and assign a score to the likelihood of fraud.

[0896] 4. Emotion Analysis: The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to assess the user's emotional state. It analyzes acoustic features such as tone, rate, and emphasis of the user's voice to estimate the user's emotional state in real time.

[0897] 5. Warning message generation: If a fraudulent activity is deemed likely, the server generates a warning message for the user. The warning message is tailored based on the user's emotional state to provide a sense of security.

[0898] Terminal

[0899] The device runs a dedicated call application and receives and processes the call data sent from the server. The process on the device is as follows:

[0900] 1. Continuing the call: The dedicated calling application transfers the call data received from the server to the audio output device, allowing the user to continue the call.

[0901] 2. Data transmission: The device continuously transmits voice data and emotional characteristic data to the server in real time during the call. This transmitted data is important for analysis.

[0902] 3. Warning Implementation: If the server determines that there is a high possibility of fraud, the terminal will automatically terminate the call and play a warning message to the user. The warning message will be played through the audio output device.

[0903] user

[0904] Users interact with the system as follows:

[0905] 1. Receiving calls: Receive calls via landline and continue the call through a dedicated calling application, limiting the use of free calling apps and reducing the risk of fraudulent calls.

[0906] 2. Warning Response: If the system detects possible fraud, the user will receive a warning message and consider taking appropriate action (e.g., contacting family, reporting to the police).

[0907] Specific examples

[0908] Next, we will explain how this system works in practice using a specific scenario.

[0909] Scenario: Detecting fraudulent calls and taking emotions into account

[0910] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[0911] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[0912] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[0913] 4. The server uses an emotion analysis engine to assess the user's emotional state in real time.

[0914] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[0915] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[0916] 7. The server stores the call analysis results and related data and updates the generative AI model.

[0917] Prompt Sentence Examples

[0918] By inputting the following prompt sentences into the generative AI model, we can get answers about the detailed operation of this system.

[0919] "Please explain the role of the server and terminal in the fraudulent call detection system."

[0920] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[0921] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0922] Step 1:

[0923] The server monitors the fixed line and detects when a call is made. When a call is started, the server prepares to acquire the call data. Specific operations include capturing voice data and allocating a buffer. The input of this step is the call signal from the fixed line, and the output is the acquisition of the ready call data.

[0924] Step 2:

[0925] The server captures call data from fixed communication lines and transfers it to the dedicated calling application in real time. The call data is buffered and format converted. The input of this step is the captured call data, and the output is the data transferred to the dedicated calling application.

[0926] Step 3:

[0927] The terminal starts a dedicated call application and receives the call data sent from the server. The received call data is sent to the audio output device, allowing the user to continue the call. The input of this step is the call data from the server, and the output is data sent to the audio output device.

[0928] Step 4:

[0929] The terminal continues to send voice data during the call to the server in real time. The server converts the received call data into text using a high-performance speech recognition system. The input of this step is the call data and the speech recognition system, and the output is text data.

[0930] Step 5:

[0931] The server then analyzes the converted text data using natural language processing techniques, specifically analyzing phrases and context to detect fraud patterns. The input for this step is the text data, and the output is a scoring result indicating the likelihood of fraud.

[0932] Step 6:

[0933] The server uses an emotion analysis engine to evaluate the user's emotional state in real time. For this analysis, acoustic features such as tone, rate, and emphasis of the voice data are analyzed. The input of this step is the call data and acoustic features, and the output is the evaluation result of the emotional state.

[0934] Step 7:

[0935] If the server determines that there is a high possibility of fraud, it sends a call termination instruction to the terminal. The instruction includes a warning message for the user. The input of this step is the fraud scoring result and the emotion evaluation result, and the output is the call termination instruction and the warning message.

[0936] Step 8:

[0937] The terminal receives the instruction from the server and automatically ends the call. After the call ends, the terminal plays a warning message to notify the user. The input of this step is the instruction to end the call, and the output is the end of the call and the playing of the warning message.

[0938] Step 9:

[0939] The user receives an automatically terminated call and a warning message, and decides what action to take based on that information. The input to this step is the warning message, and the output is the user's action.

[0940] Step 10:

[0941] The server stores and manages the call analysis results and related data, and periodically updates the generative AI model, allowing it to adapt to new fraud techniques. The inputs to this step are the analysis results and call data, and the outputs are the stored data and an updated AI model.

[0942] (Application example 2)

[0943] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0944] Conventional fraud prevention systems analyze the content of calls and issue a warning if they determine there is a possibility of fraud, but they are unable to take the user's emotional state into account. This makes it difficult to respond when the user is feeling anxious or anxious, and in some cases, they are unable to provide effective protection. Furthermore, as fraud methods evolve daily, delays in system updates have led to the problem of being unable to respond to new fraud patterns.

[0945] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0946] In this invention, the server includes a means for forwarding calls received via a fixed-line communication line to a dedicated calling application, a speech recognition means for converting received call data into text data, a natural language processing means for detecting fraud patterns, a means for terminating calls when a fraudulent call is highly likely, a means for notifying the user of a fraud warning message, an emotion engine for analyzing the user's emotional state, and a means for adjusting the warning message based on the user's emotional state. This not only enables the detection of fraudulent calls but also enables appropriate responses based on the user's emotional state. Furthermore, by using a generative AI model to analyze call content in real time and periodically updating it, the system can quickly respond to new fraud patterns.

[0947] A "fixed communication line" is a communication line that transmits data using a physical cable, and includes telephone lines and internet lines.

[0948] A "dedicated calling application" is an application designed specifically for a specific purpose (in this case, fraudulent call detection) that is installed on a smartphone or other device.

[0949] "Speech recognition means" refers to technology or devices for converting voice data into text data, including high-performance speech recognition APIs.

[0950] "Natural language processing means" refers to technologies and algorithms for analyzing text data and extracting specific patterns and meanings, including NLP libraries.

[0951] "Call termination measures" are features or technologies that automatically disconnect calls when fraud is deemed likely.

[0952] "Means for notifying warning messages" means any technology or device for sending a message to a user indicating a potential fraud, including audio or text notifications.

[0953] The "emotion engine" is a technology that analyzes the tone, speed, emphasis, etc. of a user's voice during a call to assess their emotional state.

[0954] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and make predictions or judgments about new data, including, for example, machine learning algorithms.

[0955] "Analytical means" refers to techniques and devices for processing data and extracting specific patterns or meanings, including data analysis tools and algorithms.

[0956] The present invention relates to a system that redirects calls received via fixed-line communication lines to a dedicated calling application and analyzes the received call data in real time to prevent fraudulent calls. This system can analyze not only the content of the call but also the user's emotional state and adjust warning messages accordingly.

[0957] System configuration

[0958] The system includes fixed-line communications, servers, a dedicated calling application, voice recognition technology, natural language processing technology, an emotion engine, and a generative AI model.

[0959] server

[0960] The server monitors fixed-line communications lines and detects incoming calls. Based on this detection, it prepares to transfer the call data to a dedicated calling application. The server then uses high-performance speech recognition technology, such as the Google Cloud Speech-to-Text API, to convert the call data from voice to text in real time. The converted text data is then analyzed using natural language processing (NLP) technologies, such as SpaCy and NLTK, to detect fraud patterns.

[0961] The server also uses an emotion engine (e.g., Nuance emotion analysis API) to analyze the user's emotional state in real time. If it determines that there is a high possibility of fraud, the server sends an instruction to the terminal to end the call and simultaneously notifies the terminal of a warning message. The warning message is adjusted based on the user's emotional state.

[0962] Terminal

[0963] The device launches a dedicated calling application and receives call data transferred from the server. The device continues to send the received call data to the server and receives analysis results and instructions from the server in real time. If the server determines that there is a high possibility of fraud, the device automatically terminates the call and plays a warning message to the user.

[0964] user

[0965] When a user receives a call on their landline, the call is forwarded to a dedicated calling application. If a potential scam is detected during the call, the call is automatically terminated and a warning message such as "High probability of scam. Please proceed with caution" is played. Depending on the user's emotional state, a more detailed message may be provided.

[0966] For example, if the user is feeling anxious or worried, the message could include specific instructions such as, "This may be a scam. Your call has been automatically terminated due to anxiety in your voice. Please contact a family member and report this to the police."

[0967] This system not only detects fraudulent calls, but also responds appropriately based on the user's emotional state. It also uses a generative AI model to analyze call content in real time and provides regular updates, allowing it to quickly respond to new fraud patterns.

[0968] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0969] Step 1:

[0970] The server monitors the fixed communication line and detects when a call occurs. The input is the call signal from the fixed communication line, and the output is the call data. Specifically, the server detects the call signal, captures the signal, and stores it as digital data.

[0971] Step 2:

[0972] The server prepares to transfer the detected call data to the dedicated call application. The input is the call data obtained in step 1, and the output is a flag indicating that the data is ready for transfer. Specifically, the server converts and compresses the data, and then prepares it for transfer to the dedicated call application.

[0973] Step 3:

[0974] The terminal starts a dedicated calling application and receives call data transferred from the server. The input is the call data from the server, and the output is the received call data. Specifically, the terminal starts the application and receives data from the server using the set communication protocol.

[0975] Step 4:

[0976] The server converts the received call data from voice to text in real time. The input is the call data sent from the device, and the output is the converted text data. Specifically, the server calls the Google Cloud Speech-to-Text API to convert the call voice to text.

[0977] Step 5:

[0978] The server analyzes the converted text data using natural language processing techniques to detect fraud patterns. The input is the text data obtained in step 4, and the output is a score indicating the likelihood of fraud. Specifically, the server uses NLP libraries such as SpaCy and NLTK to analyze suspicious phrases and contexts in the text data.

[0979] Step 6:

[0980] The server uses an emotion engine to analyze the user's emotional state in real time. The input is call data from the device, and the output is an indicator of the emotional state. Specifically, the server uses the Nuance emotion analysis API to analyze the tone, speed, and emphasis of the call audio to evaluate the user's emotional state.

[0981] Step 7:

[0982] The server determines that there is a high possibility of fraud and sends an instruction to terminate the call to the terminal. The input is the score and emotional state obtained in steps 5 and 6, and the output is an instruction to terminate the call. Specifically, the server evaluates the score and emotional state, and if it determines that there is a high possibility of fraud, it sends an instruction to terminate the call to the terminal.

[0983] Step 8:

[0984] The terminal automatically terminates a call based on an instruction received from a server. The input is a call termination instruction from the server, and the output is a call termination action. Specifically, the terminal receives the instruction from the server and terminates the current call through the calling application.

[0985] Step 9:

[0986] The server notifies the user of a fraud warning message. The input is the judgment result obtained in step 7, and the output is a warning message. Specifically, the server generates a message such as "There is a high possibility of fraud. Please proceed with caution" and sends it to the terminal.

[0987] Step 10:

[0988] The terminal plays back to the user the warning message received from the server. The input is the warning message from the server, and the output is the notification to the user. Specifically, the terminal reads out the message received from the server or displays it as text.

[0989] Step 11:

[0990] The server stores all analysis results and related data and periodically updates the generative AI model. The input is the data based on the analysis, and the output is an updated generative AI model. Specifically, the server periodically updates the generative AI model to reflect new fraud patterns.

[0991] These processing steps enable the system to detect fraudulent calls and provide appropriate responses based on the user's emotional state, while regular analysis using generative AI models allows the system to quickly respond to new fraud patterns.

[0992] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0993] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0994] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0995] [Third embodiment]

[0996] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0997] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0998] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0999] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1000] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1001] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1002] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1003] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1004] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1005] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1006] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1007] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1008] MODE FOR CARRYING OUT THE INVENTION

[1009] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system transfers calls received via fixed communication lines to a dedicated calling application and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[1010] 1. Transferring calls from a landline to a dedicated calling app

[1011] server

[1012] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[1013] Terminal

[1014] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[1015] user

[1016] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[1017] 2. Real-time conversion of call content

[1018] server

[1019] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[1020] Terminal

[1021] The terminal continues to send call data to the server.

[1022] 3. Fraud Pattern Detection

[1023] server

[1024] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[1025] Terminal

[1026] The terminal receives the analysis results from the server in real time.

[1027] 4. Call End and Warning Notifications

[1028] server

[1029] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[1030] Terminal

[1031] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1032] user

[1033] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[1034] 5. Call history storage and AI model updates

[1035] server

[1036] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[1037] user

[1038] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1039] Specific examples

[1040] Next, we will explain how this system works in practice using a specific scenario.

[1041] Scenario: The process of detecting fraudulent calls

[1042] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1043] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1044] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1045] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1046] 5. The user receives a warning message and considers appropriate action.

[1047] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[1048] The processing flow will be explained below.

[1049] Step 1:

[1050] The user receives a call on a landline, which is then connected to the server when the call is initiated.

[1051] Step 2:

[1052] The server detects incoming calls via fixed line and, once it has confirmed that a call has been initiated, prepares to transfer the call data to a dedicated calling application.

[1053] Step 3:

[1054] The server transfers the call data to a dedicated call application, which then sends the call data from the server to the device via the application.

[1055] Step 4:

[1056] The terminal starts the dedicated call application and receives the call data transferred from the server, so that the call continues on the terminal.

[1057] Step 5:

[1058] The device sends call data to the server in real time. Through this collaboration, the contents of the call are delivered to the server in real time.

[1059] Step 6:

[1060] The server uses a voice recognition means to convert the received call data into text data in real time.

[1061] Step 7:

[1062] The server then analyzes the converted text data using natural language processing technology, which detects phrases and patterns characteristic of fraud contained in the text data.

[1063] Step 8:

[1064] The server scores the likelihood of fraud and assesses the risk of fraud based on the results. If fraud is deemed likely, the next step is taken.

[1065] Step 9:

[1066] If the server determines that there is a high possibility of fraud, it sends an instruction to the terminal to end the call.

[1067] Step 10:

[1068] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1069] Step 11:

[1070] The user receives a warning message and is alerted to the fraud and considers appropriate action, such as contacting a family member or reporting the incident to the police.

[1071] Step 12:

[1072] The server stores the analysis results and related data, which allows the call history to be stored in a database for future analysis and trend analysis.

[1073] Step 13:

[1074] The server periodically updates the generated AI model so that it can respond to new fraud methods. To maintain analysis accuracy, the model learns the latest fraud information.

[1075] Step 14:

[1076] Users will receive notifications as the system updates, and can review and change their call settings as needed.

[1077] Step 15:

[1078] Servers and devices continue to monitor and analyze to continuously detect fraudulent calls and protect users.

[1079] Example 1

[1080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1081] In recent years, the number of victims of fraudulent phone calls has been increasing, making it difficult for users to take preventative measures. Elderly people and those who are not familiar with technology in particular have difficulty identifying fraudulent calls, which tends to increase the damage they cause. To address this issue, a system is needed that analyzes the content of calls in real time, quickly detects potentially fraudulent calls, and protects users.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1083] In this invention, the server includes a means for transferring calls received via a fixed communication line to dedicated call software, voice recognition software for converting received call data into text data, a natural language processing program for detecting fraud patterns, a means for terminating calls when there is a high possibility of fraud, a means for notifying users of a fraud warning message, and a means for analyzing fraud patterns using a generative AI model. This makes it possible to detect fraudulent calls in real time and quickly and reliably protect users.

[1084] A "fixed line" is a communications infrastructure that transmits data and voice over physical wiring.

[1085] "Purpose-built Call Software" means a call management application that is optimized for a particular use or environment.

[1086] "Speech recognition software" refers to software technology that converts voice data into text data.

[1087] A "natural language processing program" is software that has the ability to understand and analyze human language.

[1088] A "generative AI model" is an artificial intelligence model generated using machine learning and data science techniques.

[1089] "Fraud patterns" are characteristics that identify fraudulent activity based on specific phrases or patterns of behavior.

[1090] "Analytical methods" refer to techniques and methods used to analyze data and draw conclusions.

[1091] A "warning message" is a message that notifies users of risks or dangers.

[1092] The present invention is a system that provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed communication lines to dedicated calling software and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[1093] Transfer from a fixed line to dedicated calling software

[1094] server

[1095] The server monitors the fixed line and detects when a call is made. Based on this detection, the call data is transferred to the dedicated call software. The hardware used is preferably a telephone device for fixed lines (such as a general IP phone). When a call is detected, the call data is sent to the receiving port of the dedicated call software via the TCP / IP protocol.

[1096] Terminal

[1097] The terminal starts the dedicated call software and receives the call data transferred from the server. The call continues on the terminal while the call data continues to be sent to the server in real time. The dedicated call software has an interface for the call and a data transmission function.

[1098] user

[1099] Users receive calls through dedicated calling software, which reduces the risk of fraudulent calls.

[1100] Real-time conversion of call content

[1101] server

[1102] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition software (e.g., Google Cloud Speech-to-Text API). The voice data is sent to the API, which returns the converted text data.

[1103] Fraud Pattern Detection

[1104] server

[1105] The server analyzes the text data using generative AI models (such as BERT), a natural language processing program, to identify phrases and context characteristic of fraud and score the likelihood of fraud.

[1106] Terminal

[1107] The device receives the analysis results from the server in real time, and the analysis result data is sent to the application, which prepares the display and audio notification.

[1108] Call End and Warning Notifications

[1109] server

[1110] If the server determines that there is a high possibility of fraud, it will send an instruction to the device to end the call, along with information to send a warning message to the user.

[1111] Terminal

[1112] The device receives the call termination instruction from the server and automatically terminates the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please be on alert immediately."

[1113] user

[1114] The call will be automatically terminated and the user will receive a warning message, informing them of appropriate action (e.g., contacting family or the police) if necessary.

[1115] Call history storage and AI model updates

[1116] server

[1117] The server stores and manages the call analysis results and related data. The database management system used is a standard relational database (e.g., MySQL). Based on the results, the call history is stored in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud techniques.

[1118] user

[1119] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1120] Specific examples

[1121] Next, we will explain how this system works in practice using a specific scenario.

[1122] Scenario: The process of detecting fraudulent calls

[1123] 1. The user receives a call on a landline, which is then forwarded by the server to dedicated calling software.

[1124] 2. Dedicated calling software on the device receives the call and sends the call data to the server.

[1125] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1126] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1127] 5. The user receives a warning message and considers appropriate action.

[1128] Prompt Sentence Examples

[1129] "To conduct analysis on fraud call detection, please convert the following phone call into text and detect fraud patterns. The call goes like this: 'Hello, this is a bank representative. There has been fraud on your account. Please provide your card details for verification.'"

[1130] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[1131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1132] Step 1: Call detection and routing

[1133] server

[1134] The server monitors the fixed communication line and detects when a call is made.

[1135] Input: Speech signal

[1136] Specific operation: A server connected to a telephone device for fixed communication lines (a typical IP phone) detects the start of a call.

[1137] Output: Call data (audio data)

[1138] Step 2: Transferring call data to dedicated calling software

[1139] server

[1140] The server transfers the detected call data to dedicated call software.

[1141] Input: Call data

[1142] Specific operation: Call data is sent via the TCP / IP protocol to the receiving port of the dedicated call software.

[1143] Output: Transferred call data

[1144] Step 3: Receiving and sending call data

[1145] Terminal

[1146] The terminal starts dedicated call software and receives the call data transferred from the server.

[1147] Input: Transferred call data

[1148] Specific operation: The dedicated call software starts up and captures the call data from the receiving port.

[1149] Output: Received call data

[1150] The terminal continues to send the received call data to the server in real time.

[1151] Input: Received call data

[1152] Specific operation: The data transmission module transmits the voice data to the server in real time.

[1153] Output: Call data sent

[1154] Step 4: Convert audio data to text

[1155] server

[1156] The server converts the received call data into text data using voice recognition software.

[1157] Input: Call data sent

[1158] Specific operation: The call data is sent to the Google Cloud Speech-to-Text API, and the audio data is converted into text data.

[1159] Output: Converted text data

[1160] Step 5: Fraud pattern analysis of text data

[1161] server

[1162] The server analyzes the text data using a generative AI model.

[1163] Input: Converted text data

[1164] How it works: A natural language processing program using the BERT model analyzes text data and detects fraudulent patterns.

[1165] Output: Analysis results (fraud score)

[1166] Step 6: Notification of analysis results

[1167] server

[1168] If the server determines that there is a high possibility of fraud, it sends a call termination instruction and a warning message to the terminal.

[1169] Input: Analysis result (fraud score)

[1170] Specific operation: If the fraud score exceeds a certain threshold, generate call termination instructions and warning message data and send them to the terminal.

[1171] Output: End call prompt and warning message

[1172] Step 7: Ending the call and warning notifications

[1173] Terminal

[1174] The terminal receives a call end instruction from the server and automatically ends the call.

[1175] Input: End call instruction

[1176] Specific action: The dedicated call software ends the call.

[1177] Output: Call ended

[1178] It then plays a warning message to the user.

[1179] Input: Warning message information

[1180] What happens: A warning message is played.

[1181] Output: The warning message played

[1182] Step 8: Consider the best course of action

[1183] user

[1184] The user can review the automatically ended call and the warning message that is played and consider appropriate action (e.g., contacting family or reporting to the police).

[1185] Input: The warning message played

[1186] Specific Action: The user hears the warning message and decides what to do next.

[1187] Output: Considered solutions

[1188] Step 9: Save call history and update the model

[1189] server

[1190] The server stores the call analysis results and related data for future analysis, and regularly updates the generative AI model to adapt to new fraud techniques.

[1191] Input: Call analysis results and related data

[1192] How it works: A database management system stores the analysis results, and the AI ​​model is periodically retrained.

[1193] Output: Saved data and updated AI model

[1194] Step 10: Update Notification

[1195] user

[1196] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1197] Input: Update notification

[1198] Specific behavior: The user checks the notification and reviews the settings.

[1199] Output: Updated system settings

[1200] (Application example 1)

[1201] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1202] With traditional calls made over fixed-line communications, it was difficult to detect the risk of fraudulent calls in real time, putting users at high risk of falling victim to fraud. Similar issues existed with mobile communications and voice over IP (VoIP), and a comprehensive solution was needed.

[1203] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1204] In this invention, the server includes means for transferring calls received via fixed communication lines to a dedicated call application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating calls when there is a high possibility of fraud, means for notifying the user of a fraud warning message, server communication means for performing speech recognition on the voice of the call content in real time on the smartphone and detecting fraud patterns, and means for issuing a warning to the user's smartphone based on the results of scoring the possibility of fraud. This makes it possible to detect fraud risks not only on fixed communication lines but also on mobile lines and internet calls in real time and protect users.

[1205] A "fixed line" is a communications infrastructure for transmitting voice data over physical telephone lines or optical fiber.

[1206] A "dedicated calling application" is software designed for a specific purpose, and is an application for managing and analyzing call data.

[1207] "Speech recognition means" refers to technology or equipment for converting voice data into text data, and generally includes speech recognition software and hardware devices.

[1208] "Natural language processing means" refers to technologies and devices for analyzing text data and detecting fraudulent patterns, and generally includes natural language processing software and algorithms.

[1209] "Call termination measures" refers to technology or devices that automatically disconnect a call if it is determined that fraud is likely.

[1210] "Means for notifying users of fraud warning messages" refers to technologies or devices used to warn users of potentially fraudulent calls, and generally includes notification and alert systems.

[1211] "Voice recognition means" refers to technology or devices that use a smartphone's microphone to capture voice data and convert it into text.

[1212] "Server communication means" refers to the technology or device used to send voice data or text data to the server and receive the analysis results.

[1213] "Means for detecting fraud patterns" refers to technologies and devices that identify fraud patterns in real time based on analyzed text data.

[1214] "Fraud likelihood scoring result" refers to the evaluation result that quantifies the risk of fraud based on the analysis results.

[1215] "Warning Means" means any technology or device that delivers a warning message to a user in audio, text, or visual form.

[1216] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the calls in real time. Specific embodiments of this system are described in detail below.

[1217] 1. Transferring calls from a landline to a dedicated calling app

[1218] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[1219] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[1220] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[1221] 2. Real-time conversion of call content

[1222] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition methods, specifically the speech_recognition library and the Google Speech Recognition API.

[1223] The terminal continues to send call data to the server.

[1224] 3. Fraud Pattern Detection

[1225] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[1226] The device receives analysis results in real time from the server, which uses a generative AI model for analysis and updates it regularly.

[1227] 4. Call End and Warning Notifications

[1228] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[1229] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1230] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[1231] 5. Call history storage and AI model updates

[1232] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[1233] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1234] Specific examples

[1235] Next, we will explain how this system works in practice using a specific scenario.

[1236] Scenario: The process of detecting fraudulent calls

[1237] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1238] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1239] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1240] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1241] 5. The user receives a warning message and considers appropriate action.

[1242] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[1243] Example prompt using a generative AI model:

[1244] "Call text: {text of the call}

[1245] Is this call a fraud risk? Generate an appropriate fraud risk score."

[1246] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1247] Step 1:

[1248] When a call is made via a fixed line, the server detects the call and transfers it to a dedicated calling application. The input is the call signal from the fixed line, and the output is the call data transferred to the dedicated calling application. The server monitors the call content and converts the call data into an appropriate format using a dedicated protocol before sending it.

[1249] Step 2:

[1250] A terminal running a dedicated calling application receives call data transferred from the server. The input is the call data sent from the server, and the output is the call content played on the terminal. The terminal analyzes the received data in real time and continues the call.

[1251] Step 3:

[1252] The device continuously transmits the received call data to the server in real time. The input is the voice data during the call, and the output is the voice data transmitted to the server. The device uses a high-performance voice recognition engine to convert the voice into digital data and transmits it to the server.

[1253] Step 4:

[1254] The server converts the voice data received from the device into text data using a voice recognition method. The input is the voice data from the device, and the output is text data. The server analyzes the voice using a voice recognition library (e.g., speech_recognition library) and an external API (e.g., Google Speech Recognition API) and converts it into text.

[1255] Step 5:

[1256] The server uses natural language processing (NLP) technology on the text data to detect fraud patterns. The input is the text data, and the output is a fraud likelihood score. The server uses a generative AI model to analyze characteristic phrases and context contained in the text data and quantify the likelihood of fraud.

[1257] Step 6:

[1258] If the server determines that there is a high possibility of fraud based on the fraud probability score, it sends an instruction to end the call to the terminal. The input is the fraud probability score, and the output is an instruction to end the call. The server checks the fraud judgment criteria, and if the fraud risk is high, it immediately issues an instruction to end the call.

[1259] Step 7:

[1260] The terminal receives a call termination instruction from the server and automatically terminates the call. The input is the call termination instruction, and the output is disconnecting the call and playing a warning message. The terminal terminates the call and displays or plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1261] Step 8:

[1262] The user receives a warning message and considers the appropriate response. The input is the warning message, and the output is the user's action. The user checks the warning message and, if necessary, contacts a family member or calls the police.

[1263] Step 9:

[1264] The server stores and manages the analysis results and related data for calls. The input is the analysis results and call data, and the output is historical data stored in a database. The server uses this data for future analysis and to respond to new fraud methods.

[1265] Step 10:

[1266] The generative AI model is regularly updated to improve its accuracy in order to respond to the latest fraud techniques. The input is new fraud patterns and analytical data, and the output is an updated generative AI model. The server reflects the new data and improves the model's performance.

[1267] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1268] MODE FOR CARRYING OUT THE INVENTION

[1269] The present invention provides an AI calling service that combines an emotion engine to prevent damage caused by fraudulent phone calls. This system transfers calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the call in real time. It also analyzes the user's emotional state and responds accordingly. Detailed modes for implementing the present invention are described below.

[1270] 1. Transferring calls from a landline to a dedicated calling app

[1271] server

[1272] The server monitors the fixed communication line and detects when a call is made. Based on this detection, it prepares to transfer the call data to the dedicated calling application. The call data is transferred from the server to the dedicated calling application and then sent to the terminal via the application.

[1273] Terminal

[1274] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[1275] user

[1276] By receiving calls through a dedicated calling application, users can limit their use of free calling apps and reduce the risk of fraudulent calls.

[1277] 2. Real-time conversion of call content

[1278] server

[1279] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[1280] Terminal

[1281] The terminal continues to send call data to the server.

[1282] 3. Fraud Pattern Detection

[1283] server

[1284] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts of fraud, and scoring the likelihood of fraud.

[1285] Terminal

[1286] The terminal receives the analysis results from the server in real time.

[1287] 4. Call End and Warning Notifications

[1288] server

[1289] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[1290] Terminal

[1291] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1292] user

[1293] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[1294] 5. Emotion analysis using an emotion engine

[1295] server

[1296] The server is equipped with an emotion engine that analyzes not only the content of the call but also the user's emotional state. This emotion engine analyzes the tone, speed, emphasis, etc. of the user's voice during the call to assess the user's emotional state in real time.

[1297] Terminal

[1298] The device sends emotion data along with the call data to the server, which uses this data to perform emotion analysis.

[1299] 6. Emotion-based warnings and instructions

[1300] server

[1301] The server adjusts the warning message based on the user's emotional state, for example, playing a more detailed and reassuring message if the user is showing signs of impatience or anxiety.

[1302] Terminal

[1303] The terminal plays an appropriate warning message to the user based on instructions from the server.

[1304] 7. Call history storage and AI model updates

[1305] server

[1306] The server stores and manages the analysis results and related data of calls. This allows the call history to be stored in a database for future analysis and trend analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraudulent techniques.

[1307] user

[1308] Users will receive notifications as their system is updated and can review and change their settings as needed.

[1309] Specific examples

[1310] Next, we will explain how this system works in practice using a specific scenario.

[1311] Scenario: Detecting fraudulent calls and taking emotions into account

[1312] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1313] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1314] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1315] 4. The server uses the emotion engine to evaluate the user's emotional state in real time.

[1316] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1317] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[1318] 7. The server stores the call analysis results and related data and updates the generative AI model.

[1319] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[1320] The processing flow will be explained below.

[1321] Step 1:

[1322] A user receives a call on a landline. When the call begins, an initial signal is sent from the landline to the server.

[1323] Step 2:

[1324] The server detects when a call is made over a fixed line and, based on this detection, instructs the server to transfer the call data to a dedicated calling application.

[1325] Step 3:

[1326] The server forwards the call data to the dedicated calling application. In this process, the call data is sent from the server to the application.

[1327] Step 4:

[1328] The device starts a dedicated call application and receives the call data transferred from the server, and the device makes the received call available to the user.

[1329] Step 5:

[1330] The device continuously transmits call data to the server. The voice data of the call is sent to the server in real time.

[1331] Step 6:

[1332] The server converts the call data into text data using a speech recognition means, using a high-performance speech recognition engine for this purpose.

[1333] Step 7:

[1334] The server analyzes the text data using natural language processing technology to detect fraudulent patterns and characteristic phrases within the text data.

[1335] Step 8:

[1336] The server scores the likelihood of fraud and assesses the risk of the call based on the result. If it determines that fraud is likely, it proceeds to the next step.

[1337] Step 9:

[1338] The server activates an emotion engine to recognize the user's emotions, which analyzes the user's voice tone, speed, emphasis, etc. in real time.

[1339] Step 10:

[1340] The server evaluates the user's emotional state based on the analysis results of the emotion engine. If the user shows signs of impatience or anxiety, it is determined that there is a high possibility of fraud.

[1341] Step 11:

[1342] If the server determines that there is a high possibility of fraud, it will send an instruction to end the call to the terminal, which will automatically end the call.

[1343] Step 12:

[1344] The device receives the server's instructions, automatically ends the call, and then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1345] Step 13:

[1346] The user receives a warning message, and upon receiving the warning, the user considers appropriate action, such as contacting a family member or reporting the incident to the police.

[1347] Step 14:

[1348] The server stores the call data, analysis results, and emotion data, which can be used for future analysis and data analysis.

[1349] Step 15:

[1350] The server regularly updates the emotion engine and generative AI model to adapt to new scams and emotional patterns.

[1351] Step 16:

[1352] Users will receive notifications about system updates and can check and change their settings as needed, ensuring they always have the latest fraud prevention system.

[1353] Example 2

[1354] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1355] Fraudulent phone calls are a malicious method used to illegally obtain personal information and cause financial damage. The elderly are particularly targeted, and the damage caused by these calls is becoming widespread. However, existing technologies lack effective means for detecting and preventing fraudulent calls. In particular, it has been difficult for existing technologies to detect fraud in real time and take appropriate action while taking into account the emotional state of the user during the call.

[1356] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1357] In this invention, the server includes means for transferring calls received via a fixed communication line to a dedicated calling application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating the call if there is a high possibility of fraud, means for notifying the user of a fraud warning message, emotion analysis means for evaluating the user's emotional state during the call in real time, means for adjusting the warning message based on the user's emotional state, and means for saving and managing the call analysis results and related data. This reduces the risk of fraudulent calls and makes it possible to effectively prevent fraud while providing users with a sense of security.

[1358] "Fixed line" refers to infrastructure for wired communications, including the Internet and the public switched telephone network.

[1359] A "dedicated calling application" is application software designed to process specific calling data.

[1360] A "speech recognition means" is a computer program or algorithm for converting speech data into text.

[1361] "Natural language processing means" is a technology for analyzing text data and understanding its meaning.

[1362] A "pattern of fraud" is a characteristic phrase or context that suggests fraudulent activity.

[1363] A "high likelihood of fraud" means that the content and context of the call indicate a high risk of fraud.

[1364] A "warning message" is a notification intended to alert the user.

[1365] "Emotion analysis means" is a technique for assessing a user's emotional state.

[1366] A "generative AI model" is a model created based on artificial intelligence techniques and used to perform specific tasks.

[1367] "Text data" is data expressed as character information.

[1368] "Analysis results" are information or conclusions obtained based on the analysis of data.

[1369] "Related data" is additional information or metadata related to the analysis results.

[1370] "Preservation" is the act of keeping data in digital form.

[1371] "Management" is the act of keeping stored data available efficiently and effectively.

[1372] MODE FOR CARRYING OUT THE INVENTION

[1373] The present invention provides an AI call service system that combines an emotion analysis engine to prevent damage caused by fraudulent phone calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the call content and the user's emotional state in real time. Detailed embodiments of this system are described below.

[1374] Hardware and Software Configuration

[1375] server

[1376] The server monitors the fixed line and detects when a call is made. When a call is detected, the server performs the following actions:

[1377] 1. Transfer of call data: The server captures call data from fixed communication lines and transfers it to a dedicated call application in real time, buffering the data and converting its format.

[1378] 2. Speech recognition: The server converts the received call data into text using a high-performance speech recognition system (e.g., Google Cloud Speech-to-Text API), which allows the voice content to be analyzed as text information.

[1379] 3. Natural Language Processing (NLP): The server analyzes the text data using natural language processing techniques (e.g., Python's SpaCy or NLTK libraries) to detect fraud patterns, identify phrases and contexts characteristic of specific frauds, and assign a score to the likelihood of fraud.

[1380] 4. Emotion Analysis: The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to assess the user's emotional state. It analyzes acoustic features such as tone, rate, and emphasis of the user's voice to estimate the user's emotional state in real time.

[1381] 5. Warning message generation: If a fraudulent activity is deemed likely, the server generates a warning message for the user. The warning message is tailored based on the user's emotional state to provide a sense of security.

[1382] Terminal

[1383] The device runs a dedicated call application and receives and processes the call data sent from the server. The process on the device is as follows:

[1384] 1. Continuing the call: The dedicated calling application transfers the call data received from the server to the audio output device, allowing the user to continue the call.

[1385] 2. Data transmission: The device continuously transmits voice data and emotional characteristic data to the server in real time during the call. This transmitted data is important for analysis.

[1386] 3. Warning Implementation: If the server determines that there is a high possibility of fraud, the terminal will automatically terminate the call and play a warning message to the user. The warning message will be played through the audio output device.

[1387] user

[1388] Users interact with the system as follows:

[1389] 1. Receiving calls: Receive calls via landline and continue the call through a dedicated calling application, limiting the use of free calling apps and reducing the risk of fraudulent calls.

[1390] 2. Warning Response: If the system detects possible fraud, the user will receive a warning message and consider taking appropriate action (e.g., contacting family, reporting to the police).

[1391] Specific examples

[1392] Next, we will explain how this system works in practice using a specific scenario.

[1393] Scenario: Detecting fraudulent calls and taking emotions into account

[1394] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1395] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1396] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1397] 4. The server uses an emotion analysis engine to assess the user's emotional state in real time.

[1398] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1399] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[1400] 7. The server stores the call analysis results and related data and updates the generative AI model.

[1401] Prompt Sentence Examples

[1402] By inputting the following prompt sentences into the generative AI model, we can get answers about the detailed operation of this system.

[1403] "Please explain the role of the server and terminal in the fraudulent call detection system."

[1404] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[1405] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1406] Step 1:

[1407] The server monitors the fixed line and detects when a call is made. When a call is started, the server prepares to acquire the call data. Specific operations include capturing voice data and allocating a buffer. The input of this step is the call signal from the fixed line, and the output is the acquisition of the ready call data.

[1408] Step 2:

[1409] The server captures call data from fixed communication lines and transfers it to the dedicated calling application in real time. The call data is buffered and format converted. The input of this step is the captured call data, and the output is the data transferred to the dedicated calling application.

[1410] Step 3:

[1411] The terminal starts a dedicated call application and receives the call data sent from the server. The received call data is sent to the audio output device, allowing the user to continue the call. The input of this step is the call data from the server, and the output is data sent to the audio output device.

[1412] Step 4:

[1413] The terminal continues to send voice data during the call to the server in real time. The server converts the received call data into text using a high-performance speech recognition system. The input of this step is the call data and the speech recognition system, and the output is text data.

[1414] Step 5:

[1415] The server then analyzes the converted text data using natural language processing techniques, specifically analyzing phrases and context to detect fraud patterns. The input for this step is the text data, and the output is a scoring result indicating the likelihood of fraud.

[1416] Step 6:

[1417] The server uses an emotion analysis engine to evaluate the user's emotional state in real time. For this analysis, acoustic features such as tone, rate, and emphasis of the voice data are analyzed. The input of this step is the call data and acoustic features, and the output is the evaluation result of the emotional state.

[1418] Step 7:

[1419] If the server determines that there is a high possibility of fraud, it sends a call termination instruction to the terminal. The instruction includes a warning message for the user. The input of this step is the fraud scoring result and the emotion evaluation result, and the output is the call termination instruction and the warning message.

[1420] Step 8:

[1421] The terminal receives the instruction from the server and automatically ends the call. After the call ends, the terminal plays a warning message to notify the user. The input of this step is the instruction to end the call, and the output is the end of the call and the playing of the warning message.

[1422] Step 9:

[1423] The user receives an automatically terminated call and a warning message, and decides what action to take based on that information. The input to this step is the warning message, and the output is the user's action.

[1424] Step 10:

[1425] The server stores and manages the call analysis results and related data, and periodically updates the generative AI model, allowing it to adapt to new fraud techniques. The inputs to this step are the analysis results and call data, and the outputs are the stored data and an updated AI model.

[1426] (Application example 2)

[1427] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1428] Conventional fraud prevention systems analyze the content of calls and issue a warning if they determine there is a possibility of fraud, but they are unable to take the user's emotional state into account. This makes it difficult to respond when the user is feeling anxious or anxious, and in some cases, they are unable to provide effective protection. Furthermore, as fraud methods evolve daily, delays in system updates have led to the problem of being unable to respond to new fraud patterns.

[1429] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1430] In this invention, the server includes a means for forwarding calls received via a fixed-line communication line to a dedicated calling application, a speech recognition means for converting received call data into text data, a natural language processing means for detecting fraud patterns, a means for terminating calls when a fraudulent call is highly likely, a means for notifying the user of a fraud warning message, an emotion engine for analyzing the user's emotional state, and a means for adjusting the warning message based on the user's emotional state. This not only enables the detection of fraudulent calls but also enables appropriate responses based on the user's emotional state. Furthermore, by using a generative AI model to analyze call content in real time and periodically updating it, the system can quickly respond to new fraud patterns.

[1431] A "fixed communication line" is a communication line that transmits data using a physical cable, and includes telephone lines and internet lines.

[1432] A "dedicated calling application" is an application designed specifically for a specific purpose (in this case, fraudulent call detection) that is installed on a smartphone or other device.

[1433] "Speech recognition means" refers to technology or devices for converting voice data into text data, including high-performance speech recognition APIs.

[1434] "Natural language processing means" refers to technologies and algorithms for analyzing text data and extracting specific patterns and meanings, including NLP libraries.

[1435] "Call termination measures" are features or technologies that automatically disconnect calls when fraud is deemed likely.

[1436] "Means for notifying warning messages" means any technology or device for sending a message to a user indicating a potential fraud, including audio or text notifications.

[1437] The "emotion engine" is a technology that analyzes the tone, speed, emphasis, etc. of a user's voice during a call to assess their emotional state.

[1438] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and make predictions or judgments about new data, including, for example, machine learning algorithms.

[1439] "Analytical means" refers to techniques and devices for processing data and extracting specific patterns or meanings, including data analysis tools and algorithms.

[1440] The present invention relates to a system that redirects calls received via fixed-line communication lines to a dedicated calling application and analyzes the received call data in real time to prevent fraudulent calls. This system can analyze not only the content of the call but also the user's emotional state and adjust warning messages accordingly.

[1441] System configuration

[1442] The system includes fixed-line communications, servers, a dedicated calling application, voice recognition technology, natural language processing technology, an emotion engine, and a generative AI model.

[1443] server

[1444] The server monitors fixed-line communications lines and detects incoming calls. Based on this detection, it prepares to transfer the call data to a dedicated calling application. The server then uses high-performance speech recognition technology, such as the Google Cloud Speech-to-Text API, to convert the call data from voice to text in real time. The converted text data is then analyzed using natural language processing (NLP) technologies, such as SpaCy and NLTK, to detect fraud patterns.

[1445] The server also uses an emotion engine (e.g., Nuance emotion analysis API) to analyze the user's emotional state in real time. If it determines that there is a high possibility of fraud, the server sends an instruction to the terminal to end the call and simultaneously notifies the terminal of a warning message. The warning message is adjusted based on the user's emotional state.

[1446] Terminal

[1447] The device launches a dedicated calling application and receives call data transferred from the server. The device continues to send the received call data to the server and receives analysis results and instructions from the server in real time. If the server determines that there is a high possibility of fraud, the device automatically terminates the call and plays a warning message to the user.

[1448] user

[1449] When a user receives a call on their landline, the call is forwarded to a dedicated calling application. If a potential scam is detected during the call, the call is automatically terminated and a warning message such as "High probability of scam. Please proceed with caution" is played. Depending on the user's emotional state, a more detailed message may be provided.

[1450] For example, if the user is feeling anxious or worried, the message could include specific instructions such as, "This may be a scam. Your call has been automatically terminated due to anxiety in your voice. Please contact a family member and report this to the police."

[1451] This system not only detects fraudulent calls, but also responds appropriately based on the user's emotional state. It also uses a generative AI model to analyze call content in real time and provides regular updates, allowing it to quickly respond to new fraud patterns.

[1452] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1453] Step 1:

[1454] The server monitors the fixed communication line and detects when a call occurs. The input is the call signal from the fixed communication line, and the output is the call data. Specifically, the server detects the call signal, captures the signal, and stores it as digital data.

[1455] Step 2:

[1456] The server prepares to transfer the detected call data to the dedicated call application. The input is the call data obtained in step 1, and the output is a flag indicating that the data is ready for transfer. Specifically, the server converts and compresses the data, and then prepares it for transfer to the dedicated call application.

[1457] Step 3:

[1458] The terminal starts a dedicated calling application and receives call data transferred from the server. The input is the call data from the server, and the output is the received call data. Specifically, the terminal starts the application and receives data from the server using the set communication protocol.

[1459] Step 4:

[1460] The server converts the received call data from voice to text in real time. The input is the call data sent from the device, and the output is the converted text data. Specifically, the server calls the Google Cloud Speech-to-Text API to convert the call voice to text.

[1461] Step 5:

[1462] The server analyzes the converted text data using natural language processing techniques to detect fraud patterns. The input is the text data obtained in step 4, and the output is a score indicating the likelihood of fraud. Specifically, the server uses NLP libraries such as SpaCy and NLTK to analyze suspicious phrases and contexts in the text data.

[1463] Step 6:

[1464] The server uses an emotion engine to analyze the user's emotional state in real time. The input is call data from the device, and the output is an indicator of the emotional state. Specifically, the server uses the Nuance emotion analysis API to analyze the tone, speed, and emphasis of the call audio to evaluate the user's emotional state.

[1465] Step 7:

[1466] The server determines that there is a high possibility of fraud and sends an instruction to terminate the call to the terminal. The input is the score and emotional state obtained in steps 5 and 6, and the output is an instruction to terminate the call. Specifically, the server evaluates the score and emotional state, and if it determines that there is a high possibility of fraud, it sends an instruction to terminate the call to the terminal.

[1467] Step 8:

[1468] The terminal automatically terminates a call based on an instruction received from a server. The input is a call termination instruction from the server, and the output is a call termination action. Specifically, the terminal receives the instruction from the server and terminates the current call through the calling application.

[1469] Step 9:

[1470] The server notifies the user of a fraud warning message. The input is the judgment result obtained in step 7, and the output is a warning message. Specifically, the server generates a message such as "There is a high possibility of fraud. Please proceed with caution" and sends it to the terminal.

[1471] Step 10:

[1472] The terminal plays back to the user the warning message received from the server. The input is the warning message from the server, and the output is the notification to the user. Specifically, the terminal reads out the message received from the server or displays it as text.

[1473] Step 11:

[1474] The server stores all analysis results and related data and periodically updates the generative AI model. The input is the data based on the analysis, and the output is an updated generative AI model. Specifically, the server periodically updates the generative AI model to reflect new fraud patterns.

[1475] These processing steps enable the system to detect fraudulent calls and provide appropriate responses based on the user's emotional state, while regular analysis using generative AI models allows the system to quickly respond to new fraud patterns.

[1476] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1478] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1479] [Fourth embodiment]

[1480] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1481] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1482] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1483] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1484] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1485] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1486] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1487] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1488] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1491] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1492] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1493] MODE FOR CARRYING OUT THE INVENTION

[1494] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system transfers calls received via fixed communication lines to a dedicated calling application and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[1495] 1. Transferring calls from a landline to a dedicated calling app

[1496] server

[1497] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[1498] Terminal

[1499] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[1500] user

[1501] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[1502] 2. Real-time conversion of call content

[1503] server

[1504] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[1505] Terminal

[1506] The terminal continues to send call data to the server.

[1507] 3. Fraud Pattern Detection

[1508] server

[1509] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[1510] Terminal

[1511] The terminal receives the analysis results from the server in real time.

[1512] 4. Call End and Warning Notifications

[1513] server

[1514] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[1515] Terminal

[1516] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1517] user

[1518] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[1519] 5. Call history storage and AI model updates

[1520] server

[1521] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[1522] user

[1523] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1524] Specific examples

[1525] Next, we will explain how this system works in practice using a specific scenario.

[1526] Scenario: The process of detecting fraudulent calls

[1527] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1528] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1529] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1530] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1531] 5. The user receives a warning message and considers appropriate action.

[1532] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[1533] The processing flow will be explained below.

[1534] Step 1:

[1535] The user receives a call on a landline, which is then connected to the server when the call is initiated.

[1536] Step 2:

[1537] The server detects incoming calls via fixed line and, once it has confirmed that a call has been initiated, prepares to transfer the call data to a dedicated calling application.

[1538] Step 3:

[1539] The server transfers the call data to a dedicated call application, which then sends the call data from the server to the device via the application.

[1540] Step 4:

[1541] The terminal starts the dedicated call application and receives the call data transferred from the server, so that the call continues on the terminal.

[1542] Step 5:

[1543] The device sends call data to the server in real time. Through this collaboration, the contents of the call are delivered to the server in real time.

[1544] Step 6:

[1545] The server uses a voice recognition means to convert the received call data into text data in real time.

[1546] Step 7:

[1547] The server then analyzes the converted text data using natural language processing technology, which detects phrases and patterns characteristic of fraud contained in the text data.

[1548] Step 8:

[1549] The server scores the likelihood of fraud and assesses the risk of fraud based on the results. If fraud is deemed likely, the next step is taken.

[1550] Step 9:

[1551] If the server determines that there is a high possibility of fraud, it sends an instruction to the terminal to end the call.

[1552] Step 10:

[1553] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1554] Step 11:

[1555] The user receives a warning message and is alerted to the fraud and considers appropriate action, such as contacting a family member or reporting the incident to the police.

[1556] Step 12:

[1557] The server stores the analysis results and related data, which allows the call history to be stored in a database for future analysis and trend analysis.

[1558] Step 13:

[1559] The server periodically updates the generated AI model so that it can respond to new fraud methods. To maintain analysis accuracy, the model learns the latest fraud information.

[1560] Step 14:

[1561] Users will receive notifications as the system updates, and can review and change their call settings as needed.

[1562] Step 15:

[1563] Servers and devices continue to monitor and analyze to continuously detect fraudulent calls and protect users.

[1564] Example 1

[1565] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1566] In recent years, the number of victims of fraudulent phone calls has been increasing, making it difficult for users to take preventative measures. Elderly people and those who are not familiar with technology in particular have difficulty identifying fraudulent calls, which tends to increase the damage they cause. To address this issue, a system is needed that analyzes the content of calls in real time, quickly detects potentially fraudulent calls, and protects users.

[1567] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1568] In this invention, the server includes a means for transferring calls received via a fixed communication line to dedicated call software, voice recognition software for converting received call data into text data, a natural language processing program for detecting fraud patterns, a means for terminating calls when there is a high possibility of fraud, a means for notifying users of a fraud warning message, and a means for analyzing fraud patterns using a generative AI model. This makes it possible to detect fraudulent calls in real time and quickly and reliably protect users.

[1569] A "fixed line" is a communications infrastructure that transmits data and voice over physical wiring.

[1570] "Purpose-built Call Software" means a call management application that is optimized for a particular use or environment.

[1571] "Speech recognition software" refers to software technology that converts voice data into text data.

[1572] A "natural language processing program" is software that has the ability to understand and analyze human language.

[1573] A "generative AI model" is an artificial intelligence model generated using machine learning and data science techniques.

[1574] "Fraud patterns" are characteristics that identify fraudulent activity based on specific phrases or patterns of behavior.

[1575] "Analytical methods" refer to techniques and methods used to analyze data and draw conclusions.

[1576] A "warning message" is a message that notifies users of risks or dangers.

[1577] The present invention is a system that provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed communication lines to dedicated calling software and analyzes the content of the calls in real time. An embodiment of this system is described in detail below.

[1578] Transfer from a fixed line to dedicated calling software

[1579] server

[1580] The server monitors the fixed line and detects when a call is made. Based on this detection, the call data is transferred to the dedicated call software. The hardware used is preferably a telephone device for fixed lines (such as a general IP phone). When a call is detected, the call data is sent to the receiving port of the dedicated call software via the TCP / IP protocol.

[1581] Terminal

[1582] The terminal starts the dedicated call software and receives the call data transferred from the server. The call continues on the terminal while the call data continues to be sent to the server in real time. The dedicated call software has an interface for the call and a data transmission function.

[1583] user

[1584] Users receive calls through dedicated calling software, which reduces the risk of fraudulent calls.

[1585] Real-time conversion of call content

[1586] server

[1587] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition software (e.g., Google Cloud Speech-to-Text API). The voice data is sent to the API, which returns the converted text data.

[1588] Fraud Pattern Detection

[1589] server

[1590] The server analyzes the text data using generative AI models (such as BERT), a natural language processing program, to identify phrases and context characteristic of fraud and score the likelihood of fraud.

[1591] Terminal

[1592] The device receives the analysis results from the server in real time, and the analysis result data is sent to the application, which prepares the display and audio notification.

[1593] Call End and Warning Notifications

[1594] server

[1595] If the server determines that there is a high possibility of fraud, it will send an instruction to the device to end the call, along with information to send a warning message to the user.

[1596] Terminal

[1597] The device receives the call termination instruction from the server and automatically terminates the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please be on alert immediately."

[1598] user

[1599] The call will be automatically terminated and the user will receive a warning message, informing them of appropriate action (e.g., contacting family or the police) if necessary.

[1600] Call history storage and AI model updates

[1601] server

[1602] The server stores and manages the call analysis results and related data. The database management system used is a standard relational database (e.g., MySQL). Based on the results, the call history is stored in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud techniques.

[1603] user

[1604] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1605] Specific examples

[1606] Next, we will explain how this system works in practice using a specific scenario.

[1607] Scenario: The process of detecting fraudulent calls

[1608] 1. The user receives a call on a landline, which is then forwarded by the server to dedicated calling software.

[1609] 2. Dedicated calling software on the device receives the call and sends the call data to the server.

[1610] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1611] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1612] 5. The user receives a warning message and considers appropriate action.

[1613] Prompt Sentence Examples

[1614] "To conduct analysis on fraud call detection, please convert the following phone call into text and detect fraud patterns. The call goes like this: 'Hello, this is a bank representative. There has been fraud on your account. Please provide your card details for verification.'"

[1615] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[1616] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1617] Step 1: Call detection and routing

[1618] server

[1619] The server monitors the fixed communication line and detects when a call is made.

[1620] Input: Speech signal

[1621] Specific operation: A server connected to a telephone device for fixed communication lines (a typical IP phone) detects the start of a call.

[1622] Output: Call data (audio data)

[1623] Step 2: Transferring call data to dedicated calling software

[1624] server

[1625] The server transfers the detected call data to dedicated call software.

[1626] Input: Call data

[1627] Specific operation: Call data is sent via the TCP / IP protocol to the receiving port of the dedicated call software.

[1628] Output: Transferred call data

[1629] Step 3: Receiving and sending call data

[1630] Terminal

[1631] The terminal starts dedicated call software and receives the call data transferred from the server.

[1632] Input: Transferred call data

[1633] Specific operation: The dedicated call software starts up and captures the call data from the receiving port.

[1634] Output: Received call data

[1635] The terminal continues to send the received call data to the server in real time.

[1636] Input: Received call data

[1637] Specific operation: The data transmission module transmits the voice data to the server in real time.

[1638] Output: Call data sent

[1639] Step 4: Convert audio data to text

[1640] server

[1641] The server converts the received call data into text data using voice recognition software.

[1642] Input: Call data sent

[1643] Specific operation: The call data is sent to the Google Cloud Speech-to-Text API, and the audio data is converted into text data.

[1644] Output: Converted text data

[1645] Step 5: Fraud pattern analysis of text data

[1646] server

[1647] The server analyzes the text data using a generative AI model.

[1648] Input: Converted text data

[1649] How it works: A natural language processing program using the BERT model analyzes text data and detects fraudulent patterns.

[1650] Output: Analysis results (fraud score)

[1651] Step 6: Notification of analysis results

[1652] server

[1653] If the server determines that there is a high possibility of fraud, it sends a call termination instruction and a warning message to the terminal.

[1654] Input: Analysis result (fraud score)

[1655] Specific operation: If the fraud score exceeds a certain threshold, generate call termination instructions and warning message data and send them to the terminal.

[1656] Output: End call prompt and warning message

[1657] Step 7: Ending the call and warning notifications

[1658] Terminal

[1659] The terminal receives a call end instruction from the server and automatically ends the call.

[1660] Input: End call instruction

[1661] Specific action: The dedicated call software ends the call.

[1662] Output: Call ended

[1663] It then plays a warning message to the user.

[1664] Input: Warning message information

[1665] What happens: A warning message is played.

[1666] Output: The warning message played

[1667] Step 8: Consider the best course of action

[1668] user

[1669] The user can review the automatically ended call and the warning message that is played and consider appropriate action (e.g., contacting family or reporting to the police).

[1670] Input: The warning message played

[1671] Specific Action: The user hears the warning message and decides what to do next.

[1672] Output: Considered solutions

[1673] Step 9: Save call history and update the model

[1674] server

[1675] The server stores the call analysis results and related data for future analysis, and regularly updates the generative AI model to adapt to new fraud techniques.

[1676] Input: Call analysis results and related data

[1677] How it works: A database management system stores the analysis results, and the AI ​​model is periodically retrained.

[1678] Output: Saved data and updated AI model

[1679] Step 10: Update Notification

[1680] user

[1681] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1682] Input: Update notification

[1683] Specific behavior: The user checks the notification and reviews the settings.

[1684] Output: Updated system settings

[1685] (Application example 1)

[1686] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1687] With traditional calls made over fixed-line communications, it was difficult to detect the risk of fraudulent calls in real time, putting users at high risk of falling victim to fraud. Similar issues existed with mobile communications and voice over IP (VoIP), and a comprehensive solution was needed.

[1688] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1689] In this invention, the server includes means for transferring calls received via fixed communication lines to a dedicated call application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating calls when there is a high possibility of fraud, means for notifying the user of a fraud warning message, server communication means for performing speech recognition on the voice of the call content in real time on the smartphone and detecting fraud patterns, and means for issuing a warning to the user's smartphone based on the results of scoring the possibility of fraud. This makes it possible to detect fraud risks not only on fixed communication lines but also on mobile lines and internet calls in real time and protect users.

[1690] A "fixed line" is a communications infrastructure for transmitting voice data over physical telephone lines or optical fiber.

[1691] A "dedicated calling application" is software designed for a specific purpose, and is an application for managing and analyzing call data.

[1692] "Speech recognition means" refers to technology or equipment for converting voice data into text data, and generally includes speech recognition software and hardware devices.

[1693] "Natural language processing means" refers to technologies and devices for analyzing text data and detecting fraudulent patterns, and generally includes natural language processing software and algorithms.

[1694] "Call termination measures" refers to technology or devices that automatically disconnect a call if it is determined that fraud is likely.

[1695] "Means for notifying users of fraud warning messages" refers to technologies or devices used to warn users of potentially fraudulent calls, and generally includes notification and alert systems.

[1696] "Voice recognition means" refers to technology or devices that use a smartphone's microphone to capture voice data and convert it into text.

[1697] "Server communication means" refers to the technology or device used to send voice data or text data to the server and receive the analysis results.

[1698] "Means for detecting fraud patterns" refers to technologies and devices that identify fraud patterns in real time based on analyzed text data.

[1699] "Fraud likelihood scoring result" refers to the evaluation result that quantifies the risk of fraud based on the analysis results.

[1700] "Warning Means" means any technology or device that delivers a warning message to a user in audio, text, or visual form.

[1701] The present invention provides a comprehensive AI calling service to protect users from fraudulent calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the calls in real time. Specific embodiments of this system are described in detail below.

[1702] 1. Transferring calls from a landline to a dedicated calling app

[1703] The server monitors the fixed line and detects when a call is made. Based on this detection, the server transfers the call data to a dedicated calling application, so that all calls are processed via the dedicated calling application.

[1704] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[1705] Users receive calls through a dedicated calling application, which limits the use of free calling apps and reduces the risk of fraudulent calls.

[1706] 2. Real-time conversion of call content

[1707] The server converts the call data received from the device from voice to text in real time using high-performance speech recognition methods, specifically the speech_recognition library and the Google Speech Recognition API.

[1708] The terminal continues to send call data to the server.

[1709] 3. Fraud Pattern Detection

[1710] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts that indicate fraud, and scoring the likelihood of fraud.

[1711] The device receives analysis results in real time from the server, which uses a generative AI model for analysis and updates it regularly.

[1712] 4. Call End and Warning Notifications

[1713] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[1714] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1715] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[1716] 5. Call history storage and AI model updates

[1717] The server stores and manages the analysis results and related data of calls. This allows the call history to be accumulated in a database and used for future analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraud methods.

[1718] Users will receive notifications as their systems are updated and can check or change their settings as needed.

[1719] Specific examples

[1720] Next, we will explain how this system works in practice using a specific scenario.

[1721] Scenario: The process of detecting fraudulent calls

[1722] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1723] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1724] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1725] 4. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1726] 5. The user receives a warning message and considers appropriate action.

[1727] In this way, the present invention provides a concrete means for detecting fraudulent calls in real time and protecting users.

[1728] Example prompt using a generative AI model:

[1729] "Call text: {text of the call}

[1730] Is this call a fraud risk? Generate an appropriate fraud risk score."

[1731] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1732] Step 1:

[1733] When a call is made via a fixed line, the server detects the call and transfers it to a dedicated calling application. The input is the call signal from the fixed line, and the output is the call data transferred to the dedicated calling application. The server monitors the call content and converts the call data into an appropriate format using a dedicated protocol before sending it.

[1734] Step 2:

[1735] A terminal running a dedicated calling application receives call data transferred from the server. The input is the call data sent from the server, and the output is the call content played on the terminal. The terminal analyzes the received data in real time and continues the call.

[1736] Step 3:

[1737] The device continuously transmits the received call data to the server in real time. The input is the voice data during the call, and the output is the voice data transmitted to the server. The device uses a high-performance voice recognition engine to convert the voice into digital data and transmits it to the server.

[1738] Step 4:

[1739] The server converts the voice data received from the device into text data using a voice recognition method. The input is the voice data from the device, and the output is text data. The server analyzes the voice using a voice recognition library (e.g., speech_recognition library) and an external API (e.g., Google Speech Recognition API) and converts it into text.

[1740] Step 5:

[1741] The server uses natural language processing (NLP) technology on the text data to detect fraud patterns. The input is the text data, and the output is a fraud likelihood score. The server uses a generative AI model to analyze characteristic phrases and context contained in the text data and quantify the likelihood of fraud.

[1742] Step 6:

[1743] If the server determines that there is a high possibility of fraud based on the fraud probability score, it sends an instruction to end the call to the terminal. The input is the fraud probability score, and the output is an instruction to end the call. The server checks the fraud judgment criteria, and if the fraud risk is high, it immediately issues an instruction to end the call.

[1744] Step 7:

[1745] The terminal receives a call termination instruction from the server and automatically terminates the call. The input is the call termination instruction, and the output is disconnecting the call and playing a warning message. The terminal terminates the call and displays or plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1746] Step 8:

[1747] The user receives a warning message and considers the appropriate response. The input is the warning message, and the output is the user's action. The user checks the warning message and, if necessary, contacts a family member or calls the police.

[1748] Step 9:

[1749] The server stores and manages the analysis results and related data for calls. The input is the analysis results and call data, and the output is historical data stored in a database. The server uses this data for future analysis and to respond to new fraud methods.

[1750] Step 10:

[1751] The generative AI model is regularly updated to improve its accuracy in order to respond to the latest fraud techniques. The input is new fraud patterns and analytical data, and the output is an updated generative AI model. The server reflects the new data and improves the model's performance.

[1752] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1753] MODE FOR CARRYING OUT THE INVENTION

[1754] The present invention provides an AI calling service that combines an emotion engine to prevent damage caused by fraudulent phone calls. This system transfers calls received via fixed-line communication lines to a dedicated calling application and analyzes the content of the call in real time. It also analyzes the user's emotional state and responds accordingly. Detailed modes for implementing the present invention are described below.

[1755] 1. Transferring calls from a landline to a dedicated calling app

[1756] server

[1757] The server monitors the fixed communication line and detects when a call is made. Based on this detection, it prepares to transfer the call data to the dedicated calling application. The call data is transferred from the server to the dedicated calling application and then sent to the terminal via the application.

[1758] Terminal

[1759] The terminal starts the dedicated call application and receives the call data transferred from the server, allowing the call to continue on the terminal.

[1760] user

[1761] By receiving calls through a dedicated calling application, users can limit their use of free calling apps and reduce the risk of fraudulent calls.

[1762] 2. Real-time conversion of call content

[1763] server

[1764] The server converts the call data received from the terminal from voice to text in real time, using a high-performance voice recognition means.

[1765] Terminal

[1766] The terminal continues to send call data to the server.

[1767] 3. Fraud Pattern Detection

[1768] server

[1769] The server then analyzes the converted text data using natural language processing (NLP) techniques to detect fraud patterns, identifying characteristic phrases and contexts of fraud, and scoring the likelihood of fraud.

[1770] Terminal

[1771] The terminal receives the analysis results from the server in real time.

[1772] 4. Call End and Warning Notifications

[1773] server

[1774] If the server determines that there is a high possibility of fraud, it will send an instruction to the terminal to end the call, along with information to notify the user of a warning message.

[1775] Terminal

[1776] The device receives instructions from the server and automatically ends the call, then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1777] user

[1778] The call will be automatically terminated and the user will receive a warning message, allowing them to consider appropriate action (e.g., contacting family, calling the police).

[1779] 5. Emotion analysis using an emotion engine

[1780] server

[1781] The server is equipped with an emotion engine that analyzes not only the content of the call but also the user's emotional state. This emotion engine analyzes the tone, speed, emphasis, etc. of the user's voice during the call to assess the user's emotional state in real time.

[1782] Terminal

[1783] The device sends emotion data along with the call data to the server, which uses this data to perform emotion analysis.

[1784] 6. Emotion-based warnings and instructions

[1785] server

[1786] The server adjusts the warning message based on the user's emotional state, for example, playing a more detailed and reassuring message if the user is showing signs of impatience or anxiety.

[1787] Terminal

[1788] The terminal plays an appropriate warning message to the user based on instructions from the server.

[1789] 7. Call history storage and AI model updates

[1790] server

[1791] The server stores and manages the analysis results and related data of calls. This allows the call history to be stored in a database for future analysis and trend analysis. Furthermore, the generative AI model is regularly updated to adapt to new fraudulent techniques.

[1792] user

[1793] Users will receive notifications as their system is updated and can review and change their settings as needed.

[1794] Specific examples

[1795] Next, we will explain how this system works in practice using a specific scenario.

[1796] Scenario: Detecting fraudulent calls and taking emotions into account

[1797] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1798] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1799] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1800] 4. The server uses the emotion engine to evaluate the user's emotional state in real time.

[1801] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1802] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[1803] 7. The server stores the call analysis results and related data and updates the generative AI model.

[1804] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[1805] The processing flow will be explained below.

[1806] Step 1:

[1807] A user receives a call on a landline. When the call begins, an initial signal is sent from the landline to the server.

[1808] Step 2:

[1809] The server detects when a call is made over a fixed line and, based on this detection, instructs the server to transfer the call data to a dedicated calling application.

[1810] Step 3:

[1811] The server forwards the call data to the dedicated calling application. In this process, the call data is sent from the server to the application.

[1812] Step 4:

[1813] The device starts a dedicated call application and receives the call data transferred from the server, and the device makes the received call available to the user.

[1814] Step 5:

[1815] The device continuously transmits call data to the server. The voice data of the call is sent to the server in real time.

[1816] Step 6:

[1817] The server converts the call data into text data using a speech recognition means, using a high-performance speech recognition engine for this purpose.

[1818] Step 7:

[1819] The server analyzes the text data using natural language processing technology to detect fraudulent patterns and characteristic phrases within the text data.

[1820] Step 8:

[1821] The server scores the likelihood of fraud and assesses the risk of the call based on the result. If it determines that fraud is likely, it proceeds to the next step.

[1822] Step 9:

[1823] The server activates an emotion engine to recognize the user's emotions, which analyzes the user's voice tone, speed, emphasis, etc. in real time.

[1824] Step 10:

[1825] The server evaluates the user's emotional state based on the analysis results of the emotion engine. If the user shows signs of impatience or anxiety, it is determined that there is a high possibility of fraud.

[1826] Step 11:

[1827] If the server determines that there is a high possibility of fraud, it will send an instruction to end the call to the terminal, which will automatically end the call.

[1828] Step 12:

[1829] The device receives the server's instructions, automatically ends the call, and then plays a warning message to the user saying, "A potential fraudulent call has been detected. Please take immediate action."

[1830] Step 13:

[1831] The user receives a warning message, and upon receiving the warning, the user considers appropriate action, such as contacting a family member or reporting the incident to the police.

[1832] Step 14:

[1833] The server stores the call data, analysis results, and emotion data, which can be used for future analysis and data analysis.

[1834] Step 15:

[1835] The server regularly updates the emotion engine and generative AI model to adapt to new scams and emotional patterns.

[1836] Step 16:

[1837] Users will receive notifications about system updates and can check and change their settings as needed, ensuring they always have the latest fraud prevention system.

[1838] Example 2

[1839] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1840] Fraudulent phone calls are a malicious method used to illegally obtain personal information and cause financial damage. The elderly are particularly targeted, and the damage caused by these calls is becoming widespread. However, existing technologies lack effective means for detecting and preventing fraudulent calls. In particular, it has been difficult for existing technologies to detect fraud in real time and take appropriate action while taking into account the emotional state of the user during the call.

[1841] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1842] In this invention, the server includes means for transferring calls received via a fixed communication line to a dedicated calling application, speech recognition means for converting received call data into text data, natural language processing means for detecting fraud patterns, means for terminating the call if there is a high possibility of fraud, means for notifying the user of a fraud warning message, emotion analysis means for evaluating the user's emotional state during the call in real time, means for adjusting the warning message based on the user's emotional state, and means for saving and managing the call analysis results and related data. This reduces the risk of fraudulent calls and makes it possible to effectively prevent fraud while providing users with a sense of security.

[1843] "Fixed line" refers to infrastructure for wired communications, including the Internet and the public switched telephone network.

[1844] A "dedicated calling application" is application software designed to process specific calling data.

[1845] A "speech recognition means" is a computer program or algorithm for converting speech data into text.

[1846] "Natural language processing means" is a technology for analyzing text data and understanding its meaning.

[1847] A "pattern of fraud" is a characteristic phrase or context that suggests fraudulent activity.

[1848] A "high likelihood of fraud" means that the content and context of the call indicate a high risk of fraud.

[1849] A "warning message" is a notification intended to alert the user.

[1850] "Emotion analysis means" is a technique for assessing a user's emotional state.

[1851] A "generative AI model" is a model created based on artificial intelligence techniques and used to perform specific tasks.

[1852] "Text data" is data expressed as character information.

[1853] "Analysis results" are information or conclusions obtained based on the analysis of data.

[1854] "Related data" is additional information or metadata related to the analysis results.

[1855] "Preservation" is the act of keeping data in digital form.

[1856] "Management" is the act of keeping stored data available efficiently and effectively.

[1857] MODE FOR CARRYING OUT THE INVENTION

[1858] The present invention provides an AI call service system that combines an emotion analysis engine to prevent damage caused by fraudulent phone calls. This system forwards calls received via fixed-line communication lines to a dedicated calling application and analyzes the call content and the user's emotional state in real time. Detailed embodiments of this system are described below.

[1859] Hardware and Software Configuration

[1860] server

[1861] The server monitors the fixed line and detects when a call is made. When a call is detected, the server performs the following actions:

[1862] 1. Transfer of call data: The server captures call data from fixed communication lines and transfers it to a dedicated call application in real time, buffering the data and converting its format.

[1863] 2. Speech recognition: The server converts the received call data into text using a high-performance speech recognition system (e.g., Google Cloud Speech-to-Text API), which allows the voice content to be analyzed as text information.

[1864] 3. Natural Language Processing (NLP): The server analyzes the text data using natural language processing techniques (e.g., Python's SpaCy or NLTK libraries) to detect fraud patterns, identify phrases and contexts characteristic of specific frauds, and assign a score to the likelihood of fraud.

[1865] 4. Emotion Analysis: The server uses an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to assess the user's emotional state. It analyzes acoustic features such as tone, rate, and emphasis of the user's voice to estimate the user's emotional state in real time.

[1866] 5. Warning message generation: If a fraudulent activity is deemed likely, the server generates a warning message for the user. The warning message is tailored based on the user's emotional state to provide a sense of security.

[1867] Terminal

[1868] The device runs a dedicated call application and receives and processes the call data sent from the server. The process on the device is as follows:

[1869] 1. Continuing the call: The dedicated calling application transfers the call data received from the server to the audio output device, allowing the user to continue the call.

[1870] 2. Data transmission: The device continuously transmits voice data and emotional characteristic data to the server in real time during the call. This transmitted data is important for analysis.

[1871] 3. Warning Implementation: If the server determines that there is a high possibility of fraud, the terminal will automatically terminate the call and play a warning message to the user. The warning message will be played through the audio output device.

[1872] user

[1873] Users interact with the system as follows:

[1874] 1. Receiving calls: Receive calls via landline and continue the call through a dedicated calling application, limiting the use of free calling apps and reducing the risk of fraudulent calls.

[1875] 2. Warning Response: If the system detects possible fraud, the user will receive a warning message and consider taking appropriate action (e.g., contacting family, reporting to the police).

[1876] Specific examples

[1877] Next, we will explain how this system works in practice using a specific scenario.

[1878] Scenario: Detecting fraudulent calls and taking emotions into account

[1879] 1. A user receives a call on a landline, which is then forwarded by a server to a dedicated calling application.

[1880] 2. A dedicated calling application on the device receives the call and sends the call data to the server.

[1881] 3. The server converts the call data into text in real time and begins analysis, which scores the likelihood of fraud based on characteristic phrases and context.

[1882] 4. The server uses an emotion analysis engine to assess the user's emotional state in real time.

[1883] 5. The server determines that the call is likely fraudulent and sends an instruction to end the call to the device, which automatically ends the call and plays a warning message to the user.

[1884] 6. The user receives a warning message and considers the appropriate response. The warning message is adjusted based on the user's emotional state, allowing the user to respond with greater peace of mind.

[1885] 7. The server stores the call analysis results and related data and updates the generative AI model.

[1886] Prompt Sentence Examples

[1887] By inputting the following prompt sentences into the generative AI model, we can get answers about the detailed operation of this system.

[1888] "Please explain the role of the server and terminal in the fraudulent call detection system."

[1889] In this way, the present invention provides safe and effective protection by detecting fraudulent calls in real time and tailoring its response to take into account the user's emotional state.

[1890] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1891] Step 1:

[1892] The server monitors the fixed line and detects when a call is made. When a call is started, the server prepares to acquire the call data. Specific operations include capturing voice data and allocating a buffer. The input of this step is the call signal from the fixed line, and the output is the acquisition of the ready call data.

[1893] Step 2:

[1894] The server captures call data from fixed communication lines and transfers it to the dedicated calling application in real time. The call data is buffered and format converted. The input of this step is the captured call data, and the output is the data transferred to the dedicated calling application.

[1895] Step 3:

[1896] The terminal starts a dedicated call application and receives the call data sent from the server. The received call data is sent to the audio output device, allowing the user to continue the call. The input of this step is the call data from the server, and the output is data sent to the audio output device.

[1897] Step 4:

[1898] The terminal continues to send voice data during the call to the server in real time. The server converts the received call data into text using a high-performance speech recognition system. The input of this step is the call data and the speech recognition system, and the output is text data.

[1899] Step 5:

[1900] The server then analyzes the converted text data using natural language processing techniques, specifically analyzing phrases and context to detect fraud patterns. The input for this step is the text data, and the output is a scoring result indicating the likelihood of fraud.

[1901] Step 6:

[1902] The server uses an emotion analysis engine to evaluate the user's emotional state in real time. For this analysis, acoustic features such as tone, rate, and emphasis of the voice data are analyzed. The input of this step is the call data and acoustic features, and the output is the evaluation result of the emotional state.

[1903] Step 7:

[1904] If the server determines that there is a high possibility of fraud, it sends a call termination instruction to the terminal. The instruction includes a warning message for the user. The input of this step is the fraud scoring result and the emotion evaluation result, and the output is the call termination instruction and the warning message.

[1905] Step 8:

[1906] The terminal receives the instruction from the server and automatically ends the call. After the call ends, the terminal plays a warning message to notify the user. The input of this step is the instruction to end the call, and the output is the end of the call and the playing of the warning message.

[1907] Step 9:

[1908] The user receives an automatically terminated call and a warning message, and decides what action to take based on that information. The input to this step is the warning message, and the output is the user's action.

[1909] Step 10:

[1910] The server stores and manages the call analysis results and related data, and periodically updates the generative AI model, allowing it to adapt to new fraud techniques. The inputs to this step are the analysis results and call data, and the outputs are the stored data and an updated AI model.

[1911] (Application example 2)

[1912] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1913] Conventional fraud prevention systems analyze the content of calls and issue a warning if they determine there is a possibility of fraud, but they are unable to take the user's emotional state into account. This makes it difficult to respond when the user is feeling anxious or anxious, and in some cases, they are unable to provide effective protection. Furthermore, as fraud methods evolve daily, delays in system updates have led to the problem of being unable to respond to new fraud patterns.

[1914] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1915] In this invention, the server includes a means for forwarding calls received via a fixed-line communication line to a dedicated calling application, a speech recognition means for converting received call data into text data, a natural language processing means for detecting fraud patterns, a means for terminating calls when a fraudulent call is highly likely, a means for notifying the user of a fraud warning message, an emotion engine for analyzing the user's emotional state, and a means for adjusting the warning message based on the user's emotional state. This not only enables the detection of fraudulent calls but also enables appropriate responses based on the user's emotional state. Furthermore, by using a generative AI model to analyze call content in real time and periodically updating it, the system can quickly respond to new fraud patterns.

[1916] A "fixed communication line" is a communication line that transmits data using a physical cable, and includes telephone lines and internet lines.

[1917] A "dedicated calling application" is an application designed specifically for a specific purpose (in this case, fraudulent call detection) that is installed on a smartphone or other device.

[1918] "Speech recognition means" refers to technology or devices for converting voice data into text data, including high-performance speech recognition APIs.

[1919] "Natural language processing means" refers to technologies and algorithms for analyzing text data and extracting specific patterns and meanings, including NLP libraries.

[1920] "Call termination measures" are features or technologies that automatically disconnect calls when fraud is deemed likely.

[1921] "Means for notifying warning messages" means any technology or device for sending a message to a user indicating a potential fraud, including audio or text notifications.

[1922] The "emotion engine" is a technology that analyzes the tone, speed, emphasis, etc. of a user's voice during a call to assess their emotional state.

[1923] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and make predictions or judgments about new data, including, for example, machine learning algorithms.

[1924] "Analytical means" refers to techniques and devices for processing data and extracting specific patterns or meanings, including data analysis tools and algorithms.

[1925] The present invention relates to a system that redirects calls received via fixed-line communication lines to a dedicated calling application and analyzes the received call data in real time to prevent fraudulent calls. This system can analyze not only the content of the call but also the user's emotional state and adjust warning messages accordingly.

[1926] System configuration

[1927] The system includes fixed-line communications, servers, a dedicated calling application, voice recognition technology, natural language processing technology, an emotion engine, and a generative AI model.

[1928] server

[1929] The server monitors fixed-line communications lines and detects incoming calls. Based on this detection, it prepares to transfer the call data to a dedicated calling application. The server then uses high-performance speech recognition technology, such as the Google Cloud Speech-to-Text API, to convert the call data from voice to text in real time. The converted text data is then analyzed using natural language processing (NLP) technologies, such as SpaCy and NLTK, to detect fraud patterns.

[1930] The server also uses an emotion engine (e.g., Nuance emotion analysis API) to analyze the user's emotional state in real time. If it determines that there is a high possibility of fraud, the server sends an instruction to the terminal to end the call and simultaneously notifies the terminal of a warning message. The warning message is adjusted based on the user's emotional state.

[1931] Terminal

[1932] The device launches a dedicated calling application and receives call data transferred from the server. The device continues to send the received call data to the server and receives analysis results and instructions from the server in real time. If the server determines that there is a high possibility of fraud, the device automatically terminates the call and plays a warning message to the user.

[1933] user

[1934] When a user receives a call on their landline, the call is forwarded to a dedicated calling application. If a potential scam is detected during the call, the call is automatically terminated and a warning message such as "High probability of scam. Please proceed with caution" is played. Depending on the user's emotional state, a more detailed message may be provided.

[1935] For example, if the user is feeling anxious or worried, the message could include specific instructions such as, "This may be a scam. Your call has been automatically terminated due to anxiety in your voice. Please contact a family member and report this to the police."

[1936] This system not only detects fraudulent calls, but also responds appropriately based on the user's emotional state. It also uses a generative AI model to analyze call content in real time and provides regular updates, allowing it to quickly respond to new fraud patterns.

[1937] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1938] Step 1:

[1939] The server monitors the fixed communication line and detects when a call occurs. The input is the call signal from the fixed communication line, and the output is the call data. Specifically, the server detects the call signal, captures the signal, and stores it as digital data.

[1940] Step 2:

[1941] The server prepares to transfer the detected call data to the dedicated call application. The input is the call data obtained in step 1, and the output is a flag indicating that the data is ready for transfer. Specifically, the server converts and compresses the data, and then prepares it for transfer to the dedicated call application.

[1942] Step 3:

[1943] The terminal starts a dedicated calling application and receives call data transferred from the server. The input is the call data from the server, and the output is the received call data. Specifically, the terminal starts the application and receives data from the server using the set communication protocol.

[1944] Step 4:

[1945] The server converts the received call data from voice to text in real time. The input is the call data sent from the device, and the output is the converted text data. Specifically, the server calls the Google Cloud Speech-to-Text API to convert the call voice to text.

[1946] Step 5:

[1947] The server analyzes the converted text data using natural language processing techniques to detect fraud patterns. The input is the text data obtained in step 4, and the output is a score indicating the likelihood of fraud. Specifically, the server uses NLP libraries such as SpaCy and NLTK to analyze suspicious phrases and contexts in the text data.

[1948] Step 6:

[1949] The server uses an emotion engine to analyze the user's emotional state in real time. The input is call data from the device, and the output is an indicator of the emotional state. Specifically, the server uses the Nuance emotion analysis API to analyze the tone, speed, and emphasis of the call audio to evaluate the user's emotional state.

[1950] Step 7:

[1951] The server determines that there is a high possibility of fraud and sends an instruction to terminate the call to the terminal. The input is the score and emotional state obtained in steps 5 and 6, and the output is an instruction to terminate the call. Specifically, the server evaluates the score and emotional state, and if it determines that there is a high possibility of fraud, it sends an instruction to terminate the call to the terminal.

[1952] Step 8:

[1953] The terminal automatically terminates a call based on an instruction received from a server. The input is a call termination instruction from the server, and the output is a call termination action. Specifically, the terminal receives the instruction from the server and terminates the current call through the calling application.

[1954] Step 9:

[1955] The server notifies the user of a fraud warning message. The input is the judgment result obtained in step 7, and the output is a warning message. Specifically, the server generates a message such as "There is a high possibility of fraud. Please proceed with caution" and sends it to the terminal.

[1956] Step 10:

[1957] The terminal plays back to the user the warning message received from the server. The input is the warning message from the server, and the output is the notification to the user. Specifically, the terminal reads out the message received from the server or displays it as text.

[1958] Step 11:

[1959] The server stores all analysis results and related data and periodically updates the generative AI model. The input is the data based on the analysis, and the output is an updated generative AI model. Specifically, the server periodically updates the generative AI model to reflect new fraud patterns.

[1960] These processing steps enable the system to detect fraudulent calls and provide appropriate responses based on the user's emotional state, while regular analysis using generative AI models allows the system to quickly respond to new fraud patterns.

[1961] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1962] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1963] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1964] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1965] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1966] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1967] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1968] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1969] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1970] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1971] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1972] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1973] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1974] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1975] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1976] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1977] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1978] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1979] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1980] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1981] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1982] The following is further disclosed regarding the above embodiment.

[1983] (Claim 1)

[1984] means for transferring calls received via a fixed communication line to a dedicated calling application;

[1985] a speech recognition means for converting received call data into text data;

[1986] natural language processing means for detecting fraud patterns;

[1987] A means to terminate calls if fraud is likely;

[1988] a means of notifying users of fraud warning messages;

[1989] A system including:

[1990] (Claim 2)

[1991] 10. The system of claim 1, further comprising an analysis means for scoring the likelihood of fraud.

[1992] (Claim 3)

[1993] 10. The system of claim 1, further comprising: means for using a periodically updated generative AI model to analyze call content in real time.

[1994] "Example 1"

[1995] (Claim 1)

[1996] means for transferring calls received via fixed line communications to dedicated calling software;

[1997] speech recognition software for converting received call data into text data;

[1998] natural language processing programs to detect fraud patterns;

[1999] A means to terminate calls if fraud is likely;

[2000] a means for notifying users of fraud warning messages;

[2001] a means for analyzing fraud patterns using a generative AI model;

[2002] A system including:

[2003] (Claim 2)

[2004] 10. The system of claim 1, further comprising an analysis means for scoring the likelihood of fraud.

[2005] (Claim 3)

[2006] 10. The system of claim 1, further comprising: means for using a periodically updated generative AI model to analyze call content in real time.

[2007] "Application Example 1"

[2008] (Claim 1)

[2009] means for transferring calls received via a fixed communication line to a dedicated calling application;

[2010] a speech recognition means for converting received call data into text data;

[2011] natural language processing means for detecting fraud patterns;

[2012] A means to terminate calls if fraud is likely;

[2013] a means of notifying users of fraud warning messages;

[2014] A server communication means for recognizing the voice of the call content in real time on a smartphone and detecting fraud patterns;

[2015] A means for issuing a warning to the user's smartphone in response to the result of the fraud possibility scoring;

[2016] A system including:

[2017] (Claim 2)

[2018] 10. The system of claim 1, further comprising an analysis means for scoring the likelihood of fraud.

[2019] (Claim 3)

[2020] 10. The system of claim 1, further comprising: means for using a periodically updated generative AI model to analyze call content in real time.

[2021] "Example 2: Combining Emotion Engines"

[2022] (Claim 1)

[2023] means for transferring calls received via a fixed communication line to a dedicated calling application;

[2024] a speech recognition means for converting received call data into text data;

[2025] natural language processing means for detecting fraud patterns;

[2026] A means to terminate calls if fraud is likely;

[2027] a means of notifying users of fraud warning messages;

[2028] an emotion analysis means for assessing the emotional state of the user during the call in real time;

[2029] means for tailoring the warning message based on the emotional state of the user;

[2030] A means of storing and managing call analysis results and related data;

[2031] A system including:

[2032] (Claim 2)

[2033] 10. The system of claim 1, further comprising an analysis means for scoring the likelihood of fraud.

[2034] (Claim 3)

[2035] 10. The system of claim 1, further comprising: means for using a periodically updated generative AI model to analyze call content in real time.

[2036] "Application example 2 when combining emotion engines"

[2037] (Claim 1)

[2038] means for transferring calls received via a fixed communication line to a dedicated calling application;

[2039] a speech recognition means for converting received call data into text data;

[2040] natural language processing means for detecting fraud patterns;

[2041] A means to terminate calls if fraud is likely;

[2042] a means of notifying users of fraud warning messages;

[2043] An emotion engine that analyzes the user's emotional state;

[2044] means for tailoring the warning message based on the emotional state;

[2045] A system including:

[2046] (Claim 2)

[2047] 10. The system of claim 1, further comprising an analysis means for scoring the likelihood of fraud.

[2048] (Claim 3)

[2049] 10. The system of claim 1, further comprising: means for using a periodically updated generative AI model to analyze call content in real time. [Explanation of symbols]

[2050] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for transferring calls received via a fixed communication line to a dedicated calling application; a speech recognition means for converting received call data into text data; natural language processing means for detecting fraud patterns; A means to terminate calls if fraud is likely; a means of notifying users of fraud warning messages; A system including:

2. 10. The system of claim 1, further comprising an analysis means for scoring the likelihood of fraud.

3. 10. The system of claim 1, further comprising means for using a periodically updated generative AI model to analyze call content in real time.

Images