system

Abstract

There has been a rise in special fraud phone calls, and there is a need to detect fraudulent calls and respond quickly. [Solution] A system including: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to a communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for the communication server to save the call log and generate a report after the call is terminated; and a means for notifying the user of the generated report.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there has been a surge in special fraud phone calls, resulting in increased personal information theft and financial losses. These types of frauds are sophisticated and tend to target vulnerable groups, such as the elderly. This has created a need for detection of fraudulent phone calls and rapid response. However, conventional methods make it difficult to detect fraudulent phone calls in real time, making it difficult to prevent damage before it occurs. In addition, analysis of call content and reporting of damage are often done manually, which makes it difficult to respond quickly and effectively. This invention aims to solve these problems. [Means for solving the problem]

[0005] The present invention comprises a communication terminal receiving a call and detecting call information, and a means for transmitting the detected call information to a communication server. The communication server records the call information in real time and converts the recorded call content into text using voice recognition technology. It also comprises a means for analyzing and detecting fraud keywords from the text. If a fraud keyword is detected, the communication server provides a means for sending a warning signal to the communication terminal and terminating the call. It also comprises a means for saving a call log after the call is terminated, generating a detailed report on fraud risks, and notifying the user. This enables real-time detection of fraudulent calls and rapid response, preventing damage before it occurs.

[0006] A "communication terminal" is a device for receiving calls, transmitting voice, and exchanging data with a communication server.

[0007] "Communication Server" is a central control device for recording and analyzing call information, detecting fraud keywords and reporting to users.

[0008] "Call information" refers to the audio data and metadata of a call sent from a communication terminal.

[0009] "Call Log" refers to recorded data including the content of the call, detected fraud keywords, and analysis results.

[0010] "Speech recognition technology" is a technology for converting speech into text.

[0011] "Text" is text information converted using voice recognition technology.

[0012] "Fraudulent keywords" refer to specific words or phrases associated with fraudulent activity.

[0013] A "warning signal" is a signal sent to a communications terminal to alert the user to possible fraud.

[0014] "Real-time" means that processing occurs immediately while the call is in progress.

[0015] "Report" means a detailed notification of fraud risk generated based on call logs.

[0016] "User" refers to an individual or organization that uses a communications terminal. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention relates to a system for detecting and preventing special fraud calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof are described below.

[0039] 1. Call monitoring and recording

[0040] When the device receives a call, it immediately sends the call information to the server, which then records the received call information in real time, thereby recording all audio from the beginning to the end of the call.

[0041] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[0042] 2. Analysis of call content

[0043] While the recording is in progress or once it is complete, the server uses speech recognition technology to convert the recording into text, which is then analyzed against a pre-defined list of fraud keywords.

[0044] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[0045] 3. Fraudulent call detection and warning

[0046] If the call contains any fraudulent keywords, the server immediately sends a warning signal to the device, informing the user that "there is a possibility of fraud." The device then automatically ends the call.

[0047] To give a concrete example, when the server detects the above fraudulent keywords, the device will display the message "Possible fraud" and terminate the call, allowing the user to prevent damage before it occurs.

[0048] 4. Call log storage and reporting

[0049] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results, which is then sent to the user via email and app notification.

[0050] For example, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call." This report will encourage the user to be more vigilant and take measures to prevent further damage.

[0051] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[0055] Step 2:

[0056] The server receives call information sent from the device in real time, immediately begins recording the received call, and continues recording until the call ends.

[0057] Step 3:

[0058] The server uses voice recognition technology to convert the call into text in real time while recording, where the voice data is analyzed sequentially into a string of characters.

[0059] Step 4:

[0060] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are included in the text.

[0061] Step 5:

[0062] If a fraud keyword is detected, the server immediately sends a warning signal to the terminal, which then displays an alert to the user.

[0063] Step 6:

[0064] When the device receives a warning signal, it displays a warning message on the screen saying "Possible fraud" and automatically terminates the call.

[0065] Step 7:

[0066] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, and other relevant information.

[0067] Step 8:

[0068] The server generates a detailed report based on the stored call logs, including the duration of the call, the caller's details, detected fraud keywords, and analysis results.

[0069] Step 9:

[0070] Finally, the server notifies the user of the generated report via email or in-app notification, allowing the user to receive detailed and timely information about their fraud risk.

[0071] This processing flow makes it possible to detect and repel special fraud calls in real time, improving the security of individuals and society as a whole.

[0072] Example 1

[0073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0074] In modern society, the number of victims of special fraud phone calls is increasing, affecting many individuals. Conventional fraud prevention systems are inadequate in their ability to detect and prevent fraud, making it difficult to completely prevent damage. Given this background, there is a need for a new system that can detect fraudulent phone calls in real time and efficiently warn and repel them.

[0075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0076] In this invention, the server includes a means for a communication terminal to receive a call and detect call information, a means for transmitting the detected call information to the communication server, a means for the communication server to record the call information in real time, a means for the communication server to convert the recorded call content into text using voice recognition technology, a means for analyzing and detecting fraud keywords from the text, a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected, a means for the communication server to save the call log and generate a report after the call ends, a means for notifying the user of the generated report, and a means for the user to be more vigilant based on the report. This enables real-time detection of fraudulent calls and prompt warnings and call termination, thereby improving the safety of individuals and society.

[0077] A "communications terminal" is an electronic device that has the function of receiving calls and detecting and transmitting call information.

[0078] "Call information" is data that includes identification information such as the caller number and the call start time when a call is made.

[0079] A "communication server" is a central management system that records call information sent from communication terminals in real time, analyzes it, and generates reports.

[0080] "Real-time recording" is the process of recording audio instantly while a call is taking place.

[0081] "Speech recognition technology" is a technology for converting recorded voice data into text format.

[0082] "Text" is character string data converted using speech recognition technology.

[0083] "Fraud keywords" are a specific set of words or phrases that indicate possible fraud.

[0084] "Analysis" is the process of extracting fraud keywords from text data and assessing the likelihood of fraud.

[0085] The "warning signal" is a notification means for informing the communication terminal or user of danger when a fraud keyword is detected.

[0086] A "call log" is recorded data that includes detailed historical information about calls.

[0087] A "report" is a detailed document of fraud risk assessment and warnings generated based on call logs.

[0088] A "user" is a person who uses a communication terminal or a communication server to use the fraudulent call detection and rejection system.

[0089] "Further vigilance" refers to an action taken by a user who has received a report to take additional crime prevention measures based on the warning.

[0090] The present invention is a system for detecting and blocking special fraud phone calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof will be described below.

[0091] This system mainly consists of a communication terminal that receives calls, a communication server that receives and analyzes call information, and a user that receives reports.

[0092] Receiving calls and notifications

[0093] When the device receives a call, it immediately sends the call information (caller ID, call start time, etc.) to the server. Specifically, the device detects the incoming call using its internal call detection module and sends the specified call information to the server via a communication protocol.

[0094] Call recording

[0095] The server starts recording the call based on the received call information. For recording, for example, VoIP (Voice over IP) technology is used to collect voice data in real time and store it in a temporary storage area. The recorded data is saved in a compressed format (e.g., MP3) and used for subsequent analysis processing.

[0096] Call transcription

[0097] While the recording is in progress or once it is complete, the server uses speech recognition technology such as the Google® Cloud Speech-to-Text API to convert the recording into text, which is then used to parse the content of the call.

[0098] Fraudulent Keyword Analysis

[0099] The server receives the text data and analyzes it against a list of fraud keywords. This analysis can be done using natural language processing technology, for example, to search for specific keywords such as "son," "accident," and "security deposit." This allows the server to detect calls that may be fraudulent.

[0100] Scam call warning and call termination

[0101] If a fraud keyword is detected, the server immediately sends a warning signal to the device. The device receives this signal, displays a message to the user saying "Possible fraud," and terminates the call. This action allows the user to prevent damage before it occurs.

[0102] Call log storage and report generation

[0103] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email or a dedicated application.

[0104] Specific examples

[0105] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server. The server records the call and converts the recording into text such as "Hello, I'm your son. I've been in an accident and need money." The server detects the fraud keywords "son," "accident," and "money" and sends a warning signal to the device. The device displays "Possible fraud" and terminates the call. The server then generates a fraud risk report and notifies the user that "The call from 07012345678 is suspected to be fraudulent."

[0106] Prompt Sentence Examples

[0107] This system can use a generative AI model (e.g., OpenAI's GPT-4®). The following prompts can be used to automatically generate a description of the system:

[0108] This system aims to detect and block special fraud phone calls, and includes a series of processes for call monitoring, recording, analysis, warnings, and reporting. The main process flow is as follows:

[0109] 1. When a call starts, the device sends the call information to the server, and the server records the call.

[0110] 2. The recording is converted into text using voice recognition technology and analyzed against a list of fraud keywords.

[0111] 3. If a fraud keyword is detected, the server sends a warning signal to the terminal, notifying the user and terminating the call.

[0112] 4. After the call ends, the server stores the call log and generates a detailed fraud risk report to the user.

[0113] Use this description as a starting point to write a piece that highlights the features by adding detailed steps and the techniques used.

[0114] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[0115] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0116] Step 1:

[0117] The terminal receives a call. The terminal detects the incoming call and acquires call information such as the caller ID and the start time of the call. The input is the call information of the received call, and the output is the acquired call information. Specifically, the terminal uses its internal call detection module to detect the incoming call and collect the necessary call information.

[0118] Step 2:

[0119] The terminal sends the acquired call information to the server. The input is the call information acquired in step 1, and the output is the call information sent to the server. Specifically, the terminal uses a communication protocol to send call information such as the caller ID and call start time to the server via the network.

[0120] Step 3:

[0121] The server starts recording the call based on the received call information. The input is the call information sent from the device, and the output is the recorded call data. Specifically, the server uses VoIP technology to record the call content in real time and saves the recorded data in a temporary storage area.

[0122] Step 4:

[0123] The server converts the recorded call content into text using speech recognition technology. The input is the recorded call data, and the output is the converted text data. Specifically, the server uses a speech recognition service such as Google Cloud Speech-to-Text API to convert the voice data into text format.

[0124] Step 5:

[0125] The server analyzes the text data by comparing it with a list of fraudulent keywords. The input is the converted text data, and the output is the results of the fraudulent keyword detection. Specifically, the server uses natural language processing technology to extract fraudulent keywords such as "son," "accident," and "security deposit" from the text.

[0126] Step 6:

[0127] If a fraudulent keyword is detected, the server sends a warning signal to the terminal. The input is the detection result of the fraudulent keyword, and the output is the warning signal to the terminal. Specifically, the server generates a warning signal and sends it to the terminal via a communication protocol.

[0128] Step 7:

[0129] The terminal receives the warning signal, displays "Possible fraud" to the user, and terminates the call. The input is the warning signal sent from the server, and the output is the display of the warning message and the termination of the call. Specifically, the terminal displays the warning message on the screen based on the received warning signal and executes a command to terminate the call.

[0130] Step 8:

[0131] After the call is completed, the server saves the call log and generates a detailed fraud risk report. The input is the call details and analysis results, and the output is the saved call log and the generated report. Specifically, the server saves the log, including the call date and time, caller information, detected fraud keywords, and analysis results, in a database and creates a report based on that information.

[0132] Step 9:

[0133] The server notifies the user of the generated report. The input is the generated report, and the output is the report sent to the user. Specifically, the server sends the fraud risk report to the user's email address or app via email or a dedicated application.

[0134] (Application example 1)

[0135] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0136] In a society where special fraud phone calls are on the rise, many people, including the elderly, are at risk of becoming victims of fraud. The methods used by fraudsters are sophisticated, making it difficult for the average person to recognize and deal with them. Also, since there is no turning back once you have fallen victim to a fraud, preventative measures are important. Therefore, a system is needed that can detect fraudulent calls in real time, immediately issue a warning, and terminate the call.

[0137] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0138] In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for notifying the user if a fraud keyword is detected; a means for the communication server to save a call log and generate a report after the call ends and send the generated report to the user; a means for automatically terminating a call containing a fraud keyword; and a means for reporting the report to the user by email or app notification. This makes it possible to detect fraudulent calls in real time, immediately warn the user, and prevent damage from occurring.

[0139] "Communications terminal" refers to a device for receiving calls and detecting call information.

[0140] "Call Information" refers to information including audio data and associated metadata generated during a call.

[0141] "Communication Server" refers to a server system for receiving, recording, analyzing, and managing call information.

[0142] "Voice recognition technology" refers to technology that converts voice data into text data.

[0143] "Fraud keywords" refer to specific words or phrases used to identify suspected fraudulent calls.

[0144] The "warning signal" refers to a notification signal that alerts the user.

[0145] "Call log" refers to data that records details of calls (date and time, call recipient, content, etc.).

[0146] "Report" refers to documents or data that provide users with information generated based on call logs and analysis results.

[0147] "Real-time" refers to processing that occurs with almost no delay after an event occurs.

[0148] "Email" means a message sent or received electronically over the Internet.

[0149] "App notifications" refers to notifications sent to users through applications installed on smartphones or other digital devices.

[0150] This invention provides a system for detecting special fraud calls and protecting users. The main components of this system are a communication terminal, a communication server, voice recognition technology, fraud keyword analysis, and a means for notifying users.

[0151] 1. Operation of communication terminal

[0152] When a communication terminal receives a call, it immediately detects the call information and sends it to the communication server. If a user receives a call from an unknown number, the communication terminal detects the call and sends it to the communication server in real time.

[0153] 2. Communication Server Operation

[0154] The communications server records the received call information in real time. When the recording is in progress or is completed, it converts the recording into text using speech recognition technology. This speech recognition technology can use existing technologies such as the Google Speech Recognition API. After the conversion, the text is analyzed by comparing it with a list of pre-defined fraud keywords.

[0155] 3. Fraudulent Keyword Analysis

[0156] If the call content is something like, "Hello, this is my son. I've been in an accident and need a security deposit," the communication server will extract text containing fraud keywords such as "son," "accident," and "security deposit." If a fraud keyword is detected, the communication server will immediately send a warning signal to the communication terminal and notify the user that "there is a possibility of fraud."

[0157] 4. Call End and Notifications

[0158] If a call is determined to be fraudulent, the communication device will automatically terminate the call and issue a warning to the user. The communication server then stores the call log and generates a detailed fraud risk report, which includes the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email and app notification.

[0159] 5. Specific Examples

[0160] For example, if a mother receives a call on her smartphone from an unknown number, saying, "This is my son. I've been in an accident and need a security deposit," the system will immediately detect the fraud keywords, send a warning signal to the communication device, and terminate the call. A detailed report will then be sent to the mother's inbox.

[0161] 6. Example prompts for generative AI models

[0162] "Develop a system that records phone conversations in real time and uses voice recognition to detect fraudulent keywords. If the system detects fraud, it will warn the user and terminate the call. It will also send a detailed call report to the user via email."

[0163] The above is a specific embodiment for carrying out this invention. To ensure reproducibility and practicality, each component must be appropriately selected, and it is desirable to provide a manual detailing their operation.

[0164] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0165] Step 1:

[0166] The communication terminal receives the call and detects the call information. The input is the received call signal, and the output is the call information data. This data includes the caller's number, the call start time, the call content, etc.

[0167] Step 2:

[0168] The communication terminal transmits the detected call information to the communication server in real time. The input is the call information detected on the terminal, and the output is the call information transmitted to the communication server. This information is necessary for analysis by the communication server.

[0169] Step 3:

[0170] The communication server records the received call information in real time. The input is the call information sent from the communication terminal, and the output is the recorded voice data. This voice data becomes the basis for subsequent analysis.

[0171] Step 4:

[0172] The communication server converts the recorded conversation into text using speech recognition technology. The input is the recorded voice data, and the output is the generated text data. This conversion is performed using Google Speech Recognition API or similar.

[0173] Step 5:

[0174] The communication server analyzes and detects fraudulent keywords from the generated text data. The input is text data, and the output is the detection results of fraudulent keywords. Analysis is performed by comparing the data with a list of fraudulent keywords.

[0175] Step 6:

[0176] If a fraud keyword is detected, the communication server immediately sends a warning signal to the communication terminal and notifies the user. The input is the detection result indicating that a fraud keyword is included, and the output is the warning signal to the communication terminal and a notification to the user. This makes the user aware of the risk of fraud.

[0177] Step 7:

[0178] The communication terminal will automatically end the call when it receives the warning signal. The input is the warning signal and the output is the call end status. This allows fraudulent calls to be terminated quickly.

[0179] Step 8:

[0180] The communication server stores the call logs and generates reports after the call is completed. The input is the log data after the call is completed, and the output is the generated report. This report includes the call date and time, caller information, detected fraud keywords, and analysis results.

[0181] Step 9:

[0182] The communication server generates a report and sends it to the user via email or app notification. The input is the report data, and the output is the user's email mailbox or app notification, allowing the user to know the details of the call and the fraud risk.

[0183] The above is the specific processing flow of the program of this system.

[0184] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0185] The present invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of detecting fraudulent calls by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[0186] 1. Call monitoring and recording

[0187] When the device receives a call, it first detects the call information and prepares to send it to the server. The server then starts recording the received call information in real time, so that all audio from the beginning to the end of the call is recorded.

[0188] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[0189] 2. Analysis of call content

[0190] Once the recording is in progress or complete, the server uses speech recognition technology to convert the call into text, which is then analyzed against a pre-defined list of fraud keywords.

[0191] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[0192] 3. Emotional Analysis of Users by Emotion Engine

[0193] At the same time, the device is equipped with an emotion engine that analyzes the user's voice during a call to detect their stress level, anxiety level, and other emotions in real time.

[0194] For example, if the emotion engine detects signs of stress or anxiety in the user's voice, it will send that information to the server.

[0195] 4. Fraudulent call detection and warning

[0196] If a fraud keyword is detected and the user's sentiment analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the device, which then displays an alert to the user.

[0197] To give a specific example, the server will combine the above fraud keywords with the results of sentiment analysis to determine the risk, and the device will display the message "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[0198] 5. Call log storage and reporting

[0199] After the call is completed, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other relevant information.

[0200] Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[0201] 6. Fail-safe function

[0202] In addition, the device has a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold, allowing for the device to quickly end suspicious calls.

[0203] This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables fraud prevention with greater accuracy by taking into account the user's emotions, thereby improving the security of individuals and society as a whole.

[0204] The processing flow will be explained below.

[0205] Step 1:

[0206] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[0207] Step 2:

[0208] The server receives the call information sent from the device in real time and immediately starts recording, recording all audio until the end of the call.

[0209] Step 3:

[0210] While the recording is in progress, the server uses speech recognition technology to convert the conversation into text, parsing the audio data into a string of characters as they are recorded.

[0211] Step 4:

[0212] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are present in the text.

[0213] Step 5:

[0214] The device is equipped with an emotion engine that analyzes the user's voice in real time during a call to detect stress levels and anxiety.

[0215] Step 6:

[0216] If a user's stress level or anxiety exceeds a certain threshold, or if a fraudulent keyword is detected, the emotion engine sends that information to the server.

[0217] Step 7:

[0218] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server sends a warning signal to the device, which then displays an alert to the user, informing them that a "scam may have occurred."

[0219] Step 8:

[0220] When the terminal receives the warning signal, it immediately terminates the call, preventing fraud before it occurs.

[0221] Step 9:

[0222] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and related information.

[0223] Step 10:

[0224] The server generates a detailed report based on the stored call logs, including the date and time of the call, the caller's details, detected fraud keywords, and sentiment analysis results.

[0225] Step 11:

[0226] The generated report is notified to the user via email and / or in-app notifications, allowing the user to receive timely, detailed information about fraud risks.

[0227] This processing flow not only detects and repels special fraud calls, but also takes into account the user's emotions, thereby protecting individuals and society as a whole from fraud.

[0228] Example 2

[0229] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] In recent years, there has been an increase in fraudulent phone calls, causing financial damage to many people. Although the accuracy of voice analysis has improved, conventional fraudulent call detection systems have limitations in their detection rate. Furthermore, there is a lack of a system that can detect the stress and anxiety felt by users while receiving fraudulent calls in real time and take appropriate measures based on that information. Therefore, to ensure personal safety, there is a need for a system that can detect fraudulent calls with high accuracy and respond immediately.

[0231] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for transmitting a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for the communication server to save the call log and generate a report after the call is terminated; a means for an emotion engine built into the terminal to analyze the user's emotions during the call and transmit the results to the communication server; and a means for the communication terminal to forcibly terminate the call if the emotion engine detects a rapid change in the user's emotions. This enables real-time detection and response to fraudulent calls, and by taking the user's emotions into consideration, it is possible to prevent fraudulent acts with greater accuracy.

[0232] A "communications terminal" is a device that allows a user to receive calls, detect call information, and transmit it to a server.

[0233] A "communication server" is a computer system that receives call information sent from a communication terminal, records the call, and analyzes the voice data.

[0234] "Call information" is information including the start time of the call, the telephone number, and the voice data during the call.

[0235] The "recording means" is a means for saving call information as audio data.

[0236] "Speech recognition technology" is a technology for converting voice data into text.

[0237] "Fraud keywords" are specific words or phrases that suggest fraudulent activity and are pre-registered in a list that the system will detect.

[0238] An "emotion engine" is a technology or software for analyzing a user's emotions from voice data.

[0239] "Call logs" are records of phone calls, including recorded audio data, transcribed text, detected fraud keywords, and sentiment analysis results.

[0240] A "warning signal" is a warning message sent to a communication terminal when it is determined that there is a high possibility of fraud.

[0241] The "call termination means" is a means for forcibly terminating a call upon receiving a warning signal or based on the analysis results of the emotion engine.

[0242] A "report" is a written or digital message that summarizes the contents of a call log and sends it to the user.

[0243] The present invention is a system relating to a communication terminal, a communication server, an emotion engine, and a user, for detecting special fraudulent calls with high accuracy and protecting users. The present invention enables real-time detection of fraudulent calls and analysis of user emotions through multiple steps throughout the entire process from the start to the end of a call.

[0244] 1. Role of communication terminals

[0245] When a user receives a call, the communication terminal detects the call information. Specifically, a communication module built into the terminal recognizes the incoming number and receives a signal to start the call. The call information is sent from the terminal to the server, and preparations for recording are made. For example, if the user receives a call from an unknown number, the terminal automatically detects the call and sends the call information to the server.

[0246] 2. Server Roles

[0247] The server begins recording based on the call information received from the device. This recording is performed using audio recording software (e.g., Audacity or a proprietary recording engine) running on the server. All audio from the beginning to the end of the call is recorded in real time. The server then converts the recorded audio data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). This text data is then compared against a pre-set list of fraudulent keywords to detect fraudulent keywords.

[0248] 3. Utilizing the Emotion Engine

[0249] The emotion engine is built into the communication terminal and analyzes the user's voice in real time during a call. The emotion engine (e.g., IBM Watson (registered trademark) Tone Analyzer) detects stress levels and anxiety from the user's voice and sends the analysis results to the server. For example, if signs of stress or anxiety are detected in the user's voice, that information is immediately transmitted to the server.

[0250] 4. Fraudulent call detection and warning

[0251] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the risk of fraud. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication device. This causes the communication device to display "Possible fraud" and terminate the call.

[0252] 5. Call log storage and reporting

[0253] After the call ends, the server saves the call log, which includes the audio data, analyzed text, detected fraud keywords, and sentiment analysis results. The next day, a detailed report of the call is sent to the user. For example, an email may be sent stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report helps users become more vigilant against fraud risks.

[0254] 6. Fail-safe function

[0255] In the unlikely event that a user's emotions change rapidly and exceed a certain threshold, the communication terminal has a fail-safe function that immediately ends the call. This function enables a quick response based on the analysis results of the emotion engine, thereby protecting the user.

[0256] Specific prompt examples

[0257] The following prompts can be input into the generative AI model to see how the system works:

[0258] Please explain in detail the procedure for the system to analyze emotions in real time when a user who receives a fraudulent call feels highly stressed and terminate calls that are likely to be fraudulent.

[0259] As a result, by combining real-time detection of fraudulent calls with user emotion analysis, the present invention can prevent fraudulent acts with greater accuracy, contributing to improved security for users and society as a whole.

[0260] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0261] Step 1: Detect incoming calls

[0262] The terminal detects an incoming call. Specifically, the terminal's built-in communications module recognizes the phone number and receives a signal to start the call. The input is the call signal received by the terminal, and the output is call initiation information. This information includes the called number, the time of the call, and the status of the call.

[0263] Step 2: Prepare to send call information

[0264] The terminal prepares to send the detected call information to the server. Specifically, the terminal converts the call information into data packets and prepares to send the data packets to the server. The input is the call start information detected by the terminal, and the output is the call information converted into the data packets.

[0265] Step 3: Receive call information and start recording

[0266] The server receives the call information sent from the device and starts recording. Specifically, audio recording software (e.g., Audacity or a proprietary recording engine) running on the server records the call content in real time. The input is the call information sent from the device, and the output is the audio data being recorded.

[0267] Step 4: Convert audio data to text

[0268] The server converts the recorded voice data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). Specifically, the speech recognition engine analyzes the voice data and converts it into text format. The input is the voice data recorded on the server, and the output is the converted text of the conversation.

[0269] Step 5: Fraudulent Keyword Analysis

[0270] The server uses the converted text data to check against a fraud keyword list to detect fraud keywords. Specifically, a fraud keyword detection module running on the server compares the text with a pre-defined list and extracts keywords that may be fraudulent. The input is the converted call content, and the output is a list of detected fraud keywords.

[0271] Step 6: Analyze user sentiment

[0272] An emotion engine (e.g., IBM Watson Tone Analyzer) built into the device analyzes the user's voice during a call. Specifically, the emotion engine detects stress levels and anxiety levels from the user's voice in real time and sends the results to a server. The input is the user's voice data, and the output is analyzed emotion data.

[0273] Step 7: Fraud risk assessment and warning

[0274] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the fraud risk. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication terminal. Specifically, if the server determines that the call is high risk, it generates a warning signal and sends it to the terminal. The input is text data and emotion data, and the output is a warning signal.

[0275] Step 8: View warnings and end calls

[0276] Based on the warning signal received from the server, the terminal displays "Possible fraud" to the user and terminates the call. Specifically, the terminal displays a warning message on the screen and terminates the current call. The input is the warning signal from the server, and the output is to display a warning to the user and terminate the call.

[0277] Step 9: Logging the call

[0278] After the call ends, the server saves the call log, which includes the recorded voice data, the converted text of the call, detected fraud keywords, the results of sentiment analysis, etc. The input is the full data of the call, and the output is the saved call log.

[0279] Step 10: Generate and notify reports

[0280] The server generates reports based on the call logs and notifies the user. Specifically, the server analyzes the call records, creates a report, and sends the report to the user's email address. The input is the saved call logs, and the output is the report sent to the user.

[0281] Step 11: Implementing the fail-safe function

[0282] The device's emotion engine detects a rapid change in the user's emotion and immediately ends the call if it exceeds a certain threshold. Specifically, if the emotion engine detects a sudden increase in stress level, the device forcibly ends the call. The input is the rapidly changing emotion data, and the output is the forced termination of the call.

[0283] In this way, the present invention provides a system that can detect special fraud calls with high accuracy through a process spanning multiple steps, and prevent damage to users before it occurs.

[0284] (Application example 2)

[0285] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0286] In recent years, fraudulent phone scams have become increasingly sophisticated, with more sophisticated methods being used to deceive people. In particular, special fraudulent phone scams targeting the elderly have caused enormous losses and become a serious social problem. Conventional fraud detection systems issue warnings based solely on keyword detection, resulting in insufficient accuracy and a high number of false alarms. Therefore, there is a need for more accurate methods of detecting and repelling fraudulent phone calls that also take into account the user's emotional state.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0288] In this invention, the server includes means for recording call information in real time, means for converting the recorded call content into text using voice recognition technology, means for analyzing and detecting fraud keywords from the text, means for analyzing the user's emotional state using an emotion engine during the call, means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected and the user's emotional analysis result indicates high stress or anxiety, and means for saving the call log and the analysis result of the user's emotional state after the call ends, generating a report, and notifying the user. This improves the accuracy of detecting and repelling fraudulent calls, allowing users to prevent damage before it occurs.

[0289] A "communication terminal" is an electronic device such as a telephone or smartphone that has the functionality to receive voice calls and perform data communications.

[0290] "Call information" refers to information including audio data and related metadata sent and received during a call.

[0291] A "communication server" is a computer system that processes and stores call information sent from a communication terminal.

[0292] "Speech recognition technology" is a technology that analyzes recorded speech as digital data and converts it into text.

[0293] "Fraud keywords" are words or phrases contained in the content of phone calls that are likely to be related to fraudulent activity.

[0294] The "emotion engine" is a system that analyzes the user's emotional state from their voice and evaluates the level of stress and anxiety.

[0295] A "warning signal" is a signal sent to alert the user when a potentially fraudulent call is detected.

[0296] A "call log" is a record that includes audio data recorded during a call and analysis results.

[0297] A "report" is a detailed explanatory document generated and sent to the user based on call logs and analysis results.

[0298] This invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of fraudulent call detection by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[0299] 1. Call monitoring and recording

[0300] First, when a communication terminal receives a call, it detects the call information and prepares to send it to the communication server. The communication server starts recording the received call information in real time. This records all audio from the start to the end of the call.

[0301] 2. Analysis of call content

[0302] While recording is in progress or once it is complete, the communications server uses speech recognition technology to convert the call into text. This text is then analyzed against a pre-defined list of fraud keywords. For example, if the caller says, "Hello, this is my son. I've been in an accident and need a security deposit," the communications server will extract text containing fraud keywords such as "son," "accident," and "security deposit."

[0303] 3. Emotional Analysis of Users by Emotion Engine

[0304] At the same time, the communication terminal is equipped with an emotion engine that analyzes the user's voice during a call. The emotion engine detects stress levels and anxiety in real time. For example, if signs of stress or anxiety are detected in the user's voice, the emotion engine sends that information to the communication server.

[0305] 4. Fraudulent call detection and warning

[0306] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal. This causes the communication terminal to display a message saying "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[0307] 5. Call log storage and reporting

[0308] After the call ends, the communication server saves the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other related information. Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day, stating, "Yesterday's call contained a possible fraudulent message. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[0309] 6. Fail-safe function

[0310] In addition, the communication terminal is equipped with a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold. This allows for the prompt termination of suspicious calls. This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables more accurate prevention of fraud by taking the user's emotions into account. This improves the safety of individuals and society as a whole.

[0311] Hardware and Software Configuration

[0312] The hardware required to realize this system includes a communication terminal (smartphone) with recording capabilities and an Internet connection. The communication server is a high-performance computer, and an open-source speech recognition library (e.g., vosk) is used for speech recognition. An HTTP request library (e.g., requests) is used to access the sentiment analysis API.

[0313] Examples of concrete examples and prompts

[0314] As a concrete example, we will explain the operation of this system by showing the prompt sentence for the generative AI model below.

[0315] Example prompt sentence:

[0316] It recognizes the call audio and analyzes fraud keywords and the user's emotional state.

[0317] If the call says, "Hello, this is my son. I've been in an accident and need a deposit," it could be a scam.

[0318] If high stress or anxiety is detected at the same time, the call will be forcibly ended and the user will be warned.

[0319] This is expected to protect users from special fraud calls and provide a safe calling environment.

[0320] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0321] Step 1:

[0322] When the communication terminal receives a call, it detects the call information and sends it to the communication server.

[0323] Input: Call audio data

[0324] Output: Sending call audio data to the communication server

[0325] Specific operation: The communication terminal detects the start of a call and transfers the call's audio data to the communication server in real time.

[0326] Step 2:

[0327] The communication server records the call information received in real time.

[0328] Input: Transferred call audio data

[0329] Output: Recorded call audio data

[0330] Specific operation: The communication server stores the call audio in a buffer and records it as audio data in real time.

[0331] Step 3:

[0332] The communication server converts the recorded conversation into text using voice recognition technology.

[0333] Input: Recorded call audio data

[0334] Output: Transcribed call

[0335] Specific operation: The communication server uses a speech recognition library such as vosk to analyze the voice data and generate text from the voice.

[0336] Step 4:

[0337] The communication server analyzes and detects fraudulent keywords from the text.

[0338] Input: Transcribed call

[0339] Output: Fraud keyword detection results (boolean or list format)

[0340] Specific operation: Matches against a certain list of keywords (e.g., "son," "accident," "security deposit") and checks whether any of the relevant words are included.

[0341] Step 5:

[0342] The communication terminal is equipped with an emotion engine that analyzes emotions from the user's voice during a call.

[0343] Input: Call audio data

[0344] Output: Analysis results of user emotions (stress level, anxiety level, etc.)

[0345] Specific operation: The emotion engine analyzes the voice data, evaluates the user's emotional state (stress, anxiety, etc.) in real time, and sends the data to the communication server.

[0346] Step 6:

[0347] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal and terminates the call.

[0348] Input: Fraud keyword detection results, user sentiment analysis results

[0349] Output: Send warning signal, terminate call

[0350] Specific operation: The communication server evaluates the fraud risk and emotional state, and if the risk is high, it sends a warning signal to the communication terminal and forcibly terminates the call.

[0351] Step 7:

[0352] After the call is completed, the communication server stores the call log.

[0353] Input: All recorded audio data, analyzed text, fraud keyword detection results, user sentiment analysis results

[0354] Output: Saved call logs

[0355] Specific operation: Call logs including call content, text conversion results, fraud keyword detection status, and sentiment analysis results are saved in a database.

[0356] Step 8:

[0357] The analysis of the user's emotional state is communicated to the user along with the call logs stored and the reports generated.

[0358] Input: Saved call logs, emotion analysis results

[0359] Output: Report notification to user

[0360] Specific operation: The communication server generates a report document based on the saved logs and analysis results, and sends it to the user's mailbox or as an app notification.

[0361] This protects users from special fraud calls and allows them to make calls with peace of mind.

[0362] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0363] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0364] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0365] [Second embodiment]

[0366] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0367] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0368] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0369] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0370] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0371] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0372] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0373] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0374] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0375] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0376] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0377] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0378] The present invention relates to a system for detecting and preventing special fraud calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof are described below.

[0379] 1. Call monitoring and recording

[0380] When the device receives a call, it immediately sends the call information to the server, which then records the received call information in real time, thereby recording all audio from the beginning to the end of the call.

[0381] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[0382] 2. Analysis of call content

[0383] While the recording is in progress or once it is complete, the server uses speech recognition technology to convert the recording into text, which is then analyzed against a pre-defined list of fraud keywords.

[0384] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[0385] 3. Fraudulent call detection and warning

[0386] If the call contains any fraudulent keywords, the server immediately sends a warning signal to the device, informing the user that "there is a possibility of fraud." The device then automatically ends the call.

[0387] To give a concrete example, when the server detects the above fraudulent keywords, the device will display the message "Possible fraud" and terminate the call, allowing the user to prevent damage before it occurs.

[0388] 4. Call log storage and reporting

[0389] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results, which is then sent to the user via email and app notification.

[0390] For example, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call." This report will encourage the user to be more vigilant and take measures to prevent further damage.

[0391] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[0392] The processing flow will be explained below.

[0393] Step 1:

[0394] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[0395] Step 2:

[0396] The server receives call information sent from the device in real time, immediately begins recording the received call, and continues recording until the call ends.

[0397] Step 3:

[0398] The server uses voice recognition technology to convert the call into text in real time while recording, where the voice data is analyzed sequentially into a string of characters.

[0399] Step 4:

[0400] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are included in the text.

[0401] Step 5:

[0402] If a fraud keyword is detected, the server immediately sends a warning signal to the terminal, which then displays an alert to the user.

[0403] Step 6:

[0404] When the device receives a warning signal, it displays a warning message on the screen saying "Possible fraud" and automatically terminates the call.

[0405] Step 7:

[0406] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, and other relevant information.

[0407] Step 8:

[0408] The server generates a detailed report based on the stored call logs, including the duration of the call, the caller's details, detected fraud keywords, and analysis results.

[0409] Step 9:

[0410] Finally, the server notifies the user of the generated report via email or in-app notification, allowing the user to receive detailed and timely information about their fraud risk.

[0411] This processing flow makes it possible to detect and repel special fraud calls in real time, improving the security of individuals and society as a whole.

[0412] Example 1

[0413] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0414] In modern society, the number of victims of special fraud phone calls is increasing, affecting many individuals. Conventional fraud prevention systems are inadequate in their ability to detect and prevent fraud, making it difficult to completely prevent damage. Given this background, there is a need for a new system that can detect fraudulent phone calls in real time and efficiently warn and repel them.

[0415] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0416] In this invention, the server includes a means for a communication terminal to receive a call and detect call information, a means for transmitting the detected call information to the communication server, a means for the communication server to record the call information in real time, a means for the communication server to convert the recorded call content into text using voice recognition technology, a means for analyzing and detecting fraud keywords from the text, a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected, a means for the communication server to save the call log and generate a report after the call ends, a means for notifying the user of the generated report, and a means for the user to be more vigilant based on the report. This enables real-time detection of fraudulent calls and prompt warnings and call termination, thereby improving the safety of individuals and society.

[0417] A "communications terminal" is an electronic device that has the function of receiving calls and detecting and transmitting call information.

[0418] "Call information" is data that includes identification information such as the caller number and the call start time when a call is made.

[0419] A "communication server" is a central management system that records call information sent from communication terminals in real time, analyzes it, and generates reports.

[0420] "Real-time recording" is the process of recording audio instantly while a call is taking place.

[0421] "Speech recognition technology" is a technology for converting recorded voice data into text format.

[0422] "Text" is character string data converted using speech recognition technology.

[0423] "Fraud keywords" are a specific set of words or phrases that indicate possible fraud.

[0424] "Analysis" is the process of extracting fraud keywords from text data and assessing the likelihood of fraud.

[0425] The "warning signal" is a notification means for informing the communication terminal or user of danger when a fraud keyword is detected.

[0426] A "call log" is recorded data that includes detailed historical information about calls.

[0427] A "report" is a detailed document of fraud risk assessment and warnings generated based on call logs.

[0428] A "user" is a person who uses a communication terminal or a communication server to use the fraudulent call detection and rejection system.

[0429] "Further vigilance" refers to an action taken by a user who has received a report to take additional crime prevention measures based on the warning.

[0430] The present invention is a system for detecting and blocking special fraud phone calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof will be described below.

[0431] This system mainly consists of a communication terminal that receives calls, a communication server that receives and analyzes call information, and a user that receives reports.

[0432] Receiving calls and notifications

[0433] When the device receives a call, it immediately sends the call information (caller ID, call start time, etc.) to the server. Specifically, the device detects the incoming call using its internal call detection module and sends the specified call information to the server via a communication protocol.

[0434] Call recording

[0435] The server starts recording the call based on the received call information. For recording, for example, VoIP (Voice over IP) technology is used to collect voice data in real time and store it in a temporary storage area. The recorded data is saved in a compressed format (e.g., MP3) and used for subsequent analysis processing.

[0436] Call transcription

[0437] While the recording is in progress or once it is complete, the server uses speech recognition technology such as the Google Cloud Speech-to-Text API to convert the recording into text, which can then be parsed into a string of characters.

[0438] Fraudulent Keyword Analysis

[0439] The server receives the text data and analyzes it against a list of fraud keywords. This analysis can be done using natural language processing technology, for example, to search for specific keywords such as "son," "accident," and "security deposit." This allows the server to detect calls that may be fraudulent.

[0440] Scam call warning and call termination

[0441] If a fraud keyword is detected, the server immediately sends a warning signal to the device. The device receives this signal, displays a message to the user saying "Possible fraud," and terminates the call. This action allows the user to prevent damage before it occurs.

[0442] Call log storage and report generation

[0443] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email or a dedicated application.

[0444] Specific examples

[0445] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server. The server records the call and converts the recording into text such as "Hello, I'm your son. I've been in an accident and need money." The server detects the fraud keywords "son," "accident," and "money" and sends a warning signal to the device. The device displays "Possible fraud" and terminates the call. The server then generates a fraud risk report and notifies the user that "The call from 07012345678 is suspected to be fraudulent."

[0446] Prompt Sentence Examples

[0447] This system can use a generative AI model (e.g., OpenAI's GPT-4). The following prompts can be used to automatically generate a description of the system:

[0448] This system aims to detect and block special fraud phone calls, and includes a series of processes for call monitoring, recording, analysis, warnings, and reporting. The main process flow is as follows:

[0449] 1. When a call starts, the device sends the call information to the server, and the server records the call.

[0450] 2. The recording is converted into text using voice recognition technology and analyzed against a list of fraud keywords.

[0451] 3. If a fraud keyword is detected, the server sends a warning signal to the terminal, notifying the user and terminating the call.

[0452] 4. After the call ends, the server stores the call log and generates a detailed fraud risk report to the user.

[0453] Use this description as a starting point to write a piece that highlights the features by adding detailed steps and the techniques used.

[0454] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[0455] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0456] Step 1:

[0457] The terminal receives a call. The terminal detects the incoming call and acquires call information such as the caller ID and the start time of the call. The input is the call information of the received call, and the output is the acquired call information. Specifically, the terminal uses its internal call detection module to detect the incoming call and collect the necessary call information.

[0458] Step 2:

[0459] The terminal sends the acquired call information to the server. The input is the call information acquired in step 1, and the output is the call information sent to the server. Specifically, the terminal uses a communication protocol to send call information such as the caller ID and call start time to the server via the network.

[0460] Step 3:

[0461] The server starts recording the call based on the received call information. The input is the call information sent from the device, and the output is the recorded call data. Specifically, the server uses VoIP technology to record the call content in real time and saves the recorded data in a temporary storage area.

[0462] Step 4:

[0463] The server converts the recorded call content into text using speech recognition technology. The input is the recorded call data, and the output is the converted text data. Specifically, the server uses a speech recognition service such as Google Cloud Speech-to-Text API to convert the voice data into text format.

[0464] Step 5:

[0465] The server analyzes the text data by comparing it with a list of fraudulent keywords. The input is the converted text data, and the output is the results of the fraudulent keyword detection. Specifically, the server uses natural language processing technology to extract fraudulent keywords such as "son," "accident," and "security deposit" from the text.

[0466] Step 6:

[0467] If a fraudulent keyword is detected, the server sends a warning signal to the terminal. The input is the detection result of the fraudulent keyword, and the output is the warning signal to the terminal. Specifically, the server generates a warning signal and sends it to the terminal via a communication protocol.

[0468] Step 7:

[0469] The terminal receives the warning signal, displays "Possible fraud" to the user, and terminates the call. The input is the warning signal sent from the server, and the output is the display of the warning message and the termination of the call. Specifically, the terminal displays the warning message on the screen based on the received warning signal and executes a command to terminate the call.

[0470] Step 8:

[0471] After the call is completed, the server saves the call log and generates a detailed fraud risk report. The input is the call details and analysis results, and the output is the saved call log and the generated report. Specifically, the server saves the log, including the call date and time, caller information, detected fraud keywords, and analysis results, in a database and creates a report based on that information.

[0472] Step 9:

[0473] The server notifies the user of the generated report. The input is the generated report, and the output is the report sent to the user. Specifically, the server sends the fraud risk report to the user's email address or app via email or a dedicated application.

[0474] (Application example 1)

[0475] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0476] In a society where special fraud phone calls are on the rise, many people, including the elderly, are at risk of becoming victims of fraud. The methods used by fraudsters are sophisticated, making it difficult for the average person to recognize and deal with them. Also, since there is no turning back once you have fallen victim to a fraud, preventative measures are important. Therefore, a system is needed that can detect fraudulent calls in real time, immediately issue a warning, and terminate the call.

[0477] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0478] In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for notifying the user if a fraud keyword is detected; a means for the communication server to save a call log and generate a report after the call ends and send the generated report to the user; a means for automatically terminating a call containing a fraud keyword; and a means for reporting the report to the user by email or app notification. This makes it possible to detect fraudulent calls in real time, immediately warn the user, and prevent damage from occurring.

[0479] "Communications terminal" refers to a device for receiving calls and detecting call information.

[0480] "Call Information" refers to information including audio data and associated metadata generated during a call.

[0481] "Communication Server" refers to a server system for receiving, recording, analyzing, and managing call information.

[0482] "Voice recognition technology" refers to technology that converts voice data into text data.

[0483] "Fraud keywords" refer to specific words or phrases used to identify suspected fraudulent calls.

[0484] The "warning signal" refers to a notification signal that alerts the user.

[0485] "Call log" refers to data that records details of calls (date and time, call recipient, content, etc.).

[0486] "Report" refers to documents or data that provide users with information generated based on call logs and analysis results.

[0487] "Real-time" refers to processing that occurs with almost no delay after an event occurs.

[0488] "Email" means a message sent or received electronically over the Internet.

[0489] "App notifications" refers to notifications sent to users through applications installed on smartphones or other digital devices.

[0490] This invention provides a system for detecting special fraud calls and protecting users. The main components of this system are a communication terminal, a communication server, voice recognition technology, fraud keyword analysis, and a means for notifying users.

[0491] 1. Operation of communication terminal

[0492] When a communication terminal receives a call, it immediately detects the call information and sends it to the communication server. If a user receives a call from an unknown number, the communication terminal detects the call and sends it to the communication server in real time.

[0493] 2. Communication Server Operation

[0494] The communications server records the received call information in real time. When the recording is in progress or is completed, it converts the recording into text using speech recognition technology. This speech recognition technology can use existing technologies such as the Google Speech Recognition API. After the conversion, the text is analyzed by comparing it with a list of pre-defined fraud keywords.

[0495] 3. Fraudulent Keyword Analysis

[0496] If the call content is something like, "Hello, this is my son. I've been in an accident and need a security deposit," the communication server will extract text containing fraud keywords such as "son," "accident," and "security deposit." If a fraud keyword is detected, the communication server will immediately send a warning signal to the communication terminal and notify the user that "there is a possibility of fraud."

[0497] 4. Call End and Notifications

[0498] If a call is determined to be fraudulent, the communication device will automatically terminate the call and issue a warning to the user. The communication server then stores the call log and generates a detailed fraud risk report, which includes the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email and app notification.

[0499] 5. Specific Examples

[0500] For example, if a mother receives a call on her smartphone from an unknown number, saying, "This is my son. I've been in an accident and need a security deposit," the system will immediately detect the fraud keywords, send a warning signal to the communication device, and terminate the call. A detailed report will then be sent to the mother's inbox.

[0501] 6. Example prompts for generative AI models

[0502] "Develop a system that records phone conversations in real time and uses voice recognition to detect fraudulent keywords. If the system detects fraud, it will warn the user and terminate the call. It will also send a detailed call report to the user via email."

[0503] The above is a specific embodiment for carrying out this invention. To ensure reproducibility and practicality, each component must be appropriately selected, and it is desirable to provide a manual detailing their operation.

[0504] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0505] Step 1:

[0506] The communication terminal receives the call and detects the call information. The input is the received call signal, and the output is the call information data. This data includes the caller's number, the call start time, the call content, etc.

[0507] Step 2:

[0508] The communication terminal transmits the detected call information to the communication server in real time. The input is the call information detected on the terminal, and the output is the call information transmitted to the communication server. This information is necessary for analysis by the communication server.

[0509] Step 3:

[0510] The communication server records the received call information in real time. The input is the call information sent from the communication terminal, and the output is the recorded voice data. This voice data becomes the basis for subsequent analysis.

[0511] Step 4:

[0512] The communication server converts the recorded conversation into text using speech recognition technology. The input is the recorded voice data, and the output is the generated text data. This conversion is performed using Google Speech Recognition API or similar.

[0513] Step 5:

[0514] The communication server analyzes and detects fraudulent keywords from the generated text data. The input is text data, and the output is the detection results of fraudulent keywords. Analysis is performed by comparing the data with a list of fraudulent keywords.

[0515] Step 6:

[0516] If a fraud keyword is detected, the communication server immediately sends a warning signal to the communication terminal and notifies the user. The input is the detection result indicating that a fraud keyword is included, and the output is the warning signal to the communication terminal and a notification to the user. This makes the user aware of the risk of fraud.

[0517] Step 7:

[0518] The communication terminal will automatically end the call when it receives the warning signal. The input is the warning signal and the output is the call end status. This allows fraudulent calls to be terminated quickly.

[0519] Step 8:

[0520] The communication server stores the call logs and generates reports after the call is completed. The input is the log data after the call is completed, and the output is the generated report. This report includes the call date and time, caller information, detected fraud keywords, and analysis results.

[0521] Step 9:

[0522] The communication server generates a report and sends it to the user via email or app notification. The input is the report data, and the output is the user's email mailbox or app notification, allowing the user to know the details of the call and the fraud risk.

[0523] The above is the specific processing flow of the program of this system.

[0524] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0525] The present invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of detecting fraudulent calls by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[0526] 1. Call monitoring and recording

[0527] When the device receives a call, it first detects the call information and prepares to send it to the server. The server then starts recording the received call information in real time, so that all audio from the beginning to the end of the call is recorded.

[0528] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[0529] 2. Analysis of call content

[0530] Once the recording is in progress or complete, the server uses speech recognition technology to convert the call into text, which is then analyzed against a pre-defined list of fraud keywords.

[0531] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[0532] 3. Emotional Analysis of Users by Emotion Engine

[0533] At the same time, the device is equipped with an emotion engine that analyzes the user's voice during a call to detect their stress level, anxiety level, and other emotions in real time.

[0534] For example, if the emotion engine detects signs of stress or anxiety in the user's voice, it will send that information to the server.

[0535] 4. Fraudulent call detection and warning

[0536] If a fraud keyword is detected and the user's sentiment analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the device, which then displays an alert to the user.

[0537] To give a specific example, the server will combine the above fraud keywords with the results of sentiment analysis to determine the risk, and the device will display the message "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[0538] 5. Call log storage and reporting

[0539] After the call is completed, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other relevant information.

[0540] Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[0541] 6. Fail-safe function

[0542] In addition, the device has a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold, allowing for the device to quickly end suspicious calls.

[0543] This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables fraud prevention with greater accuracy by taking into account the user's emotions, thereby improving the security of individuals and society as a whole.

[0544] The processing flow will be explained below.

[0545] Step 1:

[0546] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[0547] Step 2:

[0548] The server receives the call information sent from the device in real time and immediately starts recording, recording all audio until the end of the call.

[0549] Step 3:

[0550] While the recording is in progress, the server uses speech recognition technology to convert the conversation into text, parsing the audio data into a string of characters as they are recorded.

[0551] Step 4:

[0552] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are present in the text.

[0553] Step 5:

[0554] The device is equipped with an emotion engine that analyzes the user's voice in real time during a call to detect stress levels and anxiety.

[0555] Step 6:

[0556] If a user's stress level or anxiety exceeds a certain threshold, or if a fraudulent keyword is detected, the emotion engine sends that information to the server.

[0557] Step 7:

[0558] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server sends a warning signal to the device, which then displays an alert to the user, informing them that a "scam may have occurred."

[0559] Step 8:

[0560] When the terminal receives the warning signal, it immediately terminates the call, preventing fraud before it occurs.

[0561] Step 9:

[0562] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and related information.

[0563] Step 10:

[0564] The server generates a detailed report based on the stored call logs, including the date and time of the call, the caller's details, detected fraud keywords, and sentiment analysis results.

[0565] Step 11:

[0566] The generated report is notified to the user via email and / or in-app notifications, allowing the user to receive timely, detailed information about fraud risks.

[0567] This processing flow not only detects and repels special fraud calls, but also takes into account the user's emotions, thereby protecting individuals and society as a whole from fraud.

[0568] Example 2

[0569] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0570] In recent years, there has been an increase in fraudulent phone calls, causing financial damage to many people. Although the accuracy of voice analysis has improved, conventional fraudulent call detection systems have limitations in their detection rate. Furthermore, there is a lack of a system that can detect the stress and anxiety felt by users while receiving fraudulent calls in real time and take appropriate measures based on that information. Therefore, to ensure personal safety, there is a need for a system that can detect fraudulent calls with high accuracy and respond immediately.

[0571] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for transmitting a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for the communication server to save the call log and generate a report after the call is terminated; a means for an emotion engine built into the terminal to analyze the user's emotions during the call and transmit the results to the communication server; and a means for the communication terminal to forcibly terminate the call if the emotion engine detects a rapid change in the user's emotions. This enables real-time detection and response to fraudulent calls, and by taking the user's emotions into consideration, it is possible to prevent fraudulent acts with greater accuracy.

[0572] A "communications terminal" is a device that allows a user to receive calls, detect call information, and transmit it to a server.

[0573] A "communication server" is a computer system that receives call information sent from a communication terminal, records the call, and analyzes the voice data.

[0574] "Call information" is information including the start time of the call, the telephone number, and the voice data during the call.

[0575] The "recording means" is a means for saving call information as audio data.

[0576] "Speech recognition technology" is a technology for converting voice data into text.

[0577] "Fraud keywords" are specific words or phrases that suggest fraudulent activity and are pre-registered in a list that the system will detect.

[0578] An "emotion engine" is a technology or software for analyzing a user's emotions from voice data.

[0579] "Call logs" are records of phone calls, including recorded audio data, transcribed text, detected fraud keywords, and sentiment analysis results.

[0580] A "warning signal" is a warning message sent to a communication terminal when it is determined that there is a high possibility of fraud.

[0581] The "call termination means" is a means for forcibly terminating a call upon receiving a warning signal or based on the analysis results of the emotion engine.

[0582] A "report" is a written or digital message that summarizes the contents of a call log and sends it to the user.

[0583] The present invention is a system relating to a communication terminal, a communication server, an emotion engine, and a user, for detecting special fraudulent calls with high accuracy and protecting users. The present invention enables real-time detection of fraudulent calls and analysis of user emotions through multiple steps throughout the entire process from the start to the end of a call.

[0584] 1. Role of communication terminals

[0585] When a user receives a call, the communication terminal detects the call information. Specifically, a communication module built into the terminal recognizes the incoming number and receives a signal to start the call. The call information is sent from the terminal to the server, and preparations for recording are made. For example, if the user receives a call from an unknown number, the terminal automatically detects the call and sends the call information to the server.

[0586] 2. Server Roles

[0587] The server begins recording based on the call information received from the device. This recording is performed using audio recording software (e.g., Audacity or a proprietary recording engine) running on the server. All audio from the beginning to the end of the call is recorded in real time. The server then converts the recorded audio data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). This text data is then compared against a pre-set list of fraudulent keywords to detect fraudulent keywords.

[0588] 3. Utilizing the Emotion Engine

[0589] The emotion engine is built into the communication device and analyzes the user's voice in real time during a call. The emotion engine (e.g., IBM Watson Tone Analyzer) detects stress levels and anxiety from the user's voice and sends the analysis results to a server. For example, if signs of stress or anxiety are detected in the user's voice, the information is immediately transmitted to the server.

[0590] 4. Fraudulent call detection and warning

[0591] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the risk of fraud. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication device. This causes the communication device to display "Possible fraud" and terminate the call.

[0592] 5. Call log storage and reporting

[0593] After the call ends, the server saves the call log, which includes the audio data, analyzed text, detected fraud keywords, and sentiment analysis results. The next day, a detailed report of the call is sent to the user. For example, an email may be sent stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report helps users become more vigilant against fraud risks.

[0594] 6. Fail-safe function

[0595] In the unlikely event that a user's emotions change rapidly and exceed a certain threshold, the communication terminal has a fail-safe function that immediately ends the call. This function enables a quick response based on the analysis results of the emotion engine, thereby protecting the user.

[0596] Specific prompt examples

[0597] The following prompts can be input into the generative AI model to see how the system works:

[0598] Please explain in detail the procedure for the system to analyze emotions in real time when a user who receives a fraudulent call feels highly stressed and terminate calls that are likely to be fraudulent.

[0599] As a result, by combining real-time detection of fraudulent calls with user emotion analysis, the present invention can prevent fraudulent acts with greater accuracy, contributing to improved security for users and society as a whole.

[0600] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0601] Step 1: Detect incoming calls

[0602] The terminal detects an incoming call. Specifically, the terminal's built-in communications module recognizes the phone number and receives a signal to start the call. The input is the call signal received by the terminal, and the output is call initiation information. This information includes the called number, the time of the call, and the status of the call.

[0603] Step 2: Prepare to send call information

[0604] The terminal prepares to send the detected call information to the server. Specifically, the terminal converts the call information into data packets and prepares to send the data packets to the server. The input is the call start information detected by the terminal, and the output is the call information converted into the data packets.

[0605] Step 3: Receive call information and start recording

[0606] The server receives the call information sent from the device and starts recording. Specifically, audio recording software (e.g., Audacity or a proprietary recording engine) running on the server records the call content in real time. The input is the call information sent from the device, and the output is the audio data being recorded.

[0607] Step 4: Convert audio data to text

[0608] The server converts the recorded voice data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). Specifically, the speech recognition engine analyzes the voice data and converts it into text format. The input is the voice data recorded on the server, and the output is the converted text of the conversation.

[0609] Step 5: Fraudulent Keyword Analysis

[0610] The server uses the converted text data to check against a fraud keyword list to detect fraud keywords. Specifically, a fraud keyword detection module running on the server compares the text with a pre-defined list and extracts keywords that may be fraudulent. The input is the converted call content, and the output is a list of detected fraud keywords.

[0611] Step 6: Analyze user sentiment

[0612] An emotion engine (e.g., IBM Watson Tone Analyzer) built into the device analyzes the user's voice during a call. Specifically, the emotion engine detects stress levels and anxiety levels from the user's voice in real time and sends the results to a server. The input is the user's voice data, and the output is analyzed emotion data.

[0613] Step 7: Fraud risk assessment and warning

[0614] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the fraud risk. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication terminal. Specifically, if the server determines that the call is high risk, it generates a warning signal and sends it to the terminal. The input is text data and emotion data, and the output is a warning signal.

[0615] Step 8: View warnings and end calls

[0616] Based on the warning signal received from the server, the terminal displays "Possible fraud" to the user and terminates the call. Specifically, the terminal displays a warning message on the screen and terminates the current call. The input is the warning signal from the server, and the output is to display a warning to the user and terminate the call.

[0617] Step 9: Logging the call

[0618] After the call ends, the server saves the call log, which includes the recorded voice data, the converted text of the call, detected fraud keywords, the results of sentiment analysis, etc. The input is the full data of the call, and the output is the saved call log.

[0619] Step 10: Generate and notify reports

[0620] The server generates reports based on the call logs and notifies the user. Specifically, the server analyzes the call records, creates a report, and sends the report to the user's email address. The input is the saved call logs, and the output is the report sent to the user.

[0621] Step 11: Implementing the fail-safe function

[0622] The device's emotion engine detects a rapid change in the user's emotion and immediately ends the call if it exceeds a certain threshold. Specifically, if the emotion engine detects a sudden increase in stress level, the device forcibly ends the call. The input is the rapidly changing emotion data, and the output is the forced termination of the call.

[0623] In this way, the present invention provides a system that can detect special fraud calls with high accuracy through a process spanning multiple steps, and prevent damage to users before it occurs.

[0624] (Application example 2)

[0625] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0626] In recent years, fraudulent phone scams have become increasingly sophisticated, with more sophisticated methods being used to deceive people. In particular, special fraudulent phone scams targeting the elderly have caused enormous losses and become a serious social problem. Conventional fraud detection systems issue warnings based solely on keyword detection, resulting in insufficient accuracy and a high number of false alarms. Therefore, there is a need for more accurate methods of detecting and repelling fraudulent phone calls that also take into account the user's emotional state.

[0627] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0628] In this invention, the server includes means for recording call information in real time, means for converting the recorded call content into text using voice recognition technology, means for analyzing and detecting fraud keywords from the text, means for analyzing the user's emotional state using an emotion engine during the call, means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected and the user's emotional analysis result indicates high stress or anxiety, and means for saving the call log and the analysis result of the user's emotional state after the call ends, generating a report, and notifying the user. This improves the accuracy of detecting and repelling fraudulent calls, allowing users to prevent damage before it occurs.

[0629] A "communication terminal" is an electronic device such as a telephone or smartphone that has the functionality to receive voice calls and perform data communications.

[0630] "Call information" refers to information including audio data and related metadata sent and received during a call.

[0631] A "communication server" is a computer system that processes and stores call information sent from a communication terminal.

[0632] "Speech recognition technology" is a technology that analyzes recorded speech as digital data and converts it into text.

[0633] "Fraud keywords" are words or phrases contained in the content of phone calls that are likely to be related to fraudulent activity.

[0634] The "emotion engine" is a system that analyzes the user's emotional state from their voice and evaluates the level of stress and anxiety.

[0635] A "warning signal" is a signal sent to alert the user when a potentially fraudulent call is detected.

[0636] A "call log" is a record that includes audio data recorded during a call and analysis results.

[0637] A "report" is a detailed explanatory document generated and sent to the user based on call logs and analysis results.

[0638] This invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of fraudulent call detection by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[0639] 1. Call monitoring and recording

[0640] First, when a communication terminal receives a call, it detects the call information and prepares to send it to the communication server. The communication server starts recording the received call information in real time. This records all audio from the start to the end of the call.

[0641] 2. Analysis of call content

[0642] While recording is in progress or once it is complete, the communications server uses speech recognition technology to convert the call into text. This text is then analyzed against a pre-defined list of fraud keywords. For example, if the caller says, "Hello, this is my son. I've been in an accident and need a security deposit," the communications server will extract text containing fraud keywords such as "son," "accident," and "security deposit."

[0643] 3. Emotional Analysis of Users by Emotion Engine

[0644] At the same time, the communication terminal is equipped with an emotion engine that analyzes the user's voice during a call. The emotion engine detects stress levels and anxiety in real time. For example, if signs of stress or anxiety are detected in the user's voice, the emotion engine sends that information to the communication server.

[0645] 4. Fraudulent call detection and warning

[0646] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal. This causes the communication terminal to display a message saying "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[0647] 5. Call log storage and reporting

[0648] After the call ends, the communication server saves the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other related information. Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day, stating, "Yesterday's call contained a possible fraudulent message. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[0649] 6. Fail-safe function

[0650] In addition, the communication terminal is equipped with a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold. This allows for the prompt termination of suspicious calls. This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables more accurate prevention of fraud by taking the user's emotions into account. This improves the safety of individuals and society as a whole.

[0651] Hardware and Software Configuration

[0652] The hardware required to realize this system includes a communication terminal (smartphone) with recording capabilities and an Internet connection. The communication server is a high-performance computer, and an open-source speech recognition library (e.g., vosk) is used for speech recognition. An HTTP request library (e.g., requests) is used to access the sentiment analysis API.

[0653] Examples of concrete examples and prompts

[0654] As a concrete example, we will explain the operation of this system by showing the prompt sentence for the generative AI model below.

[0655] Example prompt sentence:

[0656] It recognizes the call audio and analyzes fraud keywords and the user's emotional state.

[0657] If the call says, "Hello, this is my son. I've been in an accident and need a deposit," it could be a scam.

[0658] If high stress or anxiety is detected at the same time, the call will be forcibly ended and the user will be warned.

[0659] This is expected to protect users from special fraud calls and provide a safe calling environment.

[0660] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0661] Step 1:

[0662] When the communication terminal receives a call, it detects the call information and sends it to the communication server.

[0663] Input: Call audio data

[0664] Output: Sending call audio data to the communication server

[0665] Specific operation: The communication terminal detects the start of a call and transfers the call's audio data to the communication server in real time.

[0666] Step 2:

[0667] The communication server records the call information received in real time.

[0668] Input: Transferred call audio data

[0669] Output: Recorded call audio data

[0670] Specific operation: The communication server stores the call audio in a buffer and records it as audio data in real time.

[0671] Step 3:

[0672] The communication server converts the recorded conversation into text using voice recognition technology.

[0673] Input: Recorded call audio data

[0674] Output: Transcribed call

[0675] Specific operation: The communication server uses a speech recognition library such as vosk to analyze the voice data and generate text from the voice.

[0676] Step 4:

[0677] The communication server analyzes and detects fraudulent keywords from the text.

[0678] Input: Transcribed call

[0679] Output: Fraud keyword detection results (boolean or list format)

[0680] Specific operation: Matches against a certain list of keywords (e.g., "son," "accident," "security deposit") and checks whether any of the relevant words are included.

[0681] Step 5:

[0682] The communication terminal is equipped with an emotion engine that analyzes emotions from the user's voice during a call.

[0683] Input: Call audio data

[0684] Output: Analysis results of user emotions (stress level, anxiety level, etc.)

[0685] Specific operation: The emotion engine analyzes the voice data, evaluates the user's emotional state (stress, anxiety, etc.) in real time, and sends the data to the communication server.

[0686] Step 6:

[0687] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal and terminates the call.

[0688] Input: Fraud keyword detection results, user sentiment analysis results

[0689] Output: Send warning signal, terminate call

[0690] Specific operation: The communication server evaluates the fraud risk and emotional state, and if the risk is high, it sends a warning signal to the communication terminal and forcibly terminates the call.

[0691] Step 7:

[0692] After the call is completed, the communication server stores the call log.

[0693] Input: All recorded audio data, analyzed text, fraud keyword detection results, user sentiment analysis results

[0694] Output: Saved call logs

[0695] Specific operation: Call logs including call content, text conversion results, fraud keyword detection status, and sentiment analysis results are saved in a database.

[0696] Step 8:

[0697] The analysis of the user's emotional state is communicated to the user along with the call logs stored and the reports generated.

[0698] Input: Saved call logs, emotion analysis results

[0699] Output: Report notification to user

[0700] Specific operation: The communication server generates a report document based on the saved logs and analysis results, and sends it to the user's mailbox or as an app notification.

[0701] This protects users from special fraud calls and allows them to make calls with peace of mind.

[0702] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0703] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0704] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0705] [Third embodiment]

[0706] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0707] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0708] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0709] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0710] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0711] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0712] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0713] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0714] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0715] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0716] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0717] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0718] The present invention relates to a system for detecting and preventing special fraud calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof are described below.

[0719] 1. Call monitoring and recording

[0720] When the device receives a call, it immediately sends the call information to the server, which then records the received call information in real time, thereby recording all audio from the beginning to the end of the call.

[0721] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[0722] 2. Analysis of call content

[0723] While the recording is in progress or once it is complete, the server uses speech recognition technology to convert the recording into text, which is then analyzed against a pre-defined list of fraud keywords.

[0724] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[0725] 3. Fraudulent call detection and warning

[0726] If the call contains any fraudulent keywords, the server immediately sends a warning signal to the device, informing the user that "there is a possibility of fraud." The device then automatically ends the call.

[0727] To give a concrete example, when the server detects the above fraudulent keywords, the device will display the message "Possible fraud" and terminate the call, allowing the user to prevent damage before it occurs.

[0728] 4. Call log storage and reporting

[0729] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results, which is then sent to the user via email and app notification.

[0730] For example, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call." This report will encourage the user to be more vigilant and take measures to prevent further damage.

[0731] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[0732] The processing flow will be explained below.

[0733] Step 1:

[0734] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[0735] Step 2:

[0736] The server receives call information sent from the device in real time, immediately begins recording the received call, and continues recording until the call ends.

[0737] Step 3:

[0738] The server uses voice recognition technology to convert the call into text in real time while recording, where the voice data is analyzed sequentially into a string of characters.

[0739] Step 4:

[0740] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are included in the text.

[0741] Step 5:

[0742] If a fraud keyword is detected, the server immediately sends a warning signal to the terminal, which then displays an alert to the user.

[0743] Step 6:

[0744] When the device receives a warning signal, it displays a warning message on the screen saying "Possible fraud" and automatically terminates the call.

[0745] Step 7:

[0746] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, and other relevant information.

[0747] Step 8:

[0748] The server generates a detailed report based on the stored call logs, including the duration of the call, the caller's details, detected fraud keywords, and analysis results.

[0749] Step 9:

[0750] Finally, the server notifies the user of the generated report via email or in-app notification, allowing the user to receive detailed and timely information about their fraud risk.

[0751] This processing flow makes it possible to detect and repel special fraud calls in real time, improving the security of individuals and society as a whole.

[0752] Example 1

[0753] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0754] In modern society, the number of victims of special fraud phone calls is increasing, affecting many individuals. Conventional fraud prevention systems are inadequate in their ability to detect and prevent fraud, making it difficult to completely prevent damage. Given this background, there is a need for a new system that can detect fraudulent phone calls in real time and efficiently warn and repel them.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0756] In this invention, the server includes a means for a communication terminal to receive a call and detect call information, a means for transmitting the detected call information to the communication server, a means for the communication server to record the call information in real time, a means for the communication server to convert the recorded call content into text using voice recognition technology, a means for analyzing and detecting fraud keywords from the text, a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected, a means for the communication server to save the call log and generate a report after the call ends, a means for notifying the user of the generated report, and a means for the user to be more vigilant based on the report. This enables real-time detection of fraudulent calls and prompt warnings and call termination, thereby improving the safety of individuals and society.

[0757] A "communications terminal" is an electronic device that has the function of receiving calls and detecting and transmitting call information.

[0758] "Call information" is data that includes identification information such as the caller number and the call start time when a call is made.

[0759] A "communication server" is a central management system that records call information sent from communication terminals in real time, analyzes it, and generates reports.

[0760] "Real-time recording" is the process of recording audio instantly while a call is taking place.

[0761] "Speech recognition technology" is a technology for converting recorded voice data into text format.

[0762] "Text" is character string data converted using speech recognition technology.

[0763] "Fraud keywords" are a specific set of words or phrases that indicate possible fraud.

[0764] "Analysis" is the process of extracting fraud keywords from text data and assessing the likelihood of fraud.

[0765] The "warning signal" is a notification means for informing the communication terminal or user of danger when a fraud keyword is detected.

[0766] A "call log" is recorded data that includes detailed historical information about calls.

[0767] A "report" is a detailed document of fraud risk assessment and warnings generated based on call logs.

[0768] A "user" is a person who uses a communication terminal or a communication server to use the fraudulent call detection and rejection system.

[0769] "Further vigilance" refers to an action taken by a user who has received a report to take additional crime prevention measures based on the warning.

[0770] The present invention is a system for detecting and blocking special fraud phone calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof will be described below.

[0771] This system mainly consists of a communication terminal that receives calls, a communication server that receives and analyzes call information, and a user that receives reports.

[0772] Receiving calls and notifications

[0773] When the device receives a call, it immediately sends the call information (caller ID, call start time, etc.) to the server. Specifically, the device detects the incoming call using its internal call detection module and sends the specified call information to the server via a communication protocol.

[0774] Call recording

[0775] The server starts recording the call based on the received call information. For recording, for example, VoIP (Voice over IP) technology is used to collect voice data in real time and store it in a temporary storage area. The recorded data is saved in a compressed format (e.g., MP3) and used for subsequent analysis processing.

[0776] Call transcription

[0777] While the recording is in progress or once it is complete, the server uses speech recognition technology such as the Google Cloud Speech-to-Text API to convert the recording into text, which can then be parsed into a string of characters.

[0778] Fraudulent Keyword Analysis

[0779] The server receives the text data and analyzes it against a list of fraud keywords. This analysis can be done using natural language processing technology, for example, to search for specific keywords such as "son," "accident," and "security deposit." This allows the server to detect calls that may be fraudulent.

[0780] Scam call warning and call termination

[0781] If a fraud keyword is detected, the server immediately sends a warning signal to the device. The device receives this signal, displays a message to the user saying "Possible fraud," and terminates the call. This action allows the user to prevent damage before it occurs.

[0782] Call log storage and report generation

[0783] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email or a dedicated application.

[0784] Specific examples

[0785] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server. The server records the call and converts the recording into text such as "Hello, I'm your son. I've been in an accident and need money." The server detects the fraud keywords "son," "accident," and "money" and sends a warning signal to the device. The device displays "Possible fraud" and terminates the call. The server then generates a fraud risk report and notifies the user that "The call from 07012345678 is suspected to be fraudulent."

[0786] Prompt Sentence Examples

[0787] This system can use a generative AI model (e.g., OpenAI's GPT-4). The following prompts can be used to automatically generate a description of the system:

[0788] This system aims to detect and block special fraud phone calls, and includes a series of processes for call monitoring, recording, analysis, warnings, and reporting. The main process flow is as follows:

[0789] 1. When a call starts, the device sends the call information to the server, and the server records the call.

[0790] 2. The recording is converted into text using voice recognition technology and analyzed against a list of fraud keywords.

[0791] 3. If a fraud keyword is detected, the server sends a warning signal to the terminal, notifying the user and terminating the call.

[0792] 4. After the call ends, the server stores the call log and generates a detailed fraud risk report to the user.

[0793] Use this description as a starting point to write a piece that highlights the features by adding detailed steps and the techniques used.

[0794] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[0795] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0796] Step 1:

[0797] The terminal receives a call. The terminal detects the incoming call and acquires call information such as the caller ID and the start time of the call. The input is the call information of the received call, and the output is the acquired call information. Specifically, the terminal uses its internal call detection module to detect the incoming call and collect the necessary call information.

[0798] Step 2:

[0799] The terminal sends the acquired call information to the server. The input is the call information acquired in step 1, and the output is the call information sent to the server. Specifically, the terminal uses a communication protocol to send call information such as the caller ID and call start time to the server via the network.

[0800] Step 3:

[0801] The server starts recording the call based on the received call information. The input is the call information sent from the device, and the output is the recorded call data. Specifically, the server uses VoIP technology to record the call content in real time and saves the recorded data in a temporary storage area.

[0802] Step 4:

[0803] The server converts the recorded call content into text using speech recognition technology. The input is the recorded call data, and the output is the converted text data. Specifically, the server uses a speech recognition service such as Google Cloud Speech-to-Text API to convert the voice data into text format.

[0804] Step 5:

[0805] The server analyzes the text data by comparing it with a list of fraudulent keywords. The input is the converted text data, and the output is the results of the fraudulent keyword detection. Specifically, the server uses natural language processing technology to extract fraudulent keywords such as "son," "accident," and "security deposit" from the text.

[0806] Step 6:

[0807] If a fraudulent keyword is detected, the server sends a warning signal to the terminal. The input is the detection result of the fraudulent keyword, and the output is the warning signal to the terminal. Specifically, the server generates a warning signal and sends it to the terminal via a communication protocol.

[0808] Step 7:

[0809] The terminal receives the warning signal, displays "Possible fraud" to the user, and terminates the call. The input is the warning signal sent from the server, and the output is the display of the warning message and the termination of the call. Specifically, the terminal displays the warning message on the screen based on the received warning signal and executes a command to terminate the call.

[0810] Step 8:

[0811] After the call is completed, the server saves the call log and generates a detailed fraud risk report. The input is the call details and analysis results, and the output is the saved call log and the generated report. Specifically, the server saves the log, including the call date and time, caller information, detected fraud keywords, and analysis results, in a database and creates a report based on that information.

[0812] Step 9:

[0813] The server notifies the user of the generated report. The input is the generated report, and the output is the report sent to the user. Specifically, the server sends the fraud risk report to the user's email address or app via email or a dedicated application.

[0814] (Application example 1)

[0815] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0816] In a society where special fraud phone calls are on the rise, many people, including the elderly, are at risk of becoming victims of fraud. The methods used by fraudsters are sophisticated, making it difficult for the average person to recognize and deal with them. Also, since there is no turning back once you have fallen victim to a fraud, preventative measures are important. Therefore, a system is needed that can detect fraudulent calls in real time, immediately issue a warning, and terminate the call.

[0817] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0818] In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for notifying the user if a fraud keyword is detected; a means for the communication server to save a call log and generate a report after the call ends and send the generated report to the user; a means for automatically terminating a call containing a fraud keyword; and a means for reporting the report to the user by email or app notification. This makes it possible to detect fraudulent calls in real time, immediately warn the user, and prevent damage from occurring.

[0819] "Communications terminal" refers to a device for receiving calls and detecting call information.

[0820] "Call Information" refers to information including audio data and associated metadata generated during a call.

[0821] "Communication Server" refers to a server system for receiving, recording, analyzing, and managing call information.

[0822] "Voice recognition technology" refers to technology that converts voice data into text data.

[0823] "Fraud keywords" refer to specific words or phrases used to identify suspected fraudulent calls.

[0824] The "warning signal" refers to a notification signal that alerts the user.

[0825] "Call log" refers to data that records details of calls (date and time, call recipient, content, etc.).

[0826] "Report" refers to documents or data that provide users with information generated based on call logs and analysis results.

[0827] "Real-time" refers to processing that occurs with almost no delay after an event occurs.

[0828] "Email" means a message sent or received electronically over the Internet.

[0829] "App notifications" refers to notifications sent to users through applications installed on smartphones or other digital devices.

[0830] This invention provides a system for detecting special fraud calls and protecting users. The main components of this system are a communication terminal, a communication server, voice recognition technology, fraud keyword analysis, and a means for notifying users.

[0831] 1. Operation of communication terminal

[0832] When a communication terminal receives a call, it immediately detects the call information and sends it to the communication server. If a user receives a call from an unknown number, the communication terminal detects the call and sends it to the communication server in real time.

[0833] 2. Communication Server Operation

[0834] The communications server records the received call information in real time. When the recording is in progress or is completed, it converts the recording into text using speech recognition technology. This speech recognition technology can use existing technologies such as the Google Speech Recognition API. After the conversion, the text is analyzed by comparing it with a list of pre-defined fraud keywords.

[0835] 3. Fraudulent Keyword Analysis

[0836] If the call content is something like, "Hello, this is my son. I've been in an accident and need a security deposit," the communication server will extract text containing fraud keywords such as "son," "accident," and "security deposit." If a fraud keyword is detected, the communication server will immediately send a warning signal to the communication terminal and notify the user that "there is a possibility of fraud."

[0837] 4. Call End and Notifications

[0838] If a call is determined to be fraudulent, the communication device will automatically terminate the call and issue a warning to the user. The communication server then stores the call log and generates a detailed fraud risk report, which includes the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email and app notification.

[0839] 5. Specific Examples

[0840] For example, if a mother receives a call on her smartphone from an unknown number, saying, "This is my son. I've been in an accident and need a security deposit," the system will immediately detect the fraud keywords, send a warning signal to the communication device, and terminate the call. A detailed report will then be sent to the mother's inbox.

[0841] 6. Example prompts for generative AI models

[0842] "Develop a system that records phone conversations in real time and uses voice recognition to detect fraudulent keywords. If the system detects fraud, it will warn the user and terminate the call. It will also send a detailed call report to the user via email."

[0843] The above is a specific embodiment for carrying out this invention. To ensure reproducibility and practicality, each component must be appropriately selected, and it is desirable to provide a manual detailing their operation.

[0844] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0845] Step 1:

[0846] The communication terminal receives the call and detects the call information. The input is the received call signal, and the output is the call information data. This data includes the caller's number, the call start time, the call content, etc.

[0847] Step 2:

[0848] The communication terminal transmits the detected call information to the communication server in real time. The input is the call information detected on the terminal, and the output is the call information transmitted to the communication server. This information is necessary for analysis by the communication server.

[0849] Step 3:

[0850] The communication server records the received call information in real time. The input is the call information sent from the communication terminal, and the output is the recorded voice data. This voice data becomes the basis for subsequent analysis.

[0851] Step 4:

[0852] The communication server converts the recorded conversation into text using speech recognition technology. The input is the recorded voice data, and the output is the generated text data. This conversion is performed using Google Speech Recognition API or similar.

[0853] Step 5:

[0854] The communication server analyzes and detects fraudulent keywords from the generated text data. The input is text data, and the output is the detection results of fraudulent keywords. Analysis is performed by comparing the data with a list of fraudulent keywords.

[0855] Step 6:

[0856] If a fraud keyword is detected, the communication server immediately sends a warning signal to the communication terminal and notifies the user. The input is the detection result indicating that a fraud keyword is included, and the output is the warning signal to the communication terminal and a notification to the user. This makes the user aware of the risk of fraud.

[0857] Step 7:

[0858] The communication terminal will automatically end the call when it receives the warning signal. The input is the warning signal and the output is the call end status. This allows fraudulent calls to be terminated quickly.

[0859] Step 8:

[0860] The communication server stores the call logs and generates reports after the call is completed. The input is the log data after the call is completed, and the output is the generated report. This report includes the call date and time, caller information, detected fraud keywords, and analysis results.

[0861] Step 9:

[0862] The communication server generates a report and sends it to the user via email or app notification. The input is the report data, and the output is the user's email mailbox or app notification, allowing the user to know the details of the call and the fraud risk.

[0863] The above is the specific processing flow of the program of this system.

[0864] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0865] The present invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of detecting fraudulent calls by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[0866] 1. Call monitoring and recording

[0867] When the device receives a call, it first detects the call information and prepares to send it to the server. The server then starts recording the received call information in real time, so that all audio from the beginning to the end of the call is recorded.

[0868] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[0869] 2. Analysis of call content

[0870] Once the recording is in progress or complete, the server uses speech recognition technology to convert the call into text, which is then analyzed against a pre-defined list of fraud keywords.

[0871] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[0872] 3. Emotional Analysis of Users by Emotion Engine

[0873] At the same time, the device is equipped with an emotion engine that analyzes the user's voice during a call to detect their stress level, anxiety level, and other emotions in real time.

[0874] For example, if the emotion engine detects signs of stress or anxiety in the user's voice, it will send that information to the server.

[0875] 4. Fraudulent call detection and warning

[0876] If a fraud keyword is detected and the user's sentiment analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the device, which then displays an alert to the user.

[0877] To give a specific example, the server will combine the above fraud keywords with the results of sentiment analysis to determine the risk, and the device will display the message "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[0878] 5. Call log storage and reporting

[0879] After the call is completed, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other relevant information.

[0880] Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[0881] 6. Fail-safe function

[0882] In addition, the device has a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold, allowing for the device to quickly end suspicious calls.

[0883] This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables fraud prevention with greater accuracy by taking into account the user's emotions, thereby improving the security of individuals and society as a whole.

[0884] The processing flow will be explained below.

[0885] Step 1:

[0886] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[0887] Step 2:

[0888] The server receives the call information sent from the device in real time and immediately starts recording, recording all audio until the end of the call.

[0889] Step 3:

[0890] While the recording is in progress, the server uses speech recognition technology to convert the conversation into text, parsing the audio data into a string of characters as they are recorded.

[0891] Step 4:

[0892] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are present in the text.

[0893] Step 5:

[0894] The device is equipped with an emotion engine that analyzes the user's voice in real time during a call to detect stress levels and anxiety.

[0895] Step 6:

[0896] If a user's stress level or anxiety exceeds a certain threshold, or if a fraudulent keyword is detected, the emotion engine sends that information to the server.

[0897] Step 7:

[0898] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server sends a warning signal to the device, which then displays an alert to the user, informing them that a "scam may have occurred."

[0899] Step 8:

[0900] When the terminal receives the warning signal, it immediately terminates the call, preventing fraud before it occurs.

[0901] Step 9:

[0902] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and related information.

[0903] Step 10:

[0904] The server generates a detailed report based on the stored call logs, including the date and time of the call, the caller's details, detected fraud keywords, and sentiment analysis results.

[0905] Step 11:

[0906] The generated report is notified to the user via email and / or in-app notifications, allowing the user to receive timely, detailed information about fraud risks.

[0907] This processing flow not only detects and repels special fraud calls, but also takes into account the user's emotions, thereby protecting individuals and society as a whole from fraud.

[0908] Example 2

[0909] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0910] In recent years, there has been an increase in fraudulent phone calls, causing financial damage to many people. Although the accuracy of voice analysis has improved, conventional fraudulent call detection systems have limitations in their detection rate. Furthermore, there is a lack of a system that can detect the stress and anxiety felt by users while receiving fraudulent calls in real time and take appropriate measures based on that information. Therefore, to ensure personal safety, there is a need for a system that can detect fraudulent calls with high accuracy and respond immediately.

[0911] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for transmitting a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for the communication server to save the call log and generate a report after the call is terminated; a means for an emotion engine built into the terminal to analyze the user's emotions during the call and transmit the results to the communication server; and a means for the communication terminal to forcibly terminate the call if the emotion engine detects a rapid change in the user's emotions. This enables real-time detection and response to fraudulent calls, and by taking the user's emotions into consideration, it is possible to prevent fraudulent acts with greater accuracy.

[0912] A "communications terminal" is a device that allows a user to receive calls, detect call information, and transmit it to a server.

[0913] A "communication server" is a computer system that receives call information sent from a communication terminal, records the call, and analyzes the voice data.

[0914] "Call information" is information including the start time of the call, the telephone number, and the voice data during the call.

[0915] The "recording means" is a means for saving call information as audio data.

[0916] "Speech recognition technology" is a technology for converting voice data into text.

[0917] "Fraud keywords" are specific words or phrases that suggest fraudulent activity and are pre-registered in a list that the system will detect.

[0918] An "emotion engine" is a technology or software for analyzing a user's emotions from voice data.

[0919] "Call logs" are records of phone calls, including recorded audio data, transcribed text, detected fraud keywords, and sentiment analysis results.

[0920] A "warning signal" is a warning message sent to a communication terminal when it is determined that there is a high possibility of fraud.

[0921] The "call termination means" is a means for forcibly terminating a call upon receiving a warning signal or based on the analysis results of the emotion engine.

[0922] A "report" is a written or digital message that summarizes the contents of a call log and sends it to the user.

[0923] The present invention is a system relating to a communication terminal, a communication server, an emotion engine, and a user, for detecting special fraudulent calls with high accuracy and protecting users. The present invention enables real-time detection of fraudulent calls and analysis of user emotions through multiple steps throughout the entire process from the start to the end of a call.

[0924] 1. Role of communication terminals

[0925] When a user receives a call, the communication terminal detects the call information. Specifically, a communication module built into the terminal recognizes the incoming number and receives a signal to start the call. The call information is sent from the terminal to the server, and preparations for recording are made. For example, if the user receives a call from an unknown number, the terminal automatically detects the call and sends the call information to the server.

[0926] 2. Server Roles

[0927] The server begins recording based on the call information received from the device. This recording is performed using audio recording software (e.g., Audacity or a proprietary recording engine) running on the server. All audio from the beginning to the end of the call is recorded in real time. The server then converts the recorded audio data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). This text data is then compared against a pre-set list of fraudulent keywords to detect fraudulent keywords.

[0928] 3. Utilizing the Emotion Engine

[0929] The emotion engine is built into the communication device and analyzes the user's voice in real time during a call. The emotion engine (e.g., IBM Watson Tone Analyzer) detects stress levels and anxiety from the user's voice and sends the analysis results to a server. For example, if signs of stress or anxiety are detected in the user's voice, the information is immediately transmitted to the server.

[0930] 4. Fraudulent call detection and warning

[0931] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the risk of fraud. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication device. This causes the communication device to display "Possible fraud" and terminate the call.

[0932] 5. Call log storage and reporting

[0933] After the call ends, the server saves the call log, which includes the audio data, analyzed text, detected fraud keywords, and sentiment analysis results. The next day, a detailed report of the call is sent to the user. For example, an email may be sent stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report helps users become more vigilant against fraud risks.

[0934] 6. Fail-safe function

[0935] In the unlikely event that a user's emotions change rapidly and exceed a certain threshold, the communication terminal has a fail-safe function that immediately ends the call. This function enables a quick response based on the analysis results of the emotion engine, thereby protecting the user.

[0936] Specific prompt examples

[0937] The following prompts can be input into the generative AI model to see how the system works:

[0938] Please explain in detail the procedure for the system to analyze emotions in real time when a user who receives a fraudulent call feels highly stressed and terminate calls that are likely to be fraudulent.

[0939] As a result, by combining real-time detection of fraudulent calls with user emotion analysis, the present invention can prevent fraudulent acts with greater accuracy, contributing to improved security for users and society as a whole.

[0940] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0941] Step 1: Detect incoming calls

[0942] The terminal detects an incoming call. Specifically, the terminal's built-in communications module recognizes the phone number and receives a signal to start the call. The input is the call signal received by the terminal, and the output is call initiation information. This information includes the called number, the time of the call, and the status of the call.

[0943] Step 2: Prepare to send call information

[0944] The terminal prepares to send the detected call information to the server. Specifically, the terminal converts the call information into data packets and prepares to send the data packets to the server. The input is the call start information detected by the terminal, and the output is the call information converted into the data packets.

[0945] Step 3: Receive call information and start recording

[0946] The server receives the call information sent from the device and starts recording. Specifically, audio recording software (e.g., Audacity or a proprietary recording engine) running on the server records the call content in real time. The input is the call information sent from the device, and the output is the audio data being recorded.

[0947] Step 4: Convert audio data to text

[0948] The server converts the recorded voice data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). Specifically, the speech recognition engine analyzes the voice data and converts it into text format. The input is the voice data recorded on the server, and the output is the converted text of the conversation.

[0949] Step 5: Fraudulent Keyword Analysis

[0950] The server uses the converted text data to check against a fraud keyword list to detect fraud keywords. Specifically, a fraud keyword detection module running on the server compares the text with a pre-defined list and extracts keywords that may be fraudulent. The input is the converted call content, and the output is a list of detected fraud keywords.

[0951] Step 6: Analyze user sentiment

[0952] An emotion engine (e.g., IBM Watson Tone Analyzer) built into the device analyzes the user's voice during a call. Specifically, the emotion engine detects stress levels and anxiety levels from the user's voice in real time and sends the results to a server. The input is the user's voice data, and the output is analyzed emotion data.

[0953] Step 7: Fraud risk assessment and warning

[0954] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the fraud risk. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication terminal. Specifically, if the server determines that the call is high risk, it generates a warning signal and sends it to the terminal. The input is text data and emotion data, and the output is a warning signal.

[0955] Step 8: View warnings and end calls

[0956] Based on the warning signal received from the server, the terminal displays "Possible fraud" to the user and terminates the call. Specifically, the terminal displays a warning message on the screen and terminates the current call. The input is the warning signal from the server, and the output is to display a warning to the user and terminate the call.

[0957] Step 9: Logging the call

[0958] After the call ends, the server saves the call log, which includes the recorded voice data, the converted text of the call, detected fraud keywords, the results of sentiment analysis, etc. The input is the full data of the call, and the output is the saved call log.

[0959] Step 10: Generate and notify reports

[0960] The server generates reports based on the call logs and notifies the user. Specifically, the server analyzes the call records, creates a report, and sends the report to the user's email address. The input is the saved call logs, and the output is the report sent to the user.

[0961] Step 11: Implementing the fail-safe function

[0962] The device's emotion engine detects a rapid change in the user's emotion and immediately ends the call if it exceeds a certain threshold. Specifically, if the emotion engine detects a sudden increase in stress level, the device forcibly ends the call. The input is the rapidly changing emotion data, and the output is the forced termination of the call.

[0963] In this way, the present invention provides a system that can detect special fraud calls with high accuracy through a process spanning multiple steps, and prevent damage to users before it occurs.

[0964] (Application example 2)

[0965] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0966] In recent years, fraudulent phone scams have become increasingly sophisticated, with more sophisticated methods being used to deceive people. In particular, special fraudulent phone scams targeting the elderly have caused enormous losses and become a serious social problem. Conventional fraud detection systems issue warnings based solely on keyword detection, resulting in insufficient accuracy and a high number of false alarms. Therefore, there is a need for more accurate methods of detecting and repelling fraudulent phone calls that also take into account the user's emotional state.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0968] In this invention, the server includes means for recording call information in real time, means for converting the recorded call content into text using voice recognition technology, means for analyzing and detecting fraud keywords from the text, means for analyzing the user's emotional state using an emotion engine during the call, means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected and the user's emotional analysis result indicates high stress or anxiety, and means for saving the call log and the analysis result of the user's emotional state after the call ends, generating a report, and notifying the user. This improves the accuracy of detecting and repelling fraudulent calls, allowing users to prevent damage before it occurs.

[0969] A "communication terminal" is an electronic device such as a telephone or smartphone that has the functionality to receive voice calls and perform data communications.

[0970] "Call information" refers to information including audio data and related metadata sent and received during a call.

[0971] A "communication server" is a computer system that processes and stores call information sent from a communication terminal.

[0972] "Speech recognition technology" is a technology that analyzes recorded speech as digital data and converts it into text.

[0973] "Fraud keywords" are words or phrases contained in the content of phone calls that are likely to be related to fraudulent activity.

[0974] The "emotion engine" is a system that analyzes the user's emotional state from their voice and evaluates the level of stress and anxiety.

[0975] A "warning signal" is a signal sent to alert the user when a potentially fraudulent call is detected.

[0976] A "call log" is a record that includes audio data recorded during a call and analysis results.

[0977] A "report" is a detailed explanatory document generated and sent to the user based on call logs and analysis results.

[0978] This invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of fraudulent call detection by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[0979] 1. Call monitoring and recording

[0980] First, when a communication terminal receives a call, it detects the call information and prepares to send it to the communication server. The communication server starts recording the received call information in real time. This records all audio from the start to the end of the call.

[0981] 2. Analysis of call content

[0982] While recording is in progress or once it is complete, the communications server uses speech recognition technology to convert the call into text. This text is then analyzed against a pre-defined list of fraud keywords. For example, if the caller says, "Hello, this is my son. I've been in an accident and need a security deposit," the communications server will extract text containing fraud keywords such as "son," "accident," and "security deposit."

[0983] 3. Emotional Analysis of Users by Emotion Engine

[0984] At the same time, the communication terminal is equipped with an emotion engine that analyzes the user's voice during a call. The emotion engine detects stress levels and anxiety in real time. For example, if signs of stress or anxiety are detected in the user's voice, the emotion engine sends that information to the communication server.

[0985] 4. Fraudulent call detection and warning

[0986] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal. This causes the communication terminal to display a message saying "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[0987] 5. Call log storage and reporting

[0988] After the call ends, the communication server saves the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other related information. Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day, stating, "Yesterday's call contained a possible fraudulent message. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[0989] 6. Fail-safe function

[0990] In addition, the communication terminal is equipped with a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold. This allows for the prompt termination of suspicious calls. This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables more accurate prevention of fraud by taking the user's emotions into account. This improves the safety of individuals and society as a whole.

[0991] Hardware and Software Configuration

[0992] The hardware required to realize this system includes a communication terminal (smartphone) with recording capabilities and an Internet connection. The communication server is a high-performance computer, and an open-source speech recognition library (e.g., vosk) is used for speech recognition. An HTTP request library (e.g., requests) is used to access the sentiment analysis API.

[0993] Examples of concrete examples and prompts

[0994] As a concrete example, we will explain the operation of this system by showing the prompt sentence for the generative AI model below.

[0995] Example prompt sentence:

[0996] It recognizes the call audio and analyzes fraud keywords and the user's emotional state.

[0997] If the call says, "Hello, this is my son. I've been in an accident and need a deposit," it could be a scam.

[0998] If high stress or anxiety is detected at the same time, the call will be forcibly ended and the user will be warned.

[0999] This is expected to protect users from special fraud calls and provide a safe calling environment.

[1000] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1001] Step 1:

[1002] When the communication terminal receives a call, it detects the call information and sends it to the communication server.

[1003] Input: Call audio data

[1004] Output: Sending call audio data to the communication server

[1005] Specific operation: The communication terminal detects the start of a call and transfers the call's audio data to the communication server in real time.

[1006] Step 2:

[1007] The communication server records the call information received in real time.

[1008] Input: Transferred call audio data

[1009] Output: Recorded call audio data

[1010] Specific operation: The communication server stores the call audio in a buffer and records it as audio data in real time.

[1011] Step 3:

[1012] The communication server converts the recorded conversation into text using voice recognition technology.

[1013] Input: Recorded call audio data

[1014] Output: Transcribed call

[1015] Specific operation: The communication server uses a speech recognition library such as vosk to analyze the voice data and generate text from the voice.

[1016] Step 4:

[1017] The communication server analyzes and detects fraudulent keywords from the text.

[1018] Input: Transcribed call

[1019] Output: Fraud keyword detection results (boolean or list format)

[1020] Specific operation: Matches against a certain list of keywords (e.g., "son," "accident," "security deposit") and checks whether any of the relevant words are included.

[1021] Step 5:

[1022] The communication terminal is equipped with an emotion engine that analyzes emotions from the user's voice during a call.

[1023] Input: Call audio data

[1024] Output: Analysis results of user emotions (stress level, anxiety level, etc.)

[1025] Specific operation: The emotion engine analyzes the voice data, evaluates the user's emotional state (stress, anxiety, etc.) in real time, and sends the data to the communication server.

[1026] Step 6:

[1027] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal and terminates the call.

[1028] Input: Fraud keyword detection results, user sentiment analysis results

[1029] Output: Send warning signal, terminate call

[1030] Specific operation: The communication server evaluates the fraud risk and emotional state, and if the risk is high, it sends a warning signal to the communication terminal and forcibly terminates the call.

[1031] Step 7:

[1032] After the call is completed, the communication server stores the call log.

[1033] Input: All recorded audio data, analyzed text, fraud keyword detection results, user sentiment analysis results

[1034] Output: Saved call logs

[1035] Specific operation: Call logs including call content, text conversion results, fraud keyword detection status, and sentiment analysis results are saved in a database.

[1036] Step 8:

[1037] The analysis of the user's emotional state is communicated to the user along with the call logs stored and the reports generated.

[1038] Input: Saved call logs, emotion analysis results

[1039] Output: Report notification to user

[1040] Specific operation: The communication server generates a report document based on the saved logs and analysis results, and sends it to the user's mailbox or as an app notification.

[1041] This protects users from special fraud calls and allows them to make calls with peace of mind.

[1042] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1043] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1044] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1045] [Fourth embodiment]

[1046] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1047] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1048] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1049] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1050] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1051] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1052] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1053] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1054] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1055] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1056] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1057] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1058] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1059] The present invention relates to a system for detecting and preventing special fraud calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof are described below.

[1060] 1. Call monitoring and recording

[1061] When the device receives a call, it immediately sends the call information to the server, which then records the received call information in real time, thereby recording all audio from the beginning to the end of the call.

[1062] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[1063] 2. Analysis of call content

[1064] While the recording is in progress or once it is complete, the server uses speech recognition technology to convert the recording into text, which is then analyzed against a pre-defined list of fraud keywords.

[1065] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[1066] 3. Fraudulent call detection and warning

[1067] If the call contains any fraudulent keywords, the server immediately sends a warning signal to the device, informing the user that "there is a possibility of fraud." The device then automatically ends the call.

[1068] To give a concrete example, when the server detects the above fraudulent keywords, the device will display the message "Possible fraud" and terminate the call, allowing the user to prevent damage before it occurs.

[1069] 4. Call log storage and reporting

[1070] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results, which is then sent to the user via email and app notification.

[1071] For example, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call." This report will encourage the user to be more vigilant and take measures to prevent further damage.

[1072] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[1073] The processing flow will be explained below.

[1074] Step 1:

[1075] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[1076] Step 2:

[1077] The server receives call information sent from the device in real time, immediately begins recording the received call, and continues recording until the call ends.

[1078] Step 3:

[1079] The server uses voice recognition technology to convert the call into text in real time while recording, where the voice data is analyzed sequentially into a string of characters.

[1080] Step 4:

[1081] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are included in the text.

[1082] Step 5:

[1083] If a fraud keyword is detected, the server immediately sends a warning signal to the terminal, which then displays an alert to the user.

[1084] Step 6:

[1085] When the device receives a warning signal, it displays a warning message on the screen saying "Possible fraud" and automatically terminates the call.

[1086] Step 7:

[1087] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, and other relevant information.

[1088] Step 8:

[1089] The server generates a detailed report based on the stored call logs, including the duration of the call, the caller's details, detected fraud keywords, and analysis results.

[1090] Step 9:

[1091] Finally, the server notifies the user of the generated report via email or in-app notification, allowing the user to receive detailed and timely information about their fraud risk.

[1092] This processing flow makes it possible to detect and repel special fraud calls in real time, improving the security of individuals and society as a whole.

[1093] Example 1

[1094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1095] In modern society, the number of victims of special fraud phone calls is increasing, affecting many individuals. Conventional fraud prevention systems are inadequate in their ability to detect and prevent fraud, making it difficult to completely prevent damage. Given this background, there is a need for a new system that can detect fraudulent phone calls in real time and efficiently warn and repel them.

[1096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1097] In this invention, the server includes a means for a communication terminal to receive a call and detect call information, a means for transmitting the detected call information to the communication server, a means for the communication server to record the call information in real time, a means for the communication server to convert the recorded call content into text using voice recognition technology, a means for analyzing and detecting fraud keywords from the text, a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected, a means for the communication server to save the call log and generate a report after the call ends, a means for notifying the user of the generated report, and a means for the user to be more vigilant based on the report. This enables real-time detection of fraudulent calls and prompt warnings and call termination, thereby improving the safety of individuals and society.

[1098] A "communications terminal" is an electronic device that has the function of receiving calls and detecting and transmitting call information.

[1099] "Call information" is data that includes identification information such as the caller number and the call start time when a call is made.

[1100] A "communication server" is a central management system that records call information sent from communication terminals in real time, analyzes it, and generates reports.

[1101] "Real-time recording" is the process of recording audio instantly while a call is taking place.

[1102] "Speech recognition technology" is a technology for converting recorded voice data into text format.

[1103] "Text" is character string data converted using speech recognition technology.

[1104] "Fraud keywords" are a specific set of words or phrases that indicate possible fraud.

[1105] "Analysis" is the process of extracting fraud keywords from text data and assessing the likelihood of fraud.

[1106] The "warning signal" is a notification means for informing the communication terminal or user of danger when a fraud keyword is detected.

[1107] A "call log" is recorded data that includes detailed historical information about calls.

[1108] A "report" is a detailed document of fraud risk assessment and warnings generated based on call logs.

[1109] A "user" is a person who uses a communication terminal or a communication server to use the fraudulent call detection and rejection system.

[1110] "Further vigilance" refers to an action taken by a user who has received a report to take additional crime prevention measures based on the warning.

[1111] The present invention is a system for detecting and blocking special fraud phone calls, and includes a series of processes for mainly monitoring, recording, analyzing, warning, and reporting calls. This system is related to communication terminals, communication servers, and users, and specific embodiments thereof will be described below.

[1112] This system mainly consists of a communication terminal that receives calls, a communication server that receives and analyzes call information, and a user that receives reports.

[1113] Receiving calls and notifications

[1114] When the device receives a call, it immediately sends the call information (caller ID, call start time, etc.) to the server. Specifically, the device detects the incoming call using its internal call detection module and sends the specified call information to the server via a communication protocol.

[1115] Call recording

[1116] The server starts recording the call based on the received call information. For recording, for example, VoIP (Voice over IP) technology is used to collect voice data in real time and store it in a temporary storage area. The recorded data is saved in a compressed format (e.g., MP3) and used for subsequent analysis processing.

[1117] Call transcription

[1118] While the recording is in progress or once it is complete, the server uses speech recognition technology such as the Google Cloud Speech-to-Text API to convert the recording into text, which can then be parsed into a string of characters.

[1119] Fraudulent Keyword Analysis

[1120] The server receives the text data and analyzes it against a list of fraud keywords. This analysis can be done using natural language processing technology, for example, to search for specific keywords such as "son," "accident," and "security deposit." This allows the server to detect calls that may be fraudulent.

[1121] Scam call warning and call termination

[1122] If a fraud keyword is detected, the server immediately sends a warning signal to the device. The device receives this signal, displays a message to the user saying "Possible fraud," and terminates the call. This action allows the user to prevent damage before it occurs.

[1123] Call log storage and report generation

[1124] After the call is completed, the server stores the call log and generates a detailed fraud risk report, including the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email or a dedicated application.

[1125] Specific examples

[1126] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server. The server records the call and converts the recording into text such as "Hello, I'm your son. I've been in an accident and need money." The server detects the fraud keywords "son," "accident," and "money" and sends a warning signal to the device. The device displays "Possible fraud" and terminates the call. The server then generates a fraud risk report and notifies the user that "The call from 07012345678 is suspected to be fraudulent."

[1127] Prompt Sentence Examples

[1128] This system can use a generative AI model (e.g., OpenAI's GPT-4). The following prompts can be used to automatically generate a description of the system:

[1129] This system aims to detect and block special fraud phone calls, and includes a series of processes for call monitoring, recording, analysis, warnings, and reporting. The main process flow is as follows:

[1130] 1. When a call starts, the device sends the call information to the server, and the server records the call.

[1131] 2. The recording is converted into text using voice recognition technology and analyzed against a list of fraud keywords.

[1132] 3. If a fraud keyword is detected, the server sends a warning signal to the terminal, notifying the user and terminating the call.

[1133] 4. After the call ends, the server stores the call log and generates a detailed fraud risk report to the user.

[1134] Use this description as a starting point to write a piece that highlights the features by adding detailed steps and the techniques used.

[1135] According to the above-described embodiments, the present invention effectively detects and repels special fraud calls, thereby improving the security of individuals and society.

[1136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1137] Step 1:

[1138] The terminal receives a call. The terminal detects the incoming call and acquires call information such as the caller ID and the start time of the call. The input is the call information of the received call, and the output is the acquired call information. Specifically, the terminal uses its internal call detection module to detect the incoming call and collect the necessary call information.

[1139] Step 2:

[1140] The terminal sends the acquired call information to the server. The input is the call information acquired in step 1, and the output is the call information sent to the server. Specifically, the terminal uses a communication protocol to send call information such as the caller ID and call start time to the server via the network.

[1141] Step 3:

[1142] The server starts recording the call based on the received call information. The input is the call information sent from the device, and the output is the recorded call data. Specifically, the server uses VoIP technology to record the call content in real time and saves the recorded data in a temporary storage area.

[1143] Step 4:

[1144] The server converts the recorded call content into text using speech recognition technology. The input is the recorded call data, and the output is the converted text data. Specifically, the server uses a speech recognition service such as Google Cloud Speech-to-Text API to convert the voice data into text format.

[1145] Step 5:

[1146] The server analyzes the text data by comparing it with a list of fraudulent keywords. The input is the converted text data, and the output is the results of the fraudulent keyword detection. Specifically, the server uses natural language processing technology to extract fraudulent keywords such as "son," "accident," and "security deposit" from the text.

[1147] Step 6:

[1148] If a fraudulent keyword is detected, the server sends a warning signal to the terminal. The input is the detection result of the fraudulent keyword, and the output is the warning signal to the terminal. Specifically, the server generates a warning signal and sends it to the terminal via a communication protocol.

[1149] Step 7:

[1150] The terminal receives the warning signal, displays "Possible fraud" to the user, and terminates the call. The input is the warning signal sent from the server, and the output is the display of the warning message and the termination of the call. Specifically, the terminal displays the warning message on the screen based on the received warning signal and executes a command to terminate the call.

[1151] Step 8:

[1152] After the call is completed, the server saves the call log and generates a detailed fraud risk report. The input is the call details and analysis results, and the output is the saved call log and the generated report. Specifically, the server saves the log, including the call date and time, caller information, detected fraud keywords, and analysis results, in a database and creates a report based on that information.

[1153] Step 9:

[1154] The server notifies the user of the generated report. The input is the generated report, and the output is the report sent to the user. Specifically, the server sends the fraud risk report to the user's email address or app via email or a dedicated application.

[1155] (Application example 1)

[1156] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1157] In a society where special fraud phone calls are on the rise, many people, including the elderly, are at risk of becoming victims of fraud. The methods used by fraudsters are sophisticated, making it difficult for the average person to recognize and deal with them. Also, since there is no turning back once you have fallen victim to a fraud, preventative measures are important. Therefore, a system is needed that can detect fraudulent calls in real time, immediately issue a warning, and terminate the call.

[1158] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1159] In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for notifying the user if a fraud keyword is detected; a means for the communication server to save a call log and generate a report after the call ends and send the generated report to the user; a means for automatically terminating a call containing a fraud keyword; and a means for reporting the report to the user by email or app notification. This makes it possible to detect fraudulent calls in real time, immediately warn the user, and prevent damage from occurring.

[1160] "Communications terminal" refers to a device for receiving calls and detecting call information.

[1161] "Call Information" refers to information including audio data and associated metadata generated during a call.

[1162] "Communication Server" refers to a server system for receiving, recording, analyzing, and managing call information.

[1163] "Voice recognition technology" refers to technology that converts voice data into text data.

[1164] "Fraud keywords" refer to specific words or phrases used to identify suspected fraudulent calls.

[1165] The "warning signal" refers to a notification signal that alerts the user.

[1166] "Call log" refers to data that records details of calls (date and time, call recipient, content, etc.).

[1167] "Report" refers to documents or data that provide users with information generated based on call logs and analysis results.

[1168] "Real-time" refers to processing that occurs with almost no delay after an event occurs.

[1169] "Email" means a message sent or received electronically over the Internet.

[1170] "App notifications" refers to notifications sent to users through applications installed on smartphones or other digital devices.

[1171] This invention provides a system for detecting special fraud calls and protecting users. The main components of this system are a communication terminal, a communication server, voice recognition technology, fraud keyword analysis, and a means for notifying users.

[1172] 1. Operation of communication terminal

[1173] When a communication terminal receives a call, it immediately detects the call information and sends it to the communication server. If a user receives a call from an unknown number, the communication terminal detects the call and sends it to the communication server in real time.

[1174] 2. Communication Server Operation

[1175] The communications server records the received call information in real time. When the recording is in progress or is completed, it converts the recording into text using speech recognition technology. This speech recognition technology can use existing technologies such as the Google Speech Recognition API. After the conversion, the text is analyzed by comparing it with a list of pre-defined fraud keywords.

[1176] 3. Fraudulent Keyword Analysis

[1177] If the call content is something like, "Hello, this is my son. I've been in an accident and need a security deposit," the communication server will extract text containing fraud keywords such as "son," "accident," and "security deposit." If a fraud keyword is detected, the communication server will immediately send a warning signal to the communication terminal and notify the user that "there is a possibility of fraud."

[1178] 4. Call End and Notifications

[1179] If a call is determined to be fraudulent, the communication device will automatically terminate the call and issue a warning to the user. The communication server then stores the call log and generates a detailed fraud risk report, which includes the call date and time, caller information, detected fraud keywords, and analysis results. The report is then sent to the user via email and app notification.

[1180] 5. Specific Examples

[1181] For example, if a mother receives a call on her smartphone from an unknown number, saying, "This is my son. I've been in an accident and need a security deposit," the system will immediately detect the fraud keywords, send a warning signal to the communication device, and terminate the call. A detailed report will then be sent to the mother's inbox.

[1182] 6. Example prompts for generative AI models

[1183] "Develop a system that records phone conversations in real time and uses voice recognition to detect fraudulent keywords. If the system detects fraud, it will warn the user and terminate the call. It will also send a detailed call report to the user via email."

[1184] The above is a specific embodiment for carrying out this invention. To ensure reproducibility and practicality, each component must be appropriately selected, and it is desirable to provide a manual detailing their operation.

[1185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1186] Step 1:

[1187] The communication terminal receives the call and detects the call information. The input is the received call signal, and the output is the call information data. This data includes the caller's number, the call start time, the call content, etc.

[1188] Step 2:

[1189] The communication terminal transmits the detected call information to the communication server in real time. The input is the call information detected on the terminal, and the output is the call information transmitted to the communication server. This information is necessary for analysis by the communication server.

[1190] Step 3:

[1191] The communication server records the received call information in real time. The input is the call information sent from the communication terminal, and the output is the recorded voice data. This voice data becomes the basis for subsequent analysis.

[1192] Step 4:

[1193] The communication server converts the recorded conversation into text using speech recognition technology. The input is the recorded voice data, and the output is the generated text data. This conversion is performed using Google Speech Recognition API or similar.

[1194] Step 5:

[1195] The communication server analyzes and detects fraudulent keywords from the generated text data. The input is text data, and the output is the detection results of fraudulent keywords. Analysis is performed by comparing the data with a list of fraudulent keywords.

[1196] Step 6:

[1197] If a fraud keyword is detected, the communication server immediately sends a warning signal to the communication terminal and notifies the user. The input is the detection result indicating that a fraud keyword is included, and the output is the warning signal to the communication terminal and a notification to the user. This makes the user aware of the risk of fraud.

[1198] Step 7:

[1199] The communication terminal will automatically end the call when it receives the warning signal. The input is the warning signal and the output is the call end status. This allows fraudulent calls to be terminated quickly.

[1200] Step 8:

[1201] The communication server stores the call logs and generates reports after the call is completed. The input is the log data after the call is completed, and the output is the generated report. This report includes the call date and time, caller information, detected fraud keywords, and analysis results.

[1202] Step 9:

[1203] The communication server generates a report and sends it to the user via email or app notification. The input is the report data, and the output is the user's email mailbox or app notification, allowing the user to know the details of the call and the fraud risk.

[1204] The above is the specific processing flow of the program of this system.

[1205] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1206] The present invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of detecting fraudulent calls by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[1207] 1. Call monitoring and recording

[1208] When the device receives a call, it first detects the call information and prepares to send it to the server. The server then starts recording the received call information in real time, so that all audio from the beginning to the end of the call is recorded.

[1209] As a concrete example, consider a situation where a user receives a call from an unknown number. The device detects the call and sends it to the server, which then starts recording the call.

[1210] 2. Analysis of call content

[1211] Once the recording is in progress or complete, the server uses speech recognition technology to convert the call into text, which is then analyzed against a pre-defined list of fraud keywords.

[1212] For example, if the call content is "Hello, this is my son. I've been in an accident and need a security deposit," the server will extract text containing fraudulent keywords such as "son," "accident," and "security deposit."

[1213] 3. Emotional Analysis of Users by Emotion Engine

[1214] At the same time, the device is equipped with an emotion engine that analyzes the user's voice during a call to detect their stress level, anxiety level, and other emotions in real time.

[1215] For example, if the emotion engine detects signs of stress or anxiety in the user's voice, it will send that information to the server.

[1216] 4. Fraudulent call detection and warning

[1217] If a fraud keyword is detected and the user's sentiment analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the device, which then displays an alert to the user.

[1218] To give a specific example, the server will combine the above fraud keywords with the results of sentiment analysis to determine the risk, and the device will display the message "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[1219] 5. Call log storage and reporting

[1220] After the call is completed, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other relevant information.

[1221] Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[1222] 6. Fail-safe function

[1223] In addition, the device has a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold, allowing for the device to quickly end suspicious calls.

[1224] This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables fraud prevention with greater accuracy by taking into account the user's emotions, thereby improving the security of individuals and society as a whole.

[1225] The processing flow will be explained below.

[1226] Step 1:

[1227] The terminal receives a new call and detects the call information. After detecting the call start signal, the terminal prepares to send the call information to the server.

[1228] Step 2:

[1229] The server receives the call information sent from the device in real time and immediately starts recording, recording all audio until the end of the call.

[1230] Step 3:

[1231] While the recording is in progress, the server uses speech recognition technology to convert the conversation into text, parsing the audio data into a string of characters as they are recorded.

[1232] Step 4:

[1233] The server analyzes the generated text and checks it against a pre-defined list of fraudulent keywords to see if any of the keywords are present in the text.

[1234] Step 5:

[1235] The device is equipped with an emotion engine that analyzes the user's voice in real time during a call to detect stress levels and anxiety.

[1236] Step 6:

[1237] If a user's stress level or anxiety exceeds a certain threshold, or if a fraudulent keyword is detected, the emotion engine sends that information to the server.

[1238] Step 7:

[1239] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server sends a warning signal to the device, which then displays an alert to the user, informing them that a "scam may have occurred."

[1240] Step 8:

[1241] When the terminal receives the warning signal, it immediately terminates the call, preventing fraud before it occurs.

[1242] Step 9:

[1243] After the call ends, the server stores the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and related information.

[1244] Step 10:

[1245] The server generates a detailed report based on the stored call logs, including the date and time of the call, the caller's details, detected fraud keywords, and sentiment analysis results.

[1246] Step 11:

[1247] The generated report is notified to the user via email and / or in-app notifications, allowing the user to receive timely, detailed information about fraud risks.

[1248] This processing flow not only detects and repels special fraud calls, but also takes into account the user's emotions, thereby protecting individuals and society as a whole from fraud.

[1249] Example 2

[1250] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1251] In recent years, there has been an increase in fraudulent phone calls, causing financial damage to many people. Although the accuracy of voice analysis has improved, conventional fraudulent call detection systems have limitations in their detection rate. Furthermore, there is a lack of a system that can detect the stress and anxiety felt by users while receiving fraudulent calls in real time and take appropriate measures based on that information. Therefore, to ensure personal safety, there is a need for a system that can detect fraudulent calls with high accuracy and respond immediately.

[1252] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a communication terminal to receive a call and detect call information; a means for transmitting the detected call information to the communication server; a means for the communication server to record the call information in real time; a means for the communication server to convert the recorded call content into text using voice recognition technology; a means for analyzing and detecting fraud keywords from the text; a means for transmitting a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; a means for the communication server to save the call log and generate a report after the call is terminated; a means for an emotion engine built into the terminal to analyze the user's emotions during the call and transmit the results to the communication server; and a means for the communication terminal to forcibly terminate the call if the emotion engine detects a rapid change in the user's emotions. This enables real-time detection and response to fraudulent calls, and by taking the user's emotions into consideration, it is possible to prevent fraudulent acts with greater accuracy.

[1253] A "communications terminal" is a device that allows a user to receive calls, detect call information, and transmit it to a server.

[1254] A "communication server" is a computer system that receives call information sent from a communication terminal, records the call, and analyzes the voice data.

[1255] "Call information" is information including the start time of the call, the telephone number, and the voice data during the call.

[1256] The "recording means" is a means for saving call information as audio data.

[1257] "Speech recognition technology" is a technology for converting voice data into text.

[1258] "Fraud keywords" are specific words or phrases that suggest fraudulent activity and are pre-registered in a list that the system will detect.

[1259] An "emotion engine" is a technology or software for analyzing a user's emotions from voice data.

[1260] "Call logs" are records of phone calls, including recorded audio data, transcribed text, detected fraud keywords, and sentiment analysis results.

[1261] A "warning signal" is a warning message sent to a communication terminal when it is determined that there is a high possibility of fraud.

[1262] The "call termination means" is a means for forcibly terminating a call upon receiving a warning signal or based on the analysis results of the emotion engine.

[1263] A "report" is a written or digital message that summarizes the contents of a call log and sends it to the user.

[1264] The present invention is a system relating to a communication terminal, a communication server, an emotion engine, and a user, for detecting special fraudulent calls with high accuracy and protecting users. The present invention enables real-time detection of fraudulent calls and analysis of user emotions through multiple steps throughout the entire process from the start to the end of a call.

[1265] 1. Role of communication terminals

[1266] When a user receives a call, the communication terminal detects the call information. Specifically, a communication module built into the terminal recognizes the incoming number and receives a signal to start the call. The call information is sent from the terminal to the server, and preparations for recording are made. For example, if the user receives a call from an unknown number, the terminal automatically detects the call and sends the call information to the server.

[1267] 2. Server Roles

[1268] The server begins recording based on the call information received from the device. This recording is performed using audio recording software (e.g., Audacity or a proprietary recording engine) running on the server. All audio from the beginning to the end of the call is recorded in real time. The server then converts the recorded audio data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). This text data is then compared against a pre-set list of fraudulent keywords to detect fraudulent keywords.

[1269] 3. Utilizing the Emotion Engine

[1270] The emotion engine is built into the communication device and analyzes the user's voice in real time during a call. The emotion engine (e.g., IBM Watson Tone Analyzer) detects stress levels and anxiety from the user's voice and sends the analysis results to a server. For example, if signs of stress or anxiety are detected in the user's voice, the information is immediately transmitted to the server.

[1271] 4. Fraudulent call detection and warning

[1272] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the risk of fraud. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication device. This causes the communication device to display "Possible fraud" and terminate the call.

[1273] 5. Call log storage and reporting

[1274] After the call ends, the server saves the call log, which includes the audio data, analyzed text, detected fraud keywords, and sentiment analysis results. The next day, a detailed report of the call is sent to the user. For example, an email may be sent stating, "Yesterday's call contained a possible fraudulent call. Your stress level was high." This report helps users become more vigilant against fraud risks.

[1275] 6. Fail-safe function

[1276] In the unlikely event that a user's emotions change rapidly and exceed a certain threshold, the communication terminal has a fail-safe function that immediately ends the call. This function enables a quick response based on the analysis results of the emotion engine, thereby protecting the user.

[1277] Specific prompt examples

[1278] The following prompts can be input into the generative AI model to see how the system works:

[1279] Please explain in detail the procedure for the system to analyze emotions in real time when a user who receives a fraudulent call feels highly stressed and terminate calls that are likely to be fraudulent.

[1280] As a result, by combining real-time detection of fraudulent calls with user emotion analysis, the present invention can prevent fraudulent acts with greater accuracy, contributing to improved security for users and society as a whole.

[1281] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1282] Step 1: Detect incoming calls

[1283] The terminal detects an incoming call. Specifically, the terminal's built-in communications module recognizes the phone number and receives a signal to start the call. The input is the call signal received by the terminal, and the output is call initiation information. This information includes the called number, the time of the call, and the status of the call.

[1284] Step 2: Prepare to send call information

[1285] The terminal prepares to send the detected call information to the server. Specifically, the terminal converts the call information into data packets and prepares to send the data packets to the server. The input is the call start information detected by the terminal, and the output is the call information converted into the data packets.

[1286] Step 3: Receive call information and start recording

[1287] The server receives the call information sent from the device and starts recording. Specifically, audio recording software (e.g., Audacity or a proprietary recording engine) running on the server records the call content in real time. The input is the call information sent from the device, and the output is the audio data being recorded.

[1288] Step 4: Convert audio data to text

[1289] The server converts the recorded voice data into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text API). Specifically, the speech recognition engine analyzes the voice data and converts it into text format. The input is the voice data recorded on the server, and the output is the converted text of the conversation.

[1290] Step 5: Fraudulent Keyword Analysis

[1291] The server uses the converted text data to check against a fraud keyword list to detect fraud keywords. Specifically, a fraud keyword detection module running on the server compares the text with a pre-defined list and extracts keywords that may be fraudulent. The input is the converted call content, and the output is a list of detected fraud keywords.

[1292] Step 6: Analyze user sentiment

[1293] An emotion engine (e.g., IBM Watson Tone Analyzer) built into the device analyzes the user's voice during a call. Specifically, the emotion engine detects stress levels and anxiety levels from the user's voice in real time and sends the results to a server. The input is the user's voice data, and the output is analyzed emotion data.

[1294] Step 7: Fraud risk assessment and warning

[1295] The server combines the text data converted by speech recognition with the results of emotion analysis to determine the fraud risk. If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the server immediately sends a warning signal to the communication terminal. Specifically, if the server determines that the call is high risk, it generates a warning signal and sends it to the terminal. The input is text data and emotion data, and the output is a warning signal.

[1296] Step 8: View warnings and end calls

[1297] Based on the warning signal received from the server, the terminal displays "Possible fraud" to the user and terminates the call. Specifically, the terminal displays a warning message on the screen and terminates the current call. The input is the warning signal from the server, and the output is to display a warning to the user and terminate the call.

[1298] Step 9: Logging the call

[1299] After the call ends, the server saves the call log, which includes the recorded voice data, the converted text of the call, detected fraud keywords, the results of sentiment analysis, etc. The input is the full data of the call, and the output is the saved call log.

[1300] Step 10: Generate and notify reports

[1301] The server generates reports based on the call logs and notifies the user. Specifically, the server analyzes the call records, creates a report, and sends the report to the user's email address. The input is the saved call logs, and the output is the report sent to the user.

[1302] Step 11: Implementing the fail-safe function

[1303] The device's emotion engine detects a rapid change in the user's emotion and immediately ends the call if it exceeds a certain threshold. Specifically, if the emotion engine detects a sudden increase in stress level, the device forcibly ends the call. The input is the rapidly changing emotion data, and the output is the forced termination of the call.

[1304] In this way, the present invention provides a system that can detect special fraud calls with high accuracy through a process spanning multiple steps, and prevent damage to users before it occurs.

[1305] (Application example 2)

[1306] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1307] In recent years, fraudulent phone scams have become increasingly sophisticated, with more sophisticated methods being used to deceive people. In particular, special fraudulent phone scams targeting the elderly have caused enormous losses and become a serious social problem. Conventional fraud detection systems issue warnings based solely on keyword detection, resulting in insufficient accuracy and a high number of false alarms. Therefore, there is a need for more accurate methods of detecting and repelling fraudulent phone calls that also take into account the user's emotional state.

[1308] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1309] In this invention, the server includes means for recording call information in real time, means for converting the recorded call content into text using voice recognition technology, means for analyzing and detecting fraud keywords from the text, means for analyzing the user's emotional state using an emotion engine during the call, means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected and the user's emotional analysis result indicates high stress or anxiety, and means for saving the call log and the analysis result of the user's emotional state after the call ends, generating a report, and notifying the user. This improves the accuracy of detecting and repelling fraudulent calls, allowing users to prevent damage before it occurs.

[1310] A "communication terminal" is an electronic device such as a telephone or smartphone that has the functionality to receive voice calls and perform data communications.

[1311] "Call information" refers to information including audio data and related metadata sent and received during a call.

[1312] A "communication server" is a computer system that processes and stores call information sent from a communication terminal.

[1313] "Speech recognition technology" is a technology that analyzes recorded speech as digital data and converts it into text.

[1314] "Fraud keywords" are words or phrases contained in the content of phone calls that are likely to be related to fraudulent activity.

[1315] The "emotion engine" is a system that analyzes the user's emotional state from their voice and evaluates the level of stress and anxiety.

[1316] A "warning signal" is a signal sent to alert the user when a potentially fraudulent call is detected.

[1317] A "call log" is a record that includes audio data recorded during a call and analysis results.

[1318] A "report" is a detailed explanatory document generated and sent to the user based on call logs and analysis results.

[1319] This invention relates to a system for detecting and repelling special fraudulent calls, and further to improving the accuracy of fraudulent call detection by combining an emotion engine that recognizes the user's emotions. This system relates to a communication terminal, a communication server, an emotion engine, and a user, and specific embodiments thereof will be described below.

[1320] 1. Call monitoring and recording

[1321] First, when a communication terminal receives a call, it detects the call information and prepares to send it to the communication server. The communication server starts recording the received call information in real time. This records all audio from the start to the end of the call.

[1322] 2. Analysis of call content

[1323] While recording is in progress or once it is complete, the communications server uses speech recognition technology to convert the call into text. This text is then analyzed against a pre-defined list of fraud keywords. For example, if the caller says, "Hello, this is my son. I've been in an accident and need a security deposit," the communications server will extract text containing fraud keywords such as "son," "accident," and "security deposit."

[1324] 3. Emotional Analysis of Users by Emotion Engine

[1325] At the same time, the communication terminal is equipped with an emotion engine that analyzes the user's voice during a call. The emotion engine detects stress levels and anxiety in real time. For example, if signs of stress or anxiety are detected in the user's voice, the emotion engine sends that information to the communication server.

[1326] 4. Fraudulent call detection and warning

[1327] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal. This causes the communication terminal to display a message saying "Possible fraud" and terminate the call. This action allows the user to prevent damage before it occurs.

[1328] 5. Call log storage and reporting

[1329] After the call ends, the communication server saves the call log, which includes the call's audio data, analyzed text, detected fraud keywords, sentiment analysis results, and other related information. Specifically, if a call suspected of fraud is recorded, a detailed report will be sent to the user's mailbox the next day, stating, "Yesterday's call contained a possible fraudulent message. Your stress level was high." This report will encourage users to be more vigilant and enable them to take measures to prevent further damage.

[1330] 6. Fail-safe function

[1331] In addition, the communication terminal is equipped with a fail-safe function that immediately terminates a call if the user's emotional changes detected by the emotion engine increase rapidly and exceed a certain threshold. This allows for the prompt termination of suspicious calls. This embodiment of the present invention not only enables real-time detection and repelling of special fraud calls, but also enables more accurate prevention of fraud by taking the user's emotions into account. This improves the safety of individuals and society as a whole.

[1332] Hardware and Software Configuration

[1333] The hardware required to realize this system includes a communication terminal (smartphone) with recording capabilities and an Internet connection. The communication server is a high-performance computer, and an open-source speech recognition library (e.g., vosk) is used for speech recognition. An HTTP request library (e.g., requests) is used to access the sentiment analysis API.

[1334] Examples of concrete examples and prompts

[1335] As a concrete example, we will explain the operation of this system by showing the prompt sentence for the generative AI model below.

[1336] Example prompt sentence:

[1337] It recognizes the call audio and analyzes fraud keywords and the user's emotional state.

[1338] If the call says, "Hello, this is my son. I've been in an accident and need a deposit," it could be a scam.

[1339] If high stress or anxiety is detected at the same time, the call will be forcibly ended and the user will be warned.

[1340] This is expected to protect users from special fraud calls and provide a safe calling environment.

[1341] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1342] Step 1:

[1343] When the communication terminal receives a call, it detects the call information and sends it to the communication server.

[1344] Input: Call audio data

[1345] Output: Sending call audio data to the communication server

[1346] Specific operation: The communication terminal detects the start of a call and transfers the call's audio data to the communication server in real time.

[1347] Step 2:

[1348] The communication server records the call information received in real time.

[1349] Input: Transferred call audio data

[1350] Output: Recorded call audio data

[1351] Specific operation: The communication server stores the call audio in a buffer and records it as audio data in real time.

[1352] Step 3:

[1353] The communication server converts the recorded conversation into text using voice recognition technology.

[1354] Input: Recorded call audio data

[1355] Output: Transcribed call

[1356] Specific operation: The communication server uses a speech recognition library such as vosk to analyze the voice data and generate text from the voice.

[1357] Step 4:

[1358] The communication server analyzes and detects fraudulent keywords from the text.

[1359] Input: Transcribed call

[1360] Output: Fraud keyword detection results (boolean or list format)

[1361] Specific operation: Matches against a certain list of keywords (e.g., "son," "accident," "security deposit") and checks whether any of the relevant words are included.

[1362] Step 5:

[1363] The communication terminal is equipped with an emotion engine that analyzes emotions from the user's voice during a call.

[1364] Input: Call audio data

[1365] Output: Analysis results of user emotions (stress level, anxiety level, etc.)

[1366] Specific operation: The emotion engine analyzes the voice data, evaluates the user's emotional state (stress, anxiety, etc.) in real time, and sends the data to the communication server.

[1367] Step 6:

[1368] If a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety, the communication server immediately sends a warning signal to the communication terminal and terminates the call.

[1369] Input: Fraud keyword detection results, user sentiment analysis results

[1370] Output: Send warning signal, terminate call

[1371] Specific operation: The communication server evaluates the fraud risk and emotional state, and if the risk is high, it sends a warning signal to the communication terminal and forcibly terminates the call.

[1372] Step 7:

[1373] After the call is completed, the communication server stores the call log.

[1374] Input: All recorded audio data, analyzed text, fraud keyword detection results, user sentiment analysis results

[1375] Output: Saved call logs

[1376] Specific operation: Call logs including call content, text conversion results, fraud keyword detection status, and sentiment analysis results are saved in a database.

[1377] Step 8:

[1378] The analysis of the user's emotional state is communicated to the user along with the call logs stored and the reports generated.

[1379] Input: Saved call logs, emotion analysis results

[1380] Output: Report notification to user

[1381] Specific operation: The communication server generates a report document based on the saved logs and analysis results, and sends it to the user's mailbox or as an app notification.

[1382] This protects users from special fraud calls and allows them to make calls with peace of mind.

[1383] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1384] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1385] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1386] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1387] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1388] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1389] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1390] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1391] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1392] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1393] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1394] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1395] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1396] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1397] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1398] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1399] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1400] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1401] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1402] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1403] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1404] The following is further disclosed regarding the above embodiment.

[1405] (Claim 1)

[1406] a means for receiving a call and detecting call information in a communication terminal;

[1407] means for transmitting the detected call information to a communication server;

[1408] A means for the communication server to record call information in real time;

[1409] A means for converting the recorded conversation content into text using voice recognition technology by the communication server;

[1410] means for analyzing and detecting fraudulent keywords from the text;

[1411] means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected;

[1412] means for the communications server to store the call log and generate a report after the call is terminated;

[1413] means for notifying a user of the generated report;

[1414] A system including:

[1415] (Claim 2)

[1416] 2. The system of claim 1, wherein the communication terminal displays a warning and automatically terminates the call when a fraud keyword is detected.

[1417] (Claim 3)

[1418] 10. The system of claim 1, further comprising: storing call logs; generating a report on fraud risk; and transmitting the report to the user.

[1419] "Example 1"

[1420] (Claim 1)

[1421] a means for receiving a call and detecting call information in a communication terminal;

[1422] means for transmitting the detected call information to a communication server;

[1423] A means for the communication server to record call information in real time;

[1424] A means for converting the recorded conversation content into text using voice recognition technology by the communication server;

[1425] means for analyzing and detecting fraudulent keywords from the text;

[1426] means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected;

[1427] means for the communications server to store the call log and generate a report after the call is terminated;

[1428] means for notifying a user of the generated report;

[1429] a means by which users are encouraged to take further precautions based on the report;

[1430] A system including:

[1431] (Claim 2)

[1432] 2. The system of claim 1, wherein the communication terminal displays a warning and automatically terminates the call when a fraud keyword is detected.

[1433] (Claim 3)

[1434] 10. The system of claim 1, further comprising: storing call logs; generating a report on fraud risk; and transmitting the report to the user.

[1435] "Application Example 1"

[1436] (Claim 1)

[1437] a means for receiving a call and detecting call information in a communication terminal;

[1438] means for transmitting the detected call information to a communication server;

[1439] A means for the communication server to record call information in real time;

[1440] A means for converting the recorded conversation content into text using voice recognition technology by the communication server;

[1441] means for analyzing and detecting fraudulent keywords from the text;

[1442] means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected;

[1443] means for notifying a user when a fraudulent keyword is detected;

[1444] means for the communications server to store the call log and generate a report after the call is terminated;

[1445] means for transmitting the generated report to a user;

[1446] a means for automatically terminating calls containing fraud keywords;

[1447] A way to report to users via email or app notification,

[1448] A system including:

[1449] (Claim 2)

[1450] 10. The system of claim 1, wherein the communication terminal displays a warning, automatically terminates the call, and sends a real-time alert when a fraud keyword is detected.

[1451] (Claim 3)

[1452] 10. The system of claim 1, wherein a report on fraud risk is generated and sent to the user via email or app notification.

[1453] "Example 2: Combining Emotion Engines"

[1454] (Claim 1)

[1455] a means for receiving a call and detecting call information in a communication terminal;

[1456] means for transmitting the detected call information to a communication server;

[1457] A means for the communication server to record call information in real time;

[1458] A means for converting the recorded conversation content into text using voice recognition technology by the communication server;

[1459] means for analyzing and detecting fraudulent keywords from the text;

[1460] means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected;

[1461] means for the communications server to store the call log and generate a report after the call is terminated;

[1462] means for notifying a user of the generated report;

[1463] A means for an emotion engine built into the terminal to analyze the emotion of the user during a call and transmit the result to a communication server;

[1464] a means for the communication terminal to forcibly terminate the call when the emotion engine causes a rapid change in the user's emotion;

[1465] A system including:

[1466] (Claim 2)

[1467] 2. The system of claim 1, wherein the communication terminal displays a warning and automatically terminates the call if a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety.

[1468] (Claim 3)

[1469] 10. The system of claim 1, further comprising: storing call logs; generating reports on fraud risk and user sentiment analysis; and transmitting the reports to the user.

[1470] "Application example 2 when combining emotion engines"

[1471] (Claim 1)

[1472] a means for receiving a call and detecting call information in a communication terminal;

[1473] means for transmitting the detected call information to a communication server;

[1474] A means for the communication server to record call information in real time;

[1475] A means for converting the recorded conversation content into text using voice recognition technology by the communication server;

[1476] means for analyzing and detecting fraudulent keywords from the text;

[1477] means for analyzing the emotional state of a user during a call using an emotion engine;

[1478] means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected and the user's emotion analysis result indicates high stress or anxiety;

[1479] means for the communications server to store the call log and generate a report after the call is terminated;

[1480] means for notifying the user of the analysis of the user's emotional state together with the stored call log and the generated report;

[1481] A system including:

[1482] (Claim 2)

[1483] 2. The system of claim 1, wherein the communication terminal displays a warning and automatically terminates the call if a fraud keyword is detected and the user's emotion analysis results indicate high stress or anxiety.

[1484] (Claim 3)

[1485] 10. The system of claim 1, further comprising: storing call logs; generating reports regarding fraud risk and the user's emotional state; and transmitting the reports to the user. [Explanation of symbols]

[1486] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for receiving a call and detecting call information in a communication terminal; means for transmitting the detected call information to a communication server; A means for the communication server to record call information in real time; A means for converting the recorded conversation content into text using voice recognition technology by the communication server; means for analyzing and detecting fraudulent keywords from the text; means for sending a warning signal to the communication terminal and terminating the call if a fraud keyword is detected; means for the communications server to store the call log and generate a report after the call is terminated; means for notifying a user of the generated report; A system including:

2. 2. The system of claim 1, wherein the communication terminal displays a warning and automatically terminates the call when a fraud keyword is detected.

3. 10. The system of claim 1, further comprising: storing call logs; generating a report on fraud risk; and transmitting the report to the user.

Images