System

Abstract

To propose a system.SOLUTION: The system includes: means for recognizing voice of a user; means for analyzing the content of a conversation with the user on the basis of the recognized voice of the user by using a natural language processing technique; means for making a predetermined reaction when a specific keyword related to a crime is detected from the content of the conversation with the user; means for analyzing emotions of the user; and means for sending information of the result of the predetermined reaction and the emotion of the user to a contact registered in advance.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, the number of victims of fraud and suspicious phone calls is on the rise. The elderly are particularly susceptible to phone fraud. However, if the victim is unaware of the scam or if an elderly person living alone falls victim to the scam, it is difficult for their family and relatives to find out about it. [Means for solving the problem]

[0005] This invention utilizes voice recognition and natural language processing technology to analyze conversation content in real time. It responds to specific keywords and sends the results to family and relatives via pre-registered social media apps such as messenger apps. It also limits activation to calls from numbers not registered in the phone book or from anonymous numbers, and also requires the individual's consent and ensures confidentiality of communications. This makes it possible to prevent fraud and damage caused by suspicious phone calls. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] An embodiment of the present invention is realized as an application installed on a terminal such as a smartphone or tablet. This application captures speech through the terminal's microphone and converts it into text using speech recognition technology. The converted text is analyzed using natural language processing technology to determine whether it contains specific keywords. Keywords that may indicate fraud or suspicious phone calls include "settlement money," "accident," "bank," and "transfer."

[0029] "Example 2"

[0030] If a keyword is detected, the application will send a notification to family and relatives via pre-registered messaging apps or social media. The notification will include information about the suspicious call, its content (keywords), and the caller's phone number, allowing family and relatives to respond quickly.

[0031] "Example 3"

[0032] In addition, this embodiment of the present invention can be set to activate only for calls from numbers not registered in the phone book or from anonymous numbers. This prevents false activation for calls from acquaintances. Furthermore, the function is activated with the consent of the individual, taking legal considerations into account for the confidentiality of communications.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: A user installs the application of the present invention on a device such as a smartphone or tablet.

[0036] Step 2: The application captures audio through the device's microphone.

[0037] Step 3: The captured audio is converted into text using speech recognition technology.

[0038] Step 4: The converted text is analyzed using natural language processing techniques to determine whether it contains specific keywords.

[0039] "Example 2"

[0040] Step 1: If a specific keyword is detected in the text, the application will send a notification to family and relatives via pre-registered messenger apps or social media.

[0041] Step 2: The notification will include information about the suspicious call, its content (keywords), and the caller's phone number.

[0042] "Example 3"

[0043] Step 1: The application of the present invention can be set to launch only when receiving calls from numbers not registered in the phone book or from numbers with no identification.

[0044] Step 2: Enable the feature with the individual's consent, while also taking legal considerations into account the confidentiality of communications.

[0045] Example 1

[0046] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0047] In recent years, fraudulent and suspicious phone calls have been increasing, with many elderly people and those unfamiliar with technology falling victim to these attacks. There is a need for systems that can detect these fraudulent calls in advance and warn users. However, existing systems have low accuracy in speech recognition and natural language processing, making it difficult to respond in real time. Furthermore, there is a lack of effective methods for detecting fraudulent calls while protecting user privacy.

[0048] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0049] In this invention, the server includes a speech recognition unit, a conversation content analysis unit that utilizes natural language processing technology, and a unit that responds to specific keywords, which enables fraud and suspicious phone calls to be detected in real time and a warning given to the user.

[0050] "Speech recognition means" refers to technology that converts voice data into text data.

[0051] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning and context.

[0052] The "conversation content analysis means" refers to a means for analyzing text data converted by the speech recognition means using natural language processing technology.

[0053] "Means that react to specific keywords" refers to means that detect specific words or phrases contained in text data and take action based on them.

[0054] "Information sharing means" refers to the means for notifying other users or systems of analysis results and warnings.

[0055] "Means to limit activation to calls from numbers not registered in your contacts or from anonymous numbers" refers to means to activate the system only under certain conditions.

[0056] "Consent of the individual, measures to address confidentiality of communications" refers to measures to protect user privacy and meet legal requirements.

[0057] "Means for capturing audio through a microphone of a terminal and processing audio data in real time" refers to means for collecting audio using a microphone of a terminal and processing the audio data immediately.

[0058] "Means for converting voice data into text" refers to means for converting voice data into character data using voice recognition technology.

[0059] "Means for tokenizing the converted text and tagging parts of speech" refers to means for dividing text data into words and phrases and assigning part-of-speech information to each.

[0060] "Means for notifying the user when a keyword is detected" refers to means for issuing a warning or notification to the user when a specific keyword is found in text data.

[0061] This invention is realized as an application installed on devices such as smartphones and tablets. This application is a system that combines voice recognition technology and natural language processing technology to detect fraudulent and suspicious phone calls in real time and warn users.

[0062] When a user launches the application, speech is captured through the device's microphone. The captured speech is converted into text using speech recognition technology such as the Google® Cloud Speech-to-Text API. The converted text is then analyzed using Python® natural language processing libraries such as NLTK and spaCy. Specifically, the text is tokenized and each word is tagged with its part of speech.

[0063] The device then checks whether the analyzed text contains certain keywords (such as "settlement money," "accident," "bank," or "transfer"). If a keyword is detected, the device records the information and notifies the user, via a pop-up message or audio alert.

[0064] For example, if a user receives a call that begins with, "Hello, this is your bank and we'd like to discuss a transfer," the application can capture this speech, convert it to text, analyze the text, and detect if it contains keywords like "bank" or "transfer," thereby alerting the user to a potential scam.

[0065] Example prompt sentence:

[0066] "This application captures voice through the device's microphone and converts it into text using speech recognition technology. The converted text is then analyzed using natural language processing technology to determine whether it contains specific keywords. Keywords include 'settlement money,' 'accident,' 'bank,' and 'transfer.'"

[0067] This system includes a speech recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family and relatives through information sharing means such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the contacts or from blocked numbers, a means for ensuring the individual's consent and confidentiality of communications, a means for capturing voice through the device's microphone and processing the voice data in real time, a means for converting the voice data into text, a means for tokenizing the converted text and tagging it with parts of speech, and a means for notifying the user when a keyword is detected. This makes it possible to detect fraud and suspicious phone calls in real time and warn the user.

[0068] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0069] Step 1:

[0070] A user launches an application on their smartphone or tablet.

[0071] Input: The application is started by a user action.

[0072] Output: Application initialization complete.

[0073] Specific behavior: A user launches an application by tapping its icon on the device's home screen. The application initializes the necessary libraries and APIs for speech recognition and natural language processing (e.g., Google Cloud Speech-to-Text API, Python's NLTK library).

[0074] Step 2:

[0075] The device captures the user's voice in real time through a microphone.

[0076] Input: User's voice.

[0077] Output: Audio data.

[0078] Specific operation: When the user answers the phone and says, "Hello, this is from the bank. I'd like to talk to you about a transfer," the voice is picked up by the device's microphone. The voice data is temporarily stored in the device's memory.

[0079] Step 3:

[0080] The device sends the captured voice data to the Google Cloud Speech-to-Text API.

[0081] Input: Audio data.

[0082] Output: Text data.

[0083] How it works: The device sends the captured voice data to the Google Cloud Speech-to-Text API, which converts the voice data into text, which is then returned to the device as "Hello, this is from your bank. We'd like to discuss a transfer."

[0084] Step 4:

[0085] The text received by the device is analyzed using a Python natural language processing library (NLTK or spaCy).

[0086] Input: Text data.

[0087] Output: Tokenized text and part-of-speech tags.

[0088] Specific operation: The device divides the text it receives into tokens such as "hello," "this is," "from the bank," "but," "transfer," "about," and "we'd like to talk to you about," and tags each word with its part of speech.

[0089] Step 5:

[0090] The device checks whether certain keywords are present in the analyzed text.

[0091] Input: Tokenized text and part-of-speech tags.

[0092] Output: Keyword detection results.

[0093] What it does: The device checks whether the text contains keywords such as "bank" or "transfer." If a keyword is detected, it records the information.

[0094] Step 6:

[0095] The terminal notifies the user that the keyword has been detected.

[0096] Input: Keyword detection results.

[0097] Output: Notification to the user.

[0098] What it does: The device will display a pop-up message saying "Possible fraud. Please be careful." It can also issue a voice alert saying "Possible fraud."

[0099] (Application example 1)

[0100] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] In recent years, fraudulent and suspicious phone calls have been increasing, with elderly people and those less well-informed becoming victims in particular. Conventional countermeasures are insufficient, and there is a need for a system that can detect possible fraud in real time and issue a warning to users. There is also a need for a method to automatically detect potentially fraudulent calls and prompt users to take appropriate action.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0103] In this invention, the server includes a speech recognition means, a conversation content analysis means using natural language processing technology, and a means for responding to specific keywords, which makes it possible to issue a warning to the user when a keyword that may be fraudulent is detected and provide the user with the option of whether or not to end the call.

[0104] "Speech recognition means" refers to a device or software that converts the contents of a call into text information in real time using speech-to-text technology.

[0105] "Natural language processing technology" is a technology for analyzing text data and understanding meaning and emotions, and is used to analyze conversation content.

[0106] The "conversation content analysis means" is a device or software that utilizes natural language processing technology to analyze the text data converted by the speech recognition means and detect specific keywords and phrases.

[0107] A "means for responding to specific keywords" is a device or software that detects pre-set keywords (e.g., "settlement money," "accident," "bank," "transfer," etc.) from text data and takes action based on the results.

[0108] A "messenger app" is application software that sends and receives messages over the Internet and is used to send notifications to family and relatives.

[0109] "SNS" stands for Social Networking Service, an online platform for users to share information and communicate.

[0110] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to a device or software that activates the system only when an incoming call is received from a number not registered in the user's phone book or from an anonymous number.

[0111] "Means for obtaining consent and ensuring confidentiality of communications" refers to devices or software that obtain a user's consent when using the system in order to protect the user's privacy and ensure the confidentiality of communications.

[0112] "Means for alerting the user when a potentially fraudulent keyword is detected and offering the option of ending the call" refers to a device or software that displays a warning to the user when a specific keyword is detected and offers the user the option of continuing or ending the call.

[0113] As an embodiment of the present invention, a method for installing a fraudulent call detection system on a smartphone is described. The system includes a voice recognition unit, a conversation content analysis unit using natural language processing technology, a reaction unit for specific keywords, and a unit for issuing a warning to the user and providing the user with the option of ending the call.

[0114] System Program

[0115] The system program operates as follows.

[0116] 1. Voice recognition means:

[0117] The system captures the conversation in real time through the smartphone's microphone and converts it into text using speech recognition technology. Specifically, it uses a speech recognition library and Google's speech recognition API.

[0118] 2. Natural language processing technology:

[0119] The transformed text is analyzed using natural language processing techniques, specifically using the transformers library pipeline to perform sentiment analysis on the text.

[0120] 3. Ways to respond to specific keywords:

[0121] The text data is checked for pre-set keywords (e.g., "settlement money," "accident," "bank," "transfer," etc.) to determine whether the relevant keywords are included.

[0122] 4. By providing the user with a warning and the option to end the call:

[0123] If a keyword is detected, the user will be alerted and given the option to continue or end the call.

[0124] Hardware and Software

[0125] Hardware:

[0126] Smartphone (microphone, speaker, display)

[0127] software:

[0128] Speech Recognition Library

[0129] Google Speech Recognition API

[0130] transformers library

[0131] Specific examples

[0132] For example, if a user receives a call and the caller says, "Please transfer the settlement money," the system will detect the keywords "settlement money" and "transfer." In this case, the system will display a warning to the user saying, "This may be a scam. Do you want to end the call?" and ask the user to choose whether to continue or end the call.

[0133] Prompt Sentence Examples

[0134] If a user receives a phone call and the caller says, "Please transfer the settlement money," the application will detect the keywords "settlement money" and "transfer" and alert the user.

[0135] In this way, the fraudulent call detection system can ensure user safety and prevent fraudulent incidents.

[0136] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0137] Step 1:

[0138] The device captures the call content in real time through the smartphone's microphone. The input is voice data, and the output is raw voice data that is passed to the voice recognition means. Specifically, when the user starts a call, the microphone starts capturing the voice.

[0139] Step 2:

[0140] The device converts the captured voice data into text using a voice recognition means. The input is the voice data acquired in step 1, and the output is text data. Specifically, it uses a voice recognition library and calls Google's voice recognition API to convert the voice into text.

[0141] Step 3:

[0142] The device uses natural language processing technology to analyze the converted text data. The input is the text data generated in step 2, and the output is the analysis results on the sentiment and meaning of the text. Specifically, it uses a pipeline from the transformers library to perform sentiment analysis of the text.

[0143] Step 4:

[0144] The terminal checks the text data for pre-defined keywords using a means that responds to specific keywords. The input is the text data obtained in step 3, and the output is a flag indicating whether the keyword was detected. Specifically, it checks whether the text data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[0145] Step 5:

[0146] If a keyword is detected, the terminal will issue a warning to the user and offer the option of whether to end the call. The input is the keyword detection flag obtained in step 4, and the output is the user's choice. Specifically, the terminal will display a warning message on the display and offer the user the option of "Possible fraud. Do you want to end the call?"

[0147] Step 6:

[0148] The user confirms the warning message and chooses whether to continue or end the call. The input is the warning message displayed in step 5, and the output is the user's choice. As a specific operation, if the user selects "End," the call is automatically ended.

[0149] In this way, the fraudulent call detection system can analyze the content of calls in real time, warn users if there is a possibility of fraud, and prompt them to take appropriate action.

[0150] Example 2

[0151] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0152] In modern society, there has been an increase in fraudulent and suspicious phone calls, with elderly people and those unfamiliar with technology being particularly vulnerable to these attacks. To combat this, there is a need for systems that can quickly notify family and relatives and encourage them to take appropriate action. However, existing systems lack the ability to recognize voices, detect keywords, and send notifications in real time, making it difficult to implement effective countermeasures.

[0153] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0154] In this invention, the server includes a voice recognition unit, a conversation content analysis unit using natural language processing technology, a unit that responds to specific keywords, a unit that sends the response results to family and relatives through an information sharing service such as a pre-registered communication application, a unit that limits activation to calls from numbers not registered in the phone book or from anonymous numbers, a unit that handles the individual's consent and confidentiality of communications, a unit that converts voice data into text data, a unit that compares the text data with a keyword list, a unit that creates a notification when a keyword is detected, and a unit that sends the notification in real time. This enables a user to quickly and reliably notify family and relatives when they receive a suspicious phone call and urge them to take appropriate action.

[0155] "Speech recognition means" is a technology for converting voice data into text data.

[0156] "Means for analyzing conversation content using natural language processing technology" is a technology for analyzing text data and understanding meaning and intent.

[0157] "Means for responding to specific keywords" refers to technology that compares text data with a pre-registered keyword list and detects matching keywords.

[0158] "Information sharing services such as communication applications" refers to services for sharing information over the Internet, such as messenger apps and social networking sites.

[0159] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology for controlling the operation of the system based on specific conditions.

[0160] "Measures to address the need for consent and confidentiality of communications" refers to techniques to protect user privacy and meet legal requirements.

[0161] The "means for converting voice data into text data" is a technology for converting voice data into text information using voice recognition technology.

[0162] The "means for checking text data against a keyword list" is a technique for comparing the converted text data with a pre-set keyword list to detect matching keywords.

[0163] The "means for generating a notification when a keyword is detected" refers to a technique for generating a notification message when a particular keyword is detected.

[0164] The "means for sending notifications in real time" refers to a technique for instantly sending generated notification messages to family and relatives.

[0165] This invention is a system for quickly and reliably notifying family and relatives when a user receives a suspicious phone call, and encouraging them to take an appropriate action. This system includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through an information sharing service such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the phone book or from numbers withheld, a means for ensuring the consent of the user and confidentiality of communications, a means for converting voice data into text data, a means for comparing the text data with a keyword list, a means for creating a notification when a keyword is detected, and a means for transmitting the notification in real time.

[0166] The server monitors users' phone calls in real time and converts them into text using the Google Cloud Speech-to-Text API, a voice recognition software. The converted text is then compared with a pre-registered list of keywords.

[0167] The device is the user's smartphone or PC, and is responsible for sending the contents of the call to the server. When the call ends, the device sends the text data of the call to the server.

[0168] The server analyzes the received text data to see if it contains any keywords, and if it does, it uses a communication service API to send notifications to family and relatives via pre-registered messenger apps or social media.

[0169] The notification will include information about the suspicious call, its contents (keywords), the caller's phone number, etc. This will enable family members and relatives to respond quickly.

[0170] As a concrete example, consider the case where a user receives a phone call containing the keyword "scam." In this case, the server detects the keyword "scam" and uses the communication service API to send the following notification to the family:

[0171] Example of notification content:

[0172] There was a suspicious phone call.

[0173] Content: Fraud

[0174] Caller's phone number: +81-90-XXXX-XXXX

[0175] Example prompts to input to the generative AI model:

[0176] Create a program that will send a notification to a family member if the user receives a phone call containing the keyword "scam." The notification should include information about the suspicious call, its content (keywords), and the caller's phone number.

[0177] In this way, the system can detect suspicious calls in real time and quickly notify family and relatives to keep users safe.

[0178] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0179] Step 1:

[0180] The user receives a call.

[0181] When a user receives a call on their smartphone, the call starts. The call start event triggers the system to start working. The input is the incoming call, and the output is the call start event.

[0182] Step 2:

[0183] The device records the call.

[0184] The device records the contents of the call in real time. The recording is done using the smartphone's built-in microphone. The recorded data is temporarily stored in the device. The input is the call audio, and the output is the recorded audio data.

[0185] Step 3:

[0186] The device sends the recorded audio data to the server.

[0187] When the call ends, the device sends the recorded voice data to the server. The transmission is performed using the HTTPS protocol, and the data is encrypted. The input is the recorded voice data, and the output is the transmission of the voice data to the server.

[0188] Step 4:

[0189] The server converts the audio data into text.

[0190] The server converts the received voice data into text using the Google Cloud Speech-to-Text API. The converted text data is stored on the server. The input is voice data, and the output is text data.

[0191] Step 5:

[0192] The server matches the text data with a list of keywords.

[0193] The server compares the converted text data with a pre-registered keyword list, which includes words such as "fraud," "suspicious," and "danger." The input is the text data and the keyword list, and the output is the keyword detection results.

[0194] Step 6:

[0195] If the server detects a keyword, it creates a notification.

[0196] If the server detects a keyword, it creates a notification. The notification contains information about a suspicious call, its content (keyword), and the caller's phone number. The input is the keyword detection result, and the output is the notification message.

[0197] Step 7:

[0198] The server sends notifications to family and relatives.

[0199] The server uses the communication service API to send notifications to family and relatives via pre-registered messenger apps and SNS. The notifications are sent in real time, allowing family and relatives to respond quickly. The input is the notification message, and the output is the notification sent to family and relatives.

[0200] (Application example 2)

[0201] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0202] In recent years, there has been an increase in telephone fraud and suspicious phone calls, with elderly people and those less well-informed becoming victims in particular. In order to respond quickly to such suspicious calls, a system that can immediately send notifications to family and relatives is needed. However, current systems lack the functionality to analyze call content in real time, detect specific keywords, and send notifications. Furthermore, there is a need to provide more detailed information by including recordings of calls in the notification content.

[0203] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family members and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the person's consent and confidentiality of communications, a means for sending a notification to family members and relatives based on the detected keywords, and a means for including in the notification the content of the suspicious call (keywords), the caller's phone number, and recorded data of the call. This makes it possible to quickly provide detailed information to family members and relatives when a suspicious call is received.

[0204] "Speech recognition means" refers to technology that converts speech into text data.

[0205] "Natural language processing technology" refers to the technology of processing and analyzing human language using a computer.

[0206] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and extracts specific information.

[0207] "Means for responding to specific keywords" refers to technology that detects pre-set keywords and operates based on the detection results.

[0208] A "messenger app" refers to an application for sending and receiving messages over the Internet.

[0209] "SNS" is an abbreviation for social networking service, and refers to an online platform where users can share information with each other.

[0210] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0211] "Measures to ensure the individual's consent and confidentiality of communications" refers to technologies for obtaining the individual's consent when using the system and for protecting the confidentiality of communication content.

[0212] "Means for sending notifications to family and relatives based on detected keywords" refers to technology that sends notifications to family and relatives when specific keywords are detected.

[0213] "Means for the notification to include the content of the suspicious call (keywords), the caller's phone number, and a recording of the call" refers to a technology in which the notification sent includes the content of the suspicious call, the caller's phone number, and a recording of the call.

[0214] A system for implementing this invention includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for dealing with the consent of the individual and confidentiality of communications, a means for sending notifications to family and relatives based on detected keywords, and a means for the notification to include the content of the suspicious call (keywords), the caller's phone number, and recorded data of the call.

[0215] System Program

[0216] This system can be implemented using Python. It uses the Google Cloud Speech-to-Text API for speech recognition, the NLTK (Natural Language Toolkit) for natural language processing, and the smtplib library for sending emails.

[0217] Processing Description

[0218] The server first converts the contents of the call into text data using voice recognition. This text data is then analyzed using natural language processing technology to check whether it contains specific keywords. If a specific keyword is detected, the server will use the results to send a notification to family and relatives via pre-registered messenger apps or social media.

[0219] The notification will include the content of the suspicious call (detected keywords), the phone number from which it originated, and a recording of the call, allowing family members or relatives to take action quickly.

[0220] Hardware and software used

[0221] Hardware: Smartphones, servers

[0222] Software: Python, Google Cloud Speech-to-Text API, NLTK, smtplib

[0223] Specific examples

[0224] For example, if a user receives a call saying, "This is a fraudulent call," a speech recognition means converts this speech into text data. Next, natural language processing technology is used to analyze this text data and detect the keyword "fraud." Based on the detected keyword, the server sends a notification to family and relatives. This notification includes the following content: "There has been a suspicious call. The caller's phone number is: 090-XXXX-XXXX. The detected keyword is: fraud."

[0225] Prompt Sentence Examples

[0226] Create a program that checks the content of phone calls and sends a notification if certain keywords are included. The keyword list includes "fraud," "suspicious," "suspicious," and "transfer." The notification will include the content of the suspicious call (keywords), the caller's phone number, and a recording of the call. The notification will be sent by email.

[0227] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0228] Step 1:

[0229] The user receives a call. The voice data of the call is input to the terminal. The terminal collects this voice data in real time.

[0230] Step 2:

[0231] The device uses a speech recognition means to convert the collected voice data into text data. Specifically, it uses the Google Cloud Speech-to-Text API to analyze the voice data and generate corresponding text data. The input is voice data, and the output is text data.

[0232] Step 3:

[0233] The server analyzes the text data using natural language processing technology. Specifically, it uses NLTK (Natural Language Toolkit) to tokenize the text data and check whether it contains specific keywords. The input is the text data, and the output is a list of detected keywords.

[0234] Step 4:

[0235] If a specific keyword is detected, the server creates a notification based on the results. The notification includes the content of the suspicious call (detected keywords), the caller's phone number, and the call recording. The input is a list of detected keywords, the caller's phone number, and the call recording, and the output is a notification message.

[0236] Step 5:

[0237] The server sends the created notification message to family and relatives via pre-registered messenger apps or SNS. Specifically, it sends emails using the smtplib library. The input is the notification message, and the output is the sent notification.

[0238] Step 6:

[0239] Family members and relatives can check the received notification and respond promptly if necessary. The input is the notification message and the output is the response action.

[0240] The above processing steps enable the user to quickly provide detailed information to family and relatives when they receive a suspicious phone call.

[0241] Example 3

[0242] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0243] Conventional telephone answering systems were unable to respond appropriately to calls from numbers not registered in the phone book or from anonymous numbers, and they malfunctioned when receiving calls from acquaintances and lacked legal consideration for the confidentiality of communications. Furthermore, users had insufficient means to manage the system settings, making them inconvenient.

[0244] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through a pre-registered communication application such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, a means for generating a response message using a generative AI model, a means for sending the generated response message to the caller, and a means for the user to manage system settings. This makes it possible to respond appropriately to calls from numbers not registered in the phone book or from anonymous numbers, prevent malfunctions when receiving calls from acquaintances, and ensure legal consideration for confidentiality of communications.

[0245] "Speech recognition means" refers to technology that converts speech into text data.

[0246] "Natural language processing technology" refers to technology that processes, understands, and generates human language using a computer.

[0247] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and understands the content of the conversation.

[0248] "Means for responding to specific keywords" refers to a technology for performing specific actions based on pre-set keywords.

[0249] "Communication application" refers to software that sends and receives messages over the Internet, such as a messenger app.

[0250] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0251] "Means for addressing the individual's consent and confidentiality of communications" refers to technologies that enable systems with the user's consent and take legal considerations to protect the confidentiality of communications.

[0252] A "generative AI model" refers to a model that uses artificial intelligence to generate text data.

[0253] "Means for generating a response message" refers to technology that uses a generative AI model to create a response message for the caller.

[0254] "Means for sending a response message to the sender" refers to a technique for sending the generated response message to the sender.

[0255] "Means for users to manage system settings" refers to the interface through which users can change and manage the system's operating settings.

[0256] MODE FOR CARRYING OUT THE INVENTION

[0257] This invention is a system that activates only when receiving calls from numbers not registered in the phone book or from anonymous numbers, and uses a generative AI model to generate a response message and send it to the caller. A specific embodiment of this system is described below.

[0258] Hardware and software used

[0259] The server uses a communication API to monitor incoming calls. Specifically, it uses a communication service API to detect incoming calls in real time. The phone book database is managed using MySQL (registered trademark) and is used to verify caller IDs. The generative AI model uses GPT-4 (registered trademark) from OpenAI (registered trademark). The interface that allows users to manage system settings is built using React.

[0260] Program processing

[0261] The server monitors incoming calls through the communication service API. When a call is received, the server checks the phone number against its phone book database. If the number is not registered in the phone book or is blocked, the server activates a generative AI model, sends a prompt, and generates a response message. The generated response message is sent to the caller through the communication service API.

[0262] Specific examples

[0263] If the user receives a call from a number not in their phone book, the server checks the number against the phone book database to ensure it is not registered. The server then sends the following prompt to the generative AI model:

[0264] Example prompt sentence:

[0265] "This number is not in the phone book. Please generate a response message."

[0266] The generative AI model generates a response message like this:

[0267] "Thank you for calling. This number is not in the phone book and cannot be answered. We apologize."

[0268] The server sends this message to the caller via the communication service API, allowing users to prevent accidental calls from acquaintances and to respect legal considerations regarding the confidentiality of communications.

[0269] Managing User Settings

[0270] Users can enable or disable this feature through a system settings screen, which is built using React and includes a checkbox for user consent. When a user changes the settings, the information is sent to the server and stored in a database.

[0271] In this way, the present invention can respond appropriately to calls from numbers not registered in the telephone directory or from anonymous numbers, prevent malfunctions in response to calls from acquaintances, and take legal considerations regarding the secrecy of communications. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0272] Program processing flow

[0273] Step 1: Detecting an incoming call

[0274] The server uses the Communication Services API to monitor incoming calls in real time, and when an incoming call comes in, the Communication Services API sends a notification to the server.

[0275] Input: Call notification (caller ID, call time, etc.)

[0276] Output: Incoming call details

[0277] Specific behavior:

[0278] The server receives a notification from the communication service API and retrieves the details of the incoming call (caller ID, time of call, etc.).

[0279] Step 2: Phone number verification

[0280] The server checks the incoming caller number against a phone book database, which is managed using MySQL.

[0281] Input: Caller ID

[0282] Output: Information on whether the contact is registered in the phone book

[0283] Specific behavior:

[0284] The server queries the MySQL database to see if the caller ID is registered.

[0285] Step 3: Launching the generative AI model

[0286] If the caller ID is not registered in the phone book, the server launches a generative AI model, which uses OpenAI's GPT-4.

[0287] Input: Information that the caller number is not registered in the phone book

[0288] Output: Launch of a generative AI model

[0289] Specific behavior:

[0290] The server sends a prompt to the generative AI model, requesting it to generate a response message.

[0291] Step 4: Generate a response message

[0292] The generative AI model generates a response message based on the prompt sentence.

[0293] Input: prompt statement

[0294] Output: Response message

[0295] Specific behavior:

[0296] The generative AI model receives the following prompt and generates a response message:

[0297] "This number is not in the phone book. Please generate a response message."

[0298] Example of the message generated:

[0299] "Thank you for calling. This number is not in the phone book and cannot be answered. We apologize."

[0300] Step 5: Send a response message

[0301] The server sends the generated response message to the caller via the communication service API.

[0302] Input: Response message

[0303] Output: Send message to caller

[0304] Specific behavior:

[0305] The server uses the communication services API to send the generated message to the caller as an SMS or voice message.

[0306] Step 6: Manage User Settings

[0307] Users can enable or disable this feature through a system settings screen, which is built using React.

[0308] Input: User setting change

[0309] Output: Update configuration information

[0310] Specific behavior:

[0311] The user can enable or disable the feature by checking the checkbox on the settings screen. The server saves this setting in the database and applies it the next time a call is received.

[0312] (Application example 3)

[0313] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] In recent years, telephone fraud and nuisance calls have been increasing, and there is a particular need to respond to calls from numbers not registered in the phone book or from anonymous numbers. However, conventional systems have malfunctioned when receiving calls from acquaintances and have insufficient legal consideration for the confidentiality of communications. In addition, there have been problems with users being unable to quickly recognize suspicious calls and take appropriate action.

[0315] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means that utilizes natural language processing technology, and a means that responds to specific keywords. This makes it possible to detect an incoming call from a number not registered in the phone book as a suspicious call and notify the user.

[0316] "Speech recognition means" is a technology that converts voice into a digital signal and analyzes its content.

[0317] "Means for analyzing conversation content using natural language processing technology" refers to technology that uses natural language processing technology to analyze the content of speech and text and understand their meaning.

[0318] "Means for responding to specific keywords" refers to a technology that responds to a preset keyword and performs a specific action.

[0319] "Means of communication" refers to the technology used to send and receive information through messenger apps, etc.

[0320] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology that activates the system only when there is an incoming call from a number not registered in the phone book or from anonymous numbers.

[0321] "Measures to address the need for consent and confidentiality of communications" refers to technologies that enable functions with the user's consent and protect the confidentiality of communications.

[0322] "Means for detecting suspicious calls and notifying the user" refers to technology that detects calls from numbers not registered in the phone book as suspicious calls and notifies the user of this information.

[0323] The system for carrying out this invention is mainly composed of a smartphone and includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a communication means, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for ensuring the consent of the person and confidentiality of communications, and a means for detecting suspicious calls and notifying the user.

[0324] Program processing explanation

[0325] Voice recognition means

[0326] When a call comes in, the application installed on the smartphone performs voice recognition. The voice recognition method converts the voice into a digital signal and analyzes its content. For this, it uses voice recognition technology such as Google Speech-to-Text API.

[0327] Conversation content analysis using natural language processing technology

[0328] The speech data acquired by the speech recognition means is converted into text using natural language processing technology. This text data is then further analyzed to understand the content of the conversation. For example, natural language processing libraries such as NLTK and spaCy are used here.

[0329] How to respond to specific keywords

[0330] The analyzed text data reacts to specific keywords. For example, if keywords such as "settlement money," "accident," "bank," and "transfer" are included, the system will react to them.

[0331] communication means

[0332] If a specific keyword is triggered, the system will send a notification to family and relatives via a messaging app or other communication method, using messaging service APIs and communication service APIs.

[0333] A method to limit activation to calls from numbers not registered in the phone book or from anonymous numbers

[0334] The system will only wake up if there is a call from a number not in the phone book or from a blocked number. This function is implemented using the ANDROID (registered trademark) TelephonyManager class.

[0335] Consent of the individual, measures to ensure confidentiality of communications

[0336] The system will obtain user consent to enable the feature and take steps to protect the confidentiality of communications, including a process for obtaining consent through the user interface.

[0337] Detecting suspicious calls and notifying users

[0338] If a call comes from a number that is not registered in the phone book, the system will detect the number as a suspicious call and notify the user. This notification is done using the smartphone's notification function.

[0339] Specific examples

[0340] For example, if a user receives a call from an unknown number, the app will detect the number as suspicious and notify the user, allowing them to take action quickly.

[0341] Prompt Sentence Examples

[0342] "Please create an app that detects a call from a number not registered in the phone book as a suspicious call and notifies the user."

[0343] In this way, specific embodiments for carrying out the invention have been described.

[0344] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0345] Step 1:

[0346] The device detects an incoming call. The input is the incoming number, and the output is a signal indicating that an incoming call has been received. Specifically, the Android TelephonyManager class is used to monitor the phone state.

[0347] Step 2:

[0348] The terminal checks the incoming number against the phone book. The input is the incoming number and the phone book data, and the output is the result of determining whether the incoming number is registered in the phone book. Specifically, it searches the phone book database to see if the incoming number exists.

[0349] Step 3:

[0350] The device checks whether the call is from a number not registered in the phone book or from an anonymous number. The input is the result of the judgment in step 2, and the output is the result of the judgment on whether to start the system. Specifically, if the judgment result is "unregistered," start the system.

[0351] Step 4:

[0352] The device begins speech recognition. The input is the incoming voice and the output is the audio data. Specifically, it uses the Google Speech-to-Text API to convert the voice into a digital signal.

[0353] Step 5:

[0354] The device analyzes the voice data using natural language processing technology. The input is voice data and the output is text data. Specifically, it uses natural language processing libraries such as NLTK and spaCy to convert the voice data into text and analyze the content.

[0355] Step 6:

[0356] The device checks the text data against specific keywords. The input is the text data, and the output is the keyword detection results. Specifically, it checks whether the data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[0357] Step 7:

[0358] If a keyword is detected on the device, a notification is sent to family and relatives using a communication method. The input is the keyword detection result and the notification content, and the output is the notification sending result. Specifically, the notification is sent using a messaging service API or a communication service API.

[0359] Step 8:

[0360] The device detects the call as suspicious and notifies the user. The input is the incoming number and keyword detection results, and the output is a notification to the user. Specifically, the smartphone's notification function is used to notify the user of the suspicious call information.

[0361] In this way, the specific operations, inputs, and outputs in each processing step have been explained.

[0362] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0363] "Example 1"

[0364] One embodiment of the present invention is a system incorporating an emotion engine. This system captures a user's voice and converts it into text using speech recognition technology. The converted text is analyzed using natural language processing technology to determine whether it contains specific keywords. The emotion engine then analyzes the user's voice for emotion and feeds the results back to the conversation content analysis means.

[0365] "Example 2"

[0366] For example, if a user utters keywords such as "settlement money," "accident," "bank," or "transfer" over the phone, the system will detect this and send a notification to family and relatives via pre-registered messenger apps or social media. Furthermore, if the emotion engine detects emotions such as anger or fear from the user's tone of voice or speaking style, these will also be sent as part of the notification.

[0367] "Example 3"

[0368] Additionally, if the emotion engine determines that a user's emotions exceed a certain threshold, for example, that the user is extremely angry or extremely scared, the system will automatically send a response to family and relatives via pre-registered social media apps such as messenger apps, allowing users to quickly notify those around them when they are in danger.

[0369] The processing flow of each embodiment will be described below.

[0370] "Example 1"

[0371] Step 1: Capture the user's voice.

[0372] Step 2: Convert the speech to text using speech recognition technology.

[0373] Step 3: The converted text is analyzed using natural language processing techniques to determine whether it contains specific keywords.

[0374] Step 4: The emotion engine analyzes the emotions from the user's voice and feeds the results back to the conversation content analysis means.

[0375] "Example 2"

[0376] Step 1: If the user utters keywords such as "settlement money," "accident," "bank," or "transfer" over the phone, the system will detect this.

[0377] Step 2: The emotion engine detects emotions from the user's tone of voice and speaking style.

[0378] Step 3: Combine the detected keywords and emotions and send notifications to family and relatives via pre-registered messenger apps or social media.

[0379] "Example 3"

[0380] Step 1: The emotion engine detects when a user's emotion exceeds a certain threshold.

[0381] Step 2: If the emotion engine determines that the person is very angry or very scared, the system will automatically send the reaction result to family and relatives via pre-registered social media apps such as messenger apps.

[0382] Example 1

[0383] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] In recent years, fraudulent and suspicious phone calls have been increasing, with elderly people and those less well-informed becoming victims in particular. In order to prevent such fraudulent calls, a system is needed that can analyze the content of phone calls in real time and detect dangerous keywords and changes in emotion. However, current systems are limited to voice recognition and keyword detection, and lack comprehensive measures that include emotion analysis. This makes it difficult to accurately detect possible fraud and respond quickly.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0386] In this invention, the server includes a voice recognition unit, a conversation content analysis unit that utilizes natural language processing technology, and a unit that responds to specific keywords, which enables the server to analyze the contents of fraudulent or suspicious phone calls in real time and detect dangerous keywords and emotional changes with high accuracy.

[0387] "Speech recognition means" refers to technology that converts voice data into text data.

[0388] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning and context.

[0389] "Conversation content analysis means" refers to a means for analyzing the content of a conversation using natural language processing technology and extracting specific information.

[0390] "Means for responding to specific keywords" refers to means for detecting whether a preset keyword is included in text data.

[0391] "Communications application" refers to software that allows you to send and receive messages and information over the Internet.

[0392] "Information and communication means" refers to technologies and devices for sending and receiving data.

[0393] "Emotion analysis means" refers to a means for analyzing emotions from voice data or text data and outputting the results.

[0394] "Means for providing feedback to the conversation content analysis means" refers to a means for returning the results of the emotion analysis to the conversation content analysis means and reflecting them in the analysis results.

[0395] MODE FOR CARRYING OUT THE INVENTION

[0396] The present invention is realized as an application installed on devices such as smartphones and tablets. This application is a system that detects fraudulent and suspicious phone calls and issues a warning to the user by combining speech recognition technology, natural language processing technology, and emotion analysis technology.

[0397] Hardware and software used

[0398] Hardware: Smartphones, tablets

[0399] Software: Speech recognition technology (e.g., speech recognition API), natural language processing technology (e.g., natural language processing library), sentiment analysis technology (e.g., sentiment analysis engine)

[0400] System Operation

[0401] 1. Audio capture:

[0402] The user speaks into the microphone on their smartphone or tablet.

[0403] The device captures audio through the microphone.

[0404] 2. Speech Recognition:

[0405] The device sends the captured voice data to the voice recognition API.

[0406] The server receives the voice data and performs voice recognition processing.

[0407] The server converts the voice data into text data and returns the converted text to the terminal.

[0408] 3. Text analysis:

[0409] The text data received by the device is passed to a natural language processing library.

[0410] The device analyzes the text data and determines whether it contains specific keywords.

[0411] 4. Emotion analysis:

[0412] The device sends text and voice data to the emotion analysis engine.

[0413] The server analyzes emotions from the voice data and returns the results to the device.

[0414] The terminal feeds back the results of the emotion analysis to the conversation content analysis means.

[0415] 5. Feedback of results:

[0416] The device integrates the results of text analysis and sentiment analysis and notifies the user.

[0417] The terminal displays a warning message to the user.

[0418] Specific examples

[0419] For example, if the user says, "I received confirmation of the transfer from my bank," the process will proceed as follows:

[0420] 1. Audio capture:

[0421] The user speaks into the microphone on their smartphone, "I have received confirmation of the transfer from the bank."

[0422] The device captures the audio.

[0423] 2. Speech Recognition:

[0424] The device sends voice data to the voice recognition API.

[0425] The server converts the voice data into text data and returns it to the terminal.

[0426] The device receives a text message saying, "Bank has confirmed transfer."

[0427] 3. Text analysis:

[0428] The device passes the text data to a natural language processing library.

[0429] The terminal extracts the keywords "bank" and "transfer."

[0430] 4. Emotion analysis:

[0431] The device sends text and voice data to the emotion analysis engine.

[0432] The server analyzes the emotion of "anxiety" and returns it to the device.

[0433] The terminal feeds back the results of the emotion analysis to the conversation content analysis means.

[0434] 5. Feedback of results:

[0435] The device combines the results of text analysis and sentiment analysis and displays a warning message to the user saying, "This may be a scam. Be careful."

[0436] Prompt Sentence Examples

[0437] Examples of prompts to be input to a generative AI model include:

[0438] If a user says, "I received confirmation of a transfer from my bank," explain how the system converts this into text using speech recognition technology, analyzes keywords using natural language processing technology, and analyzes emotions using an emotion engine.

[0439] By inputting this prompt sentence into a generative AI model, a sentence can be generated that explains the system's processing in detail.

[0440] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0441] Step 1:

[0442] Audio Capture

[0443] The user speaks into the microphone on their smartphone or tablet.

[0444] The device captures audio through the microphone.

[0445] Input: User's voice data

[0446] Output: Captured audio data

[0447] What happens: The user says, "I received confirmation of transfer from my bank."

[0448] Step 2:

[0449] Voice Recognition

[0450] The device sends the captured voice data to the voice recognition API.

[0451] The server receives the voice data and performs voice recognition processing.

[0452] The server converts the voice data into text data and returns the converted text to the terminal.

[0453] Input: Captured audio data

[0454] Output: Converted text data

[0455] Specific operation: The audio "The bank has confirmed the transfer" is converted to the text "The bank has confirmed the transfer".

[0456] Step 3:

[0457] Text analytics

[0458] The text data received by the device is passed to a natural language processing library.

[0459] The device analyzes the text data and determines whether it contains specific keywords.

[0460] Input: Converted text data

[0461] Output: Keyword extraction results

[0462] Specific operation: The keywords "bank" and "transfer" are extracted from the text "The bank has confirmed the transfer."

[0463] Step 4:

[0464] Emotion analysis

[0465] The device sends text and voice data to the emotion analysis engine.

[0466] The server analyzes emotions from the voice data and returns the results to the device.

[0467] The terminal feeds back the results of the emotion analysis to the conversation content analysis means.

[0468] Input: Text and audio data

[0469] Output: Sentiment analysis results

[0470] Specific behavior: The emotion "anxiety" is detected from the voice.

[0471] Step 5:

[0472] Feedback of results

[0473] The device integrates the results of text analysis and sentiment analysis and notifies the user.

[0474] The terminal displays a warning message to the user.

[0475] Input: Keyword extraction results and sentiment analysis results

[0476] Output: A warning message to the user.

[0477] Specific behavior: A warning message will appear saying, "This may be a scam. Be careful."

[0478] (Application example 1)

[0479] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0480] In recent years, fraudulent and suspicious phone calls have been increasing, with the number of victims targeting the elderly becoming particularly serious. Conventional countermeasures only respond to specific keywords, making it difficult to completely eliminate the possibility of fraud. Furthermore, because they do not take into account the user's emotions, they may miss signs of fraud. This poses a challenge as users remain at high risk of becoming victims of fraud.

[0481] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0482] In this invention, the server includes a speech recognition unit, a conversation content analysis unit that utilizes natural language processing technology, a unit that responds to specific keywords, a sentiment analysis unit, and a unit that issues a warning based on the sentiment analysis results. This allows for real-time analysis of the content of calls, and by combining the detection of specific keywords with the sentiment analysis, it is possible to detect the possibility of fraud with high accuracy and issue a warning to the user.

[0483] "Speech recognition means" refers to a device or software that uses speech-to-text technology to analyze the content of a user's speech.

[0484] "Natural language processing technology" is a technology for analyzing text data and understanding its meaning and context.

[0485] The "conversation content analysis means" is a device or software that uses natural language processing technology to analyze the text converted by the speech recognition means and extract specific information.

[0486] A "means for responding to specific keywords" is a device or software that detects whether a predetermined keyword is contained within a text.

[0487] An "emotion analysis means" is a device or software that analyzes a user's emotions from voice or text and outputs the results.

[0488] The "means for issuing a warning based on the result of emotion analysis" refers to a device or software that issues a warning to the user based on the result of the emotion analysis means.

[0489] A "messenger app" is an application that sends and receives messages over the Internet.

[0490] "SNS" is an abbreviation for social networking service, an online platform for users to share information with each other.

[0491] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to a device or software that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0492] "Means for protecting the privacy of communications with the user's consent" refers to devices or software that protect the privacy of communications with the user's consent.

[0493] A system for carrying out this invention is configured as an application installed on a terminal such as a smartphone, tablet, etc. This application includes a speech recognition means, natural language processing technology, a conversation content analysis means, a means for responding to specific keywords, a sentiment analysis means, and a means for issuing a warning based on the sentiment analysis results.

[0494] Hardware and software used

[0495] Hardware: Smartphone, tablet (with microphone)

[0496] Software: Speech recognition library, transformers library pipeline

[0497] Data processing and calculation

[0498] 1. Speech recognition: Speech is captured through the device's microphone and converted into text using a speech recognition library, thereby obtaining the user's speech as text data.

[0499] 2. Keyword detection: The converted text is checked to see if it contains specific keywords (e.g., "settlement money," "accident," "bank," "transfer"), which helps detect calls that may be fraudulent.

[0500] 3. Sentiment Analysis: Using the transformers library pipeline, the text is analyzed for sentiment. If negative sentiment is detected, it is determined to be highly likely to be fraudulent.

[0501] 4. Warning: Based on the results of sentiment analysis, if there is a high possibility of fraud, a warning is issued to the user. The warning is displayed using the device's notification function.

[0502] Specific examples

[0503] For example, if a user receives a call containing keywords such as "settlement money" or "transfer money," the app will automatically detect this and perform sentiment analysis. If it determines that the call is likely to be fraudulent, it will display a warning to the user.

[0504] Prompt Sentence Examples

[0505] When a user receives a call, the call content is analyzed in real time to determine whether it contains specific keywords (e.g., "settlement money," "accident," "bank," "transfer"). In addition, an emotion engine is used to analyze the emotions of the other party during the call, and if there is a high possibility of fraud, a warning is issued to the user.

[0506] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0507] Step 1:

[0508] The device initiates a call. During the call, audio data is captured in real time through the device's microphone. The input is audio data, and the output is a stream of audio data. Specifically, the device's microphone captures audio and stores the audio data in a buffer.

[0509] Step 2:

[0510] The device uses a speech recognition means to convert the captured voice data into text. The input is a stream of voice data, and the output is text data. Specifically, the device uses a speech recognition library to analyze the voice data and generate the corresponding text.

[0511] Step 3:

[0512] The device then uses natural language processing technology to analyze the converted text data. The input is text data, and the output is the analysis results. Specifically, the text data is tokenized and analyzed for context and meaning.

[0513] Step 4:

[0514] The terminal uses a conversation content analysis means to check whether specific keywords are included. The input is the analyzed text data, and the output is the keyword detection results. Specifically, it searches for the presence of specific keywords (e.g., "settlement money," "accident," "bank," "transfer") in the text.

[0515] Step 5:

[0516] The device uses a sentiment analysis method to analyze the sentiment of text data. The input is text data, and the output is the sentiment analysis result. Specifically, it uses a pipeline from the transformers library to analyze the sentiment of the text (positive, negative, neutral).

[0517] Step 6:

[0518] Based on the sentiment analysis results, the device issues a warning to the user if there is a high possibility of fraud. The input is the sentiment analysis result and keyword detection result, and the output is a warning message. Specifically, if the sentiment analysis result is negative and keywords are detected, a warning is displayed to the user using the device's notification function.

[0519] Step 7:

[0520] After the terminal displays the warning message, the user can take appropriate action. The input is the warning message, and the output is the user's response. Specific actions include the user ending the call or contacting family or relatives.

[0521] Example 2

[0522] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0523] In recent years, there has been an increase in fraudulent and suspicious phone calls, with elderly people and those in socially vulnerable positions often falling victim. In such situations, there is a need for a system that allows family members and relatives to respond quickly. Furthermore, more appropriate responses can be made by providing notifications that take into account the user's emotional state. However, current systems only use voice recognition and keyword detection, and lack comprehensive support including emotion analysis.

[0524] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family members and relatives through an information sharing service such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, a sentiment analysis means, and a means for including the sentiment analysis results in a notification. This enables family members and relatives to respond quickly and appropriately when the user receives a suspicious phone call.

[0525] "Speech recognition means" refers to technology that converts voice data into text data.

[0526] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning and intent.

[0527] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and extracts specific information.

[0528] "Means for responding to specific keywords" refers to technology that detects pre-set keywords from text data.

[0529] "Communication applications" refers to software for sending and receiving messages over the Internet, such as messenger apps and social networking sites.

[0530] "Information sharing service" refers to a platform for sharing information via the Internet.

[0531] "Sentiment analysis means" refers to technology that analyzes user emotions from voice data and text data.

[0532] The "means for including emotion analysis results in notifications" refers to a technique for adding emotion information obtained by emotion analysis means to notification messages.

[0533] "Consent and confidentiality of communications measures" refers to technologies that protect user privacy and meet legal requirements.

[0534] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only under specific conditions.

[0535] MODE FOR CARRYING OUT THE INVENTION

[0536] This invention is a system that sends a notification to family and relatives via a pre-registered communication application or information sharing service when a user utters a specific keyword during a phone call. The notification includes information about a suspicious call, the content of the call (keyword), the caller's phone number, and the user's emotional state.

[0537] Hardware and software used

[0538] Hardware: Smartphones, servers

[0539] Software: speech recognition engines (e.g., Google Speech-to-Text), sentiment analysis engines (e.g., IBM Watson (registered trademark) Tone Analyzer), communication applications (e.g., messaging services), information sharing services (e.g., SNS)

[0540] System Overview

[0541] 1. Acquire audio data:

[0542] When a user receives a call, the device will record the call in real time. Specifically, the device's microphone will capture the audio data and temporarily store it in the internal storage.

[0543] 2. Speech Recognition:

[0544] The device sends the recorded voice data to a voice recognition engine (Google Speech-to-Text). The voice recognition engine converts the voice data into text data and sends it back to the device. Specifically, the voice data is sent via the Internet to the voice recognition engine's server, where it is analyzed.

[0545] 3. Keyword detection:

[0546] The server receives the text data sent from the device and detects specific keywords (e.g., settlement amount, accident, bank, transfer). The server analyzes the text data and checks whether the keywords are included. Specifically, the server parses the text data and compares it with a pre-set keyword list.

[0547] 4. Sentiment analysis:

[0548] The server sends the text and voice data to a sentiment analysis engine (IBM Watson Tone Analyzer). The sentiment analysis engine detects the user's emotions (e.g., anger, fear) and sends the results back to the server. Specifically, it analyzes emotions from the tone of the voice data and the content of the text data, and quantifies the type and intensity of the emotion.

[0549] 5. Generate and send notifications:

[0550] The server generates a notification message based on the results of keyword and emotion analysis. The server then sends the notification to family and relatives via pre-registered communication applications or information sharing services. Specifically, the notification message includes information about a suspicious call, its content (keywords), the caller's phone number, and the detected emotion.

[0551] Specific examples

[0552] When a user utters keywords such as "settlement money," "accident," "bank," or "transfer" over the phone, the device records this and sends it to a voice recognition engine. The server analyzes the text data and detects keywords. The server then sends the data to a sentiment analysis engine to analyze the user's emotions. Finally, the server generates a notification message and sends it to family and relatives via messaging services or social networking sites.

[0553] Prompt Sentence Examples

[0554] "When a user utters a specific keyword (e.g., settlement money, accident, bank, transfer) during a call, the device records it and sends it to a voice recognition engine. The server analyzes the text data and detects the keyword. The server then sends the data to a sentiment analysis engine to analyze the user's emotions. Finally, the server generates a notification message and sends it to family and relatives via a messaging service or social networking site."

[0555] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0556] Step 1:

[0557] Acquiring audio data

[0558] When a user receives a call, the device records the call in real time. Specifically, the device's microphone captures the audio data and temporarily stores it in the internal storage. The input is the user's call voice, and the output is the recorded audio data.

[0559] Step 2:

[0560] Voice Recognition

[0561] The device sends the recorded voice data to the voice recognition engine. The voice recognition engine converts the voice data into text data and sends it back to the device. Specifically, the voice data is sent via the Internet to the voice recognition engine's server, where it is analyzed. The input is the recorded voice data, and the output is text data.

[0562] Step 3:

[0563] Keyword detection

[0564] The server receives text data sent from the device and detects specific keywords. The server analyzes the text data to see if it contains any keywords. Specifically, the server parses the text data and compares it with a pre-defined keyword list. The input is the text data, and the output is a list of detected keywords.

[0565] Step 4:

[0566] sentiment analysis

[0567] The server sends the text data and voice data to the sentiment analysis engine. The sentiment analysis engine detects the user's emotions and returns the results to the server. Specifically, it analyzes emotions from the tone of the voice data and the content of the text data, and quantifies the type and intensity of the emotions. The input is text data and voice data, and the output is the sentiment analysis results.

[0568] Step 5:

[0569] Generate and send notifications

[0570] The server generates a notification message based on the results of the keyword and sentiment analysis. The server then sends the notification to family and relatives via a pre-registered communication application or information sharing service. Specifically, the notification message contains information about a suspicious call, its content (keywords), the caller's phone number, and the detected emotion. The input is a list of keywords and the results of sentiment analysis, and the output is a notification message.

[0571] (Application example 2)

[0572] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0573] In recent years, the number of fraudulent and suspicious phone calls has been increasing, with elderly people and those less well-informed becoming victims in particular. In such situations, there is a need for a system that allows family members and relatives to respond quickly. In addition, it is necessary to provide more accurate notifications by not only detecting keywords but also taking into account the user's emotional state.

[0574] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0575] In this invention, the server includes a speech recognition unit, a conversation content analysis unit using natural language processing technology, a unit that responds to specific keywords, a unit that sends the response results to family and relatives via a pre-registered SNS such as a messenger app, a unit that limits activation to calls from numbers not registered in the phone book or from anonymous numbers, a unit that handles the person's consent and confidentiality of communications, a sentiment analysis unit, and a unit that includes the sentiment analysis results in a notification. This makes it possible to quickly send a notification containing keywords and sentiment information to family and relatives when a suspicious call is received.

[0576] "Speech recognition means" is a technology for converting speech into text.

[0577] "Natural language processing technology" is a technology that allows computers to process and understand human language.

[0578] "Conversation content analysis means" is a technology for analyzing text obtained by speech recognition and understanding its content.

[0579] The "means for responding to a specific keyword" is a technology for detecting a preset keyword and performing a specific action based on it.

[0580] A "messenger app" is an application for sending and receiving messages over the Internet.

[0581] "SNS" is an abbreviation for social networking service, an online platform for users to share information with each other.

[0582] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology for activating the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0583] "Measures to address the need for consent and confidentiality of communications" refers to technologies to protect the privacy of users and ensure the confidentiality of communications.

[0584] "Sentiment analysis means" is a technology for analyzing user emotions from voice and text.

[0585] The "means for including the result of sentiment analysis in the notification" is a technique for including the result obtained by sentiment analysis in the notification message.

[0586] A system for implementing this invention includes a voice recognition means, natural language processing technology, a conversation content analysis means, a means for responding to specific keywords, a means for sending notifications via a messenger app or SNS, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to consent from the individual and confidentiality of communications, a sentiment analysis means, and a means for including the results of the sentiment analysis in notifications.

[0587] Hardware and software used

[0588] Hardware: Smartphone (microphone, internet connection)

[0589] software:

[0590] Google Speech Recognition API: Used to convert speech to text.

[0591] IBM Watson Tone Analyzer: Used to analyze sentiment from text.

[0592] Requests library: Used to send notifications to messenger apps and social media.

[0593] System Operation

[0594] 1. Speech Recognition: When a user receives a call, the smartphone's microphone captures the voice, which is then converted into text using the Google Speech Recognition API.

[0595] 2. Keyword detection: Detect specific keywords (e.g., "settlement money," "accident," "bank," "transfer") from the converted text.

[0596] 3. Sentiment Analysis: Analyze the sentiment of the text using IBM Watson Tone Analyzer, which detects emotions such as anger or fear from the user's tone of voice and speaking style.

[0597] 4. Notification sending: Based on the detected keywords and emotional information, a notification message is created and sent to pre-registered family and relatives via messenger apps or social media.

[0598] Specific examples

[0599] If a user calls and says, "I've had an accident and would like a settlement payment transferred to me," the system will detect the keywords "accident," "settlement payment," and "transfer." It will then use sentiment analysis to detect "fear." A notification message containing this information will be sent to family and relatives.

[0600] Prompt Sentence Examples

[0601] If a user calls and says, "I was in an accident and would like the settlement money transferred," the application detects the keywords "accident," "settlement money," and "transfer," and then uses sentiment analysis to detect "fear." It then notifies the family of this information.

[0602] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0603] Step 1:

[0604] When a user receives a call, the microphone on the device (smartphone) captures the voice. The input is the user's voice, and the output is the captured voice data. This voice data is used in subsequent processing steps.

[0605] Step 2:

[0606] The device sends the captured voice data to the Google Speech Recognition API, which converts the voice into text. The input is voice data, and the output is text data. This text data is the basis for analyzing the conversation content.

[0607] Step 3:

[0608] The terminal detects specific keywords from the converted text data. The input is text data, and the output is a list of detected keywords. Specifically, it checks whether the text data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[0609] Step 4:

[0610] The device sends text data to the IBM Watson Tone Analyzer for sentiment analysis. The input is text data, and the output is the sentiment analysis results. Specifically, the device analyzes the user's emotions (e.g., anger, fear, etc.) from the text data.

[0611] Step 5:

[0612] The device creates a notification message based on the detected keywords and the results of sentiment analysis. The input is a list of keywords and the results of sentiment analysis, and the output is a notification message. Specific behavior is to include content such as "There was a suspicious call. Keyword: [Detected keyword]. Emotion: [Sentiment analysis result]" in the notification message.

[0613] Step 6:

[0614] The device sends the created notification message to pre-registered family and relatives via a messenger app or SNS. The input is the notification message, and the output is the sent notification. The specific operation is to send the notification message using the API of the messenger app or SNS.

[0615] Step 7:

[0616] The server checks the status of the sent notification and logs the success or failure result. The input is the notification sending result and the output is the log data. The specific operation is to check whether the notification was sent successfully and record the result.

[0617] Example 3

[0618] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0619] Conventional telephone systems have difficulty responding appropriately to calls from numbers not registered in the phone book or from anonymous numbers, and lack a means to quickly respond to situations in which users feel anxious or in danger. Furthermore, the lack of functionality to analyze the user's emotions and notify those around them of dangerous situations makes it difficult to ensure the user's safety.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0621] In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through a pre-registered information transmission means such as a communication application, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, an emotion analysis means for analyzing the user's emotions, and a means for sending a notification when the user's emotions exceed a specific threshold. This makes it possible to quickly notify people around the user when they are in a dangerous situation.

[0622] "Speech recognition means" is a technology that analyzes voice data and converts it into text data.

[0623] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0624] The "conversation content analysis means" is a technology that analyzes the text data obtained by the voice recognition means and understands the content of the conversation.

[0625] "Means for responding to specific keywords" refers to a technique for taking specific actions based on pre-set keywords.

[0626] "Information transmission means" refers to the technology used to transmit information to other devices or users through communication applications.

[0627] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0628] "Measures to ensure the consent of the individual and the confidentiality of communications" refers to technologies that enable the system with the user's consent and protect the confidentiality of communications.

[0629] "Emotion analysis means" is a technology that analyzes emotions from a user's voice and text data and determines their emotional state.

[0630] The "means for sending a notification" is a technique for sending a notification to a pre-defined recipient when a specific condition is met.

[0631] MODE FOR CARRYING OUT THE INVENTION

[0632] This invention is a system that is activated only for calls from numbers not registered in the phone book or from anonymous numbers, analyzes the user's emotions, and sends notifications to family and relatives when the emotions exceed a certain threshold. A specific embodiment of this system is described below.

[0633] Hardware and software used

[0634] Hardware: Smartphones, servers

[0635] Software: speech recognition engines, natural language processing engines, sentiment analysis engines, communication applications (e.g. messaging services)

[0636] System Overview

[0637] 1. When a call comes in, the terminal obtains the incoming call information.

[0638] 2. The terminal checks whether the incoming number is registered in the phone book or is a blocked number.

[0639] 3. The terminal sends the incoming call information to the server.

[0640] 4. The server starts the emotion analysis engine based on the received incoming call information.

[0641] 5. The server uses an emotion analysis engine to analyze emotions from the user's voice and text.

[0642] 6. If the user's emotions exceed a certain threshold, the server will send a notification to family and relatives via a pre-registered communication application.

[0643] Specific examples

[0644] Example 1: A user receives an anonymous call. The device detects the call and sends the call information to the server. The server activates an emotion analysis engine and analyzes the emotion from the user's voice. If the server determines that the user is very angry, it sends a notification to the family via LINE saying, "The user is very angry."

[0645] Example 2: A user receives a call from a number that is not registered in the phone book. The device detects the call and sends the call information to a server. The server activates an emotion analysis engine and analyzes the emotion in the user's voice. If the server determines that the user is very scared, it sends a notification to relatives via a messaging service saying, "The user is very scared."

[0646] Prompt Sentence Examples

[0647] Prompt 1: "When an anonymous call comes in, analyze the user's emotions and notify their family via LINE if they are angry."

[0648] Prompt 2: "If the user is scared when they receive a call from a number not in their phone book, notify a relative via messaging service."

[0649] This system makes it possible to quickly notify people around the user when the user is in a dangerous situation. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0650] Step 1:

[0651] When a call comes in, the terminal obtains incoming call information. As input, it receives the incoming number and time of the call. The terminal checks whether the incoming number is not registered in the phone book or is an anonymous number. As output, it generates incoming call information (incoming number, time of the call, whether the call is registered in the phone book). Specifically, the terminal displays "You have received a call" and checks the incoming number against the phone book.

[0652] Step 2:

[0653] The terminal sends the incoming call information to the server. As input, it receives the incoming call information generated in step 1. The terminal sends the incoming call information to the server via the network. As output, it generates a status indicating that transmission is complete. Specifically, the terminal displays "Sending incoming call information to server," and when transmission is complete, it displays "Transmission completed."

[0654] Step 3:

[0655] The server starts the emotion analysis engine based on the received incoming call information. As input, it receives the incoming call information sent from the terminal. The server starts the emotion analysis engine and prepares to acquire the user's voice data. As output, it generates the startup status of the emotion analysis engine. As a specific operation, the server records "Starting emotion analysis engine" in the log.

[0656] Step 4:

[0657] The server uses an emotion analysis engine to analyze emotions from the user's voice and text. As input, it receives the user's voice data. The server inputs the voice data into the emotion analysis engine and analyzes the emotional state. As output, it generates emotion analysis results. Specifically, the server records "emotion analysis in progress" in the log and obtains the analysis results.

[0658] Step 5:

[0659] If the user's emotion exceeds a certain threshold, the server sends a notification to family and relatives through a pre-registered communication application. The server receives the emotion analysis results as input. The server generates notification content based on the emotion analysis results and sends the notification through the communication application. The server generates a notification sending status as output. Specifically, the server logs "Generating notification content" and displays "Notification sent."

[0660] In this way, it is possible for the user to quickly notify those around him when he is in a dangerous situation.

[0661] (Application example 3)

[0662] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0663] Conventional telephone systems have difficulty in responding appropriately to calls from numbers not registered in the phone book or from anonymous numbers, which can put users in dangerous situations. Furthermore, they lack the means to monitor users' emotions in real time and quickly notify family and relatives in case of an emergency. Therefore, an effective system to ensure user safety is needed.

[0664] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, a means for monitoring the user's emotions in real time using an emotion engine, and a means for automatically notifying family and relatives when the user's emotions exceed a specific threshold. This makes it possible to quickly notify people around the user even when the user is in a dangerous situation.

[0665] "Speech recognition means" is a technology that converts speech into text data.

[0666] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0667] The "conversation content analysis means" is a technology for analyzing the text data obtained by the voice recognition means and understanding the content of the conversation.

[0668] "Means for responding to specific keywords" refers to a technique for executing specific actions based on pre-set keywords.

[0669] "Means of sending reaction results to family and relatives via pre-registered messenger apps or other social networking sites" refers to a technology that sends notifications to family and relatives via pre-registered messenger apps or social networking sites when certain conditions are met.

[0670] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0671] "Measures to ensure the consent of the individual and the confidentiality of communications" refers to technologies that enable the system with the user's consent and protect the confidentiality of communications.

[0672] "Means for monitoring user emotions in real time using an emotion engine" is a technology for detecting and monitoring user emotions in real time.

[0673] "Means for automatically notifying family and relatives when a user's emotions exceed a certain threshold" refers to technology that automatically sends notifications to family and relatives when a user's emotions exceed a set threshold.

[0674] The system for implementing the present invention uses the following hardware and software.

[0675] Hardware

[0676] Smartphone

[0677] software

[0678] EmotionEngine: A library for detecting user emotions in real time

[0679] PhoneCallDetector: A library for detecting calls from numbers not registered in the phonebook or from anonymous numbers.

[0680] smtplib: Python's standard library for sending email

[0681] System Operation

[0682] 1. Voice recognition means:

[0683] Voice recognition software installed on the smartphone converts voice into text data when a call comes in.

[0684] 2. Natural language processing technology:

[0685] The converted text data is analyzed and natural language processing technology is used to understand the content of the conversation.

[0686] 3. Ways to respond to specific keywords:

[0687] Specific keywords (e.g., "settlement money," "accident," "bank," and "transfer") are detected from the analyzed text data, and specific actions are taken based on these.

[0688] 4. How to send your test results to family and relatives via pre-registered social media apps such as messenger apps:

[0689] If certain conditions are met, notifications will be sent to family and relatives via pre-registered messenger apps or social media.

[0690] 5. How to limit activation to calls from numbers not registered in your phone book or from anonymous numbers:

[0691] The system will only be activated for calls from numbers not registered in the phone book or from anonymous numbers.

[0692] 6. Consent of the individual, means of dealing with confidentiality of communications:

[0693] The system is enabled with the user's consent and protects the confidentiality of communications.

[0694] 7. Real-time monitoring of user sentiment using an emotion engine:

[0695] EmotionEngine is used to detect and monitor user emotions in real time.

[0696] 8. A way to automatically notify family and relatives if a user's emotions exceed a certain threshold:

[0697] If a user's emotions exceed a set threshold, notifications are automatically sent to family and relatives.

[0698] Specific examples

[0699] For example, if a user becomes very angry when they receive a call from a number that is not registered in their phone book, an email will be sent to their family saying, "The user is very angry. Please check."

[0700] Prompt Sentence Examples

[0701] Create a Python program that will detect the user's emotions when they receive a call from a number not registered in their phone book using an emotion engine, and send an emergency notification email to their family if their anger or fear exceeds a threshold.

[0702] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0703] Step 1:

[0704] The device detects an incoming call. The input is the incoming call signal, and the output is a flag indicating that an incoming call has occurred. The device uses PhoneCallDetector to detect calls from numbers not registered in the phone book or from anonymous numbers.

[0705] Step 2:

[0706] The terminal uses a voice recognition means to convert the incoming voice into text data. The input is the incoming voice data and the output is text data. The terminal uses voice recognition software to convert the voice into text in real time.

[0707] Step 3:

[0708] The device uses natural language processing technology to analyze text data and understand the content of the conversation. The input is text data and the output is the analysis results. The device uses a natural language processing library to analyze the text data and extract specific keywords.

[0709] Step 4:

[0710] The terminal detects specific keywords from the analysis results using a means to respond to specific keywords. The input is the analysis results, and the output is the keyword detection results. The terminal determines an action based on the specific keywords (e.g., "settlement money," "accident," "bank," "transfer").

[0711] Step 5:

[0712] The device uses an emotion engine to monitor the user's emotions in real time. The input is the user's voice data and text data, and the output is the emotional state. The device uses the Emotion Engine to detect the user's emotions and monitor them in real time.

[0713] Step 6:

[0714] The device automatically notifies family and relatives when the user's emotions exceed a certain threshold. The input is the emotional state, and the output is a notification message. If the device determines that the user is very angry or very scared based on the output of the emotion engine, it will send a notification to family and relatives via a pre-registered messenger app or social media.

[0715] Step 7:

[0716] The terminal activates the system using the user's consent and measures for protecting the confidentiality of communications, and protects the confidentiality of communications. The input is the user's consent information, and the output is the system's activation status. The terminal confirms the user's consent and activates the system, thereby protecting the confidentiality of communications.

[0717] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0718] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0719] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0720] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0721] [Second embodiment]

[0722] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0723] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0724] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0725] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0726] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0727] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0728] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0729] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0730] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0731] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0732] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0733] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0734] "Example 1"

[0735] An embodiment of the present invention is realized as an application installed on a terminal such as a smartphone or tablet. This application captures speech through the terminal's microphone and converts it into text using speech recognition technology. The converted text is analyzed using natural language processing technology to determine whether it contains specific keywords. Keywords that may indicate fraud or suspicious phone calls include "settlement money," "accident," "bank," and "transfer."

[0736] "Example 2"

[0737] If a keyword is detected, the application will send a notification to family and relatives via pre-registered messaging apps or social media. The notification will include information about the suspicious call, its content (keywords), and the caller's phone number, allowing family and relatives to respond quickly.

[0738] "Example 3"

[0739] In addition, this embodiment of the present invention can be set to activate only for calls from numbers not registered in the phone book or from anonymous numbers. This prevents false activation for calls from acquaintances. Furthermore, the function is activated with the consent of the individual, taking legal considerations into account for the confidentiality of communications.

[0740] The processing flow of each embodiment will be described below.

[0741] "Example 1"

[0742] Step 1: A user installs the application of the present invention on a device such as a smartphone or tablet.

[0743] Step 2: The application captures audio through the device's microphone.

[0744] Step 3: The captured audio is converted into text using speech recognition technology.

[0745] Step 4: The converted text is analyzed using natural language processing techniques to determine whether it contains specific keywords.

[0746] "Example 2"

[0747] Step 1: If a specific keyword is detected in the text, the application will send a notification to family and relatives via pre-registered messenger apps or social media.

[0748] Step 2: The notification will include information about the suspicious call, its content (keywords), and the caller's phone number.

[0749] "Example 3"

[0750] Step 1: The application of the present invention can be set to launch only when receiving calls from numbers not registered in the phone book or from numbers with no identification.

[0751] Step 2: Enable the feature with the individual's consent, while also taking legal considerations into account the confidentiality of communications.

[0752] Example 1

[0753] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0754] In recent years, fraudulent and suspicious phone calls have been increasing, with many elderly people and those unfamiliar with technology falling victim to these attacks. There is a need for systems that can detect these fraudulent calls in advance and warn users. However, existing systems have low accuracy in speech recognition and natural language processing, making it difficult to respond in real time. Furthermore, there is a lack of effective methods for detecting fraudulent calls while protecting user privacy.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0756] In this invention, the server includes a speech recognition unit, a conversation content analysis unit that utilizes natural language processing technology, and a unit that responds to specific keywords, which enables fraud and suspicious phone calls to be detected in real time and a warning given to the user.

[0757] "Speech recognition means" refers to technology that converts voice data into text data.

[0758] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning and context.

[0759] The "conversation content analysis means" refers to a means for analyzing text data converted by the speech recognition means using natural language processing technology.

[0760] "Means that react to specific keywords" refers to means that detect specific words or phrases contained in text data and take action based on them.

[0761] "Information sharing means" refers to the means for notifying other users or systems of analysis results and warnings.

[0762] "Means to limit activation to calls from numbers not registered in your contacts or from anonymous numbers" refers to means to activate the system only under certain conditions.

[0763] "Consent of the individual, measures to address confidentiality of communications" refers to measures to protect user privacy and meet legal requirements.

[0764] "Means for capturing audio through a microphone of a terminal and processing audio data in real time" refers to means for collecting audio using a microphone of a terminal and processing the audio data immediately.

[0765] "Means for converting voice data into text" refers to means for converting voice data into character data using voice recognition technology.

[0766] "Means for tokenizing the converted text and tagging parts of speech" refers to means for dividing text data into words and phrases and assigning part-of-speech information to each.

[0767] "Means for notifying the user when a keyword is detected" refers to means for issuing a warning or notification to the user when a specific keyword is found in text data.

[0768] This invention is realized as an application installed on devices such as smartphones and tablets. This application is a system that combines voice recognition technology and natural language processing technology to detect fraudulent and suspicious phone calls in real time and warn users.

[0769] When a user launches the application, speech is captured through the device's microphone. The captured speech is converted into text using speech recognition technology such as the Google Cloud Speech-to-Text API. The converted text is then analyzed using Python natural language processing libraries such as NLTK and spaCy. Specifically, the text is tokenized and each word is tagged with its part of speech.

[0770] The device then checks whether the analyzed text contains certain keywords (such as "settlement money," "accident," "bank," or "transfer"). If a keyword is detected, the device records the information and notifies the user, via a pop-up message or audio alert.

[0771] For example, if a user receives a call that begins with, "Hello, this is your bank and we'd like to discuss a transfer," the application can capture this speech, convert it to text, analyze the text, and detect if it contains keywords like "bank" or "transfer," thereby alerting the user to a potential scam.

[0772] Example prompt sentence:

[0773] "This application captures voice through the device's microphone and converts it into text using speech recognition technology. The converted text is then analyzed using natural language processing technology to determine whether it contains specific keywords. Keywords include 'settlement money,' 'accident,' 'bank,' and 'transfer.'"

[0774] This system includes a speech recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family and relatives through information sharing means such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the contacts or from blocked numbers, a means for ensuring the individual's consent and confidentiality of communications, a means for capturing voice through the device's microphone and processing the voice data in real time, a means for converting the voice data into text, a means for tokenizing the converted text and tagging it with parts of speech, and a means for notifying the user when a keyword is detected. This makes it possible to detect fraud and suspicious phone calls in real time and warn the user.

[0775] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0776] Step 1:

[0777] A user launches an application on their smartphone or tablet.

[0778] Input: The application is started by a user action.

[0779] Output: Application initialization complete.

[0780] Specific behavior: A user launches an application by tapping its icon on the device's home screen. The application initializes the necessary libraries and APIs for speech recognition and natural language processing (e.g., Google Cloud Speech-to-Text API, Python's NLTK library).

[0781] Step 2:

[0782] The device captures the user's voice in real time through a microphone.

[0783] Input: User's voice.

[0784] Output: Audio data.

[0785] Specific operation: When the user answers the phone and says, "Hello, this is from the bank. I'd like to talk to you about a transfer," the voice is picked up by the device's microphone. The voice data is temporarily stored in the device's memory.

[0786] Step 3:

[0787] The device sends the captured voice data to the Google Cloud Speech-to-Text API.

[0788] Input: Audio data.

[0789] Output: Text data.

[0790] How it works: The device sends the captured voice data to the Google Cloud Speech-to-Text API, which converts the voice data into text, which is then returned to the device as "Hello, this is from your bank. We'd like to discuss a transfer."

[0791] Step 4:

[0792] The text received by the device is analyzed using a Python natural language processing library (NLTK or spaCy).

[0793] Input: Text data.

[0794] Output: Tokenized text and part-of-speech tags.

[0795] Specific operation: The device divides the text it receives into tokens such as "hello," "this is," "from the bank," "but," "transfer," "about," and "we'd like to talk to you about," and tags each word with its part of speech.

[0796] Step 5:

[0797] The device checks whether certain keywords are present in the analyzed text.

[0798] Input: Tokenized text and part-of-speech tags.

[0799] Output: Keyword detection results.

[0800] What it does: The device checks whether the text contains keywords such as "bank" or "transfer." If a keyword is detected, it records the information.

[0801] Step 6:

[0802] The terminal notifies the user that the keyword has been detected.

[0803] Input: Keyword detection results.

[0804] Output: Notification to the user.

[0805] What it does: The device will display a pop-up message saying "Possible fraud. Please be careful." It can also issue a voice alert saying "Possible fraud."

[0806] (Application example 1)

[0807] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0808] In recent years, fraudulent and suspicious phone calls have been increasing, with elderly people and those less well-informed becoming victims in particular. Conventional countermeasures are insufficient, and there is a need for a system that can detect possible fraud in real time and issue a warning to users. There is also a need for a method to automatically detect potentially fraudulent calls and prompt users to take appropriate action.

[0809] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0810] In this invention, the server includes a speech recognition means, a conversation content analysis means using natural language processing technology, and a means for responding to specific keywords, which makes it possible to issue a warning to the user when a keyword that may be fraudulent is detected and provide the user with the option of whether or not to end the call.

[0811] "Speech recognition means" refers to a device or software that converts the contents of a call into text information in real time using speech-to-text technology.

[0812] "Natural language processing technology" is a technology for analyzing text data and understanding meaning and emotions, and is used to analyze conversation content.

[0813] The "conversation content analysis means" is a device or software that utilizes natural language processing technology to analyze the text data converted by the speech recognition means and detect specific keywords and phrases.

[0814] A "means for responding to specific keywords" is a device or software that detects pre-set keywords (e.g., "settlement money," "accident," "bank," "transfer," etc.) from text data and takes action based on the results.

[0815] A "messenger app" is application software that sends and receives messages over the Internet and is used to send notifications to family and relatives.

[0816] "SNS" stands for Social Networking Service, an online platform for users to share information and communicate.

[0817] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to a device or software that activates the system only when an incoming call is received from a number not registered in the user's phone book or from an anonymous number.

[0818] "Means for obtaining consent and ensuring confidentiality of communications" refers to devices or software that obtain a user's consent when using the system in order to protect the user's privacy and ensure the confidentiality of communications.

[0819] "Means for alerting the user when a potentially fraudulent keyword is detected and offering the option of ending the call" refers to a device or software that displays a warning to the user when a specific keyword is detected and offers the user the option of continuing or ending the call.

[0820] As an embodiment of the present invention, a method for installing a fraudulent call detection system on a smartphone is described. The system includes a voice recognition unit, a conversation content analysis unit using natural language processing technology, a reaction unit for specific keywords, and a unit for issuing a warning to the user and providing the user with the option of ending the call.

[0821] System Program

[0822] The system program operates as follows.

[0823] 1. Voice recognition means:

[0824] The system captures the conversation in real time through the smartphone's microphone and converts it into text using speech recognition technology. Specifically, it uses a speech recognition library and Google's speech recognition API.

[0825] 2. Natural language processing technology:

[0826] The transformed text is analyzed using natural language processing techniques, specifically using the transformers library pipeline to perform sentiment analysis on the text.

[0827] 3. Ways to respond to specific keywords:

[0828] The text data is checked for pre-set keywords (e.g., "settlement money," "accident," "bank," "transfer," etc.) to determine whether the relevant keywords are included.

[0829] 4. By providing the user with a warning and the option to end the call:

[0830] If a keyword is detected, the user will be alerted and given the option to continue or end the call.

[0831] Hardware and Software

[0832] Hardware:

[0833] Smartphone (microphone, speaker, display)

[0834] software:

[0835] Speech Recognition Library

[0836] Google Speech Recognition API

[0837] transformers library

[0838] Specific examples

[0839] For example, if a user receives a call and the caller says, "Please transfer the settlement money," the system will detect the keywords "settlement money" and "transfer." In this case, the system will display a warning to the user saying, "This may be a scam. Do you want to end the call?" and ask the user to choose whether to continue or end the call.

[0840] Prompt Sentence Examples

[0841] If a user receives a phone call and the caller says, "Please transfer the settlement money," the application will detect the keywords "settlement money" and "transfer" and alert the user.

[0842] In this way, the fraudulent call detection system can ensure user safety and prevent fraudulent incidents.

[0843] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0844] Step 1:

[0845] The device captures the call content in real time through the smartphone's microphone. The input is voice data, and the output is raw voice data that is passed to the voice recognition means. Specifically, when the user starts a call, the microphone starts capturing the voice.

[0846] Step 2:

[0847] The device converts the captured voice data into text using a voice recognition means. The input is the voice data acquired in step 1, and the output is text data. Specifically, it uses a voice recognition library and calls Google's voice recognition API to convert the voice into text.

[0848] Step 3:

[0849] The device uses natural language processing technology to analyze the converted text data. The input is the text data generated in step 2, and the output is the analysis results on the sentiment and meaning of the text. Specifically, it uses a pipeline from the transformers library to perform sentiment analysis of the text.

[0850] Step 4:

[0851] The terminal checks the text data for pre-defined keywords using a means that responds to specific keywords. The input is the text data obtained in step 3, and the output is a flag indicating whether the keyword was detected. Specifically, it checks whether the text data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[0852] Step 5:

[0853] If a keyword is detected, the terminal will issue a warning to the user and offer the option of whether to end the call. The input is the keyword detection flag obtained in step 4, and the output is the user's choice. Specifically, the terminal will display a warning message on the display and offer the user the option of "Possible fraud. Do you want to end the call?"

[0854] Step 6:

[0855] The user confirms the warning message and chooses whether to continue or end the call. The input is the warning message displayed in step 5, and the output is the user's choice. As a specific operation, if the user selects "End," the call is automatically ended.

[0856] In this way, the fraudulent call detection system can analyze the content of calls in real time, warn users if there is a possibility of fraud, and prompt them to take appropriate action.

[0857] Example 2

[0858] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0859] In modern society, there has been an increase in fraudulent and suspicious phone calls, with elderly people and those unfamiliar with technology being particularly vulnerable to these attacks. To combat this, there is a need for systems that can quickly notify family and relatives and encourage them to take appropriate action. However, existing systems lack the ability to recognize voices, detect keywords, and send notifications in real time, making it difficult to implement effective countermeasures.

[0860] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0861] In this invention, the server includes a voice recognition unit, a conversation content analysis unit using natural language processing technology, a unit that responds to specific keywords, a unit that sends the response results to family and relatives through an information sharing service such as a pre-registered communication application, a unit that limits activation to calls from numbers not registered in the phone book or from anonymous numbers, a unit that handles the individual's consent and confidentiality of communications, a unit that converts voice data into text data, a unit that compares the text data with a keyword list, a unit that creates a notification when a keyword is detected, and a unit that sends the notification in real time. This enables a user to quickly and reliably notify family and relatives when they receive a suspicious phone call and urge them to take appropriate action.

[0862] "Speech recognition means" is a technology for converting voice data into text data.

[0863] "Means for analyzing conversation content using natural language processing technology" is a technology for analyzing text data and understanding meaning and intent.

[0864] "Means for responding to specific keywords" refers to technology that compares text data with a pre-registered keyword list and detects matching keywords.

[0865] "Information sharing services such as communication applications" refers to services for sharing information over the Internet, such as messenger apps and social networking sites.

[0866] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology for controlling the operation of the system based on specific conditions.

[0867] "Measures to address the need for consent and confidentiality of communications" refers to techniques to protect user privacy and meet legal requirements.

[0868] The "means for converting voice data into text data" is a technology for converting voice data into text information using voice recognition technology.

[0869] The "means for checking text data against a keyword list" is a technique for comparing the converted text data with a pre-set keyword list to detect matching keywords.

[0870] The "means for generating a notification when a keyword is detected" refers to a technique for generating a notification message when a particular keyword is detected.

[0871] The "means for sending notifications in real time" refers to a technique for instantly sending generated notification messages to family and relatives.

[0872] This invention is a system for quickly and reliably notifying family and relatives when a user receives a suspicious phone call, and encouraging them to take an appropriate action. This system includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through an information sharing service such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the phone book or from numbers withheld, a means for ensuring the consent of the user and confidentiality of communications, a means for converting voice data into text data, a means for comparing the text data with a keyword list, a means for creating a notification when a keyword is detected, and a means for transmitting the notification in real time.

[0873] The server monitors users' phone calls in real time and converts them into text using the Google Cloud Speech-to-Text API, a voice recognition software. The converted text is then compared with a pre-registered list of keywords.

[0874] The device is the user's smartphone or PC, and is responsible for sending the contents of the call to the server. When the call ends, the device sends the text data of the call to the server.

[0875] The server analyzes the received text data to see if it contains any keywords, and if it does, it uses a communication service API to send notifications to family and relatives via pre-registered messenger apps or social media.

[0876] The notification will include information about the suspicious call, its contents (keywords), the caller's phone number, etc. This will enable family members and relatives to respond quickly.

[0877] As a concrete example, consider the case where a user receives a phone call containing the keyword "scam." In this case, the server detects the keyword "scam" and uses the communication service API to send the following notification to the family:

[0878] Example of notification content:

[0879] There was a suspicious phone call.

[0880] Content: Fraud

[0881] Caller's phone number: +81-90-XXXX-XXXX

[0882] Example prompts to input to the generative AI model:

[0883] Create a program that will send a notification to a family member if the user receives a phone call containing the keyword "scam." The notification should include information about the suspicious call, its content (keywords), and the caller's phone number.

[0884] In this way, the system can detect suspicious calls in real time and quickly notify family and relatives to keep users safe.

[0885] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0886] Step 1:

[0887] The user receives a call.

[0888] When a user receives a call on their smartphone, the call starts. The call start event triggers the system to start working. The input is the incoming call, and the output is the call start event.

[0889] Step 2:

[0890] The device records the call.

[0891] The device records the contents of the call in real time. The recording is done using the smartphone's built-in microphone. The recorded data is temporarily stored in the device. The input is the call audio, and the output is the recorded audio data.

[0892] Step 3:

[0893] The device sends the recorded audio data to the server.

[0894] When the call ends, the device sends the recorded voice data to the server. The transmission is performed using the HTTPS protocol, and the data is encrypted. The input is the recorded voice data, and the output is the transmission of the voice data to the server.

[0895] Step 4:

[0896] The server converts the audio data into text.

[0897] The server converts the received voice data into text using the Google Cloud Speech-to-Text API. The converted text data is stored on the server. The input is voice data, and the output is text data.

[0898] Step 5:

[0899] The server matches the text data with a list of keywords.

[0900] The server compares the converted text data with a pre-registered keyword list, which includes words such as "fraud," "suspicious," and "danger." The input is the text data and the keyword list, and the output is the keyword detection results.

[0901] Step 6:

[0902] If the server detects a keyword, it creates a notification.

[0903] If the server detects a keyword, it creates a notification. The notification contains information about a suspicious call, its content (keyword), and the caller's phone number. The input is the keyword detection result, and the output is the notification message.

[0904] Step 7:

[0905] The server sends notifications to family and relatives.

[0906] The server uses the communication service API to send notifications to family and relatives via pre-registered messenger apps and SNS. The notifications are sent in real time, allowing family and relatives to respond quickly. The input is the notification message, and the output is the notification sent to family and relatives.

[0907] (Application example 2)

[0908] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0909] In recent years, there has been an increase in telephone fraud and suspicious phone calls, with elderly people and those less well-informed becoming victims in particular. In order to respond quickly to such suspicious calls, a system that can immediately send notifications to family and relatives is needed. However, current systems lack the functionality to analyze call content in real time, detect specific keywords, and send notifications. Furthermore, there is a need to provide more detailed information by including recordings of calls in the notification content.

[0910] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family members and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the person's consent and confidentiality of communications, a means for sending a notification to family members and relatives based on the detected keywords, and a means for including in the notification the content of the suspicious call (keywords), the caller's phone number, and recorded data of the call. This makes it possible to quickly provide detailed information to family members and relatives when a suspicious call is received.

[0911] "Speech recognition means" refers to technology that converts speech into text data.

[0912] "Natural language processing technology" refers to the technology of processing and analyzing human language using a computer.

[0913] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and extracts specific information.

[0914] "Means for responding to specific keywords" refers to technology that detects pre-set keywords and operates based on the detection results.

[0915] A "messenger app" refers to an application for sending and receiving messages over the Internet.

[0916] "SNS" is an abbreviation for social networking service, and refers to an online platform where users can share information with each other.

[0917] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0918] "Measures to ensure the individual's consent and confidentiality of communications" refers to technologies for obtaining the individual's consent when using the system and for protecting the confidentiality of communication content.

[0919] "Means for sending notifications to family and relatives based on detected keywords" refers to technology that sends notifications to family and relatives when specific keywords are detected.

[0920] "Means for the notification to include the content of the suspicious call (keywords), the caller's phone number, and a recording of the call" refers to a technology in which the notification sent includes the content of the suspicious call, the caller's phone number, and a recording of the call.

[0921] A system for implementing this invention includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for dealing with the consent of the individual and confidentiality of communications, a means for sending notifications to family and relatives based on detected keywords, and a means for the notification to include the content of the suspicious call (keywords), the caller's phone number, and recorded data of the call.

[0922] System Program

[0923] This system can be implemented using Python. It uses the Google Cloud Speech-to-Text API for speech recognition, the NLTK (Natural Language Toolkit) for natural language processing, and the smtplib library for sending emails.

[0924] Processing Description

[0925] The server first converts the contents of the call into text data using voice recognition. This text data is then analyzed using natural language processing technology to check whether it contains specific keywords. If a specific keyword is detected, the server will use the results to send a notification to family and relatives via pre-registered messenger apps or social media.

[0926] The notification will include the content of the suspicious call (detected keywords), the phone number from which it originated, and a recording of the call, allowing family members or relatives to take action quickly.

[0927] Hardware and software used

[0928] Hardware: Smartphones, servers

[0929] Software: Python, Google Cloud Speech-to-Text API, NLTK, smtplib

[0930] Specific examples

[0931] For example, if a user receives a call saying, "This is a fraudulent call," a speech recognition means converts this speech into text data. Next, natural language processing technology is used to analyze this text data and detect the keyword "fraud." Based on the detected keyword, the server sends a notification to family and relatives. This notification includes the following content: "There has been a suspicious call. The caller's phone number is: 090-XXXX-XXXX. The detected keyword is: fraud."

[0932] Prompt Sentence Examples

[0933] Create a program that checks the content of phone calls and sends a notification if certain keywords are included. The keyword list includes "fraud," "suspicious," "suspicious," and "transfer." The notification will include the content of the suspicious call (keywords), the caller's phone number, and a recording of the call. The notification will be sent by email.

[0934] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0935] Step 1:

[0936] The user receives a call. The voice data of the call is input to the terminal. The terminal collects this voice data in real time.

[0937] Step 2:

[0938] The device uses a speech recognition means to convert the collected voice data into text data. Specifically, it uses the Google Cloud Speech-to-Text API to analyze the voice data and generate corresponding text data. The input is voice data, and the output is text data.

[0939] Step 3:

[0940] The server analyzes the text data using natural language processing technology. Specifically, it uses NLTK (Natural Language Toolkit) to tokenize the text data and check whether it contains specific keywords. The input is the text data, and the output is a list of detected keywords.

[0941] Step 4:

[0942] If a specific keyword is detected, the server creates a notification based on the results. The notification includes the content of the suspicious call (detected keywords), the caller's phone number, and the call recording. The input is a list of detected keywords, the caller's phone number, and the call recording, and the output is a notification message.

[0943] Step 5:

[0944] The server sends the created notification message to family and relatives via pre-registered messenger apps or SNS. Specifically, it sends emails using the smtplib library. The input is the notification message, and the output is the sent notification.

[0945] Step 6:

[0946] Family members and relatives can check the received notification and respond promptly if necessary. The input is the notification message and the output is the response action.

[0947] The above processing steps enable the user to quickly provide detailed information to family and relatives when they receive a suspicious phone call.

[0948] Example 3

[0949] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0950] Conventional telephone answering systems were unable to respond appropriately to calls from numbers not registered in the phone book or from anonymous numbers, and they malfunctioned when receiving calls from acquaintances and lacked legal consideration for the confidentiality of communications. Furthermore, users had insufficient means to manage the system settings, making them inconvenient.

[0951] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through a pre-registered communication application such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, a means for generating a response message using a generative AI model, a means for sending the generated response message to the caller, and a means for the user to manage system settings. This makes it possible to respond appropriately to calls from numbers not registered in the phone book or from anonymous numbers, prevent malfunctions when receiving calls from acquaintances, and ensure legal consideration for confidentiality of communications.

[0952] "Speech recognition means" refers to technology that converts speech into text data.

[0953] "Natural language processing technology" refers to technology that processes, understands, and generates human language using a computer.

[0954] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and understands the content of the conversation.

[0955] "Means for responding to specific keywords" refers to a technology for performing specific actions based on pre-set keywords.

[0956] "Communication application" refers to software that sends and receives messages over the Internet, such as a messenger app.

[0957] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[0958] "Means for addressing the individual's consent and confidentiality of communications" refers to technologies that enable systems with the user's consent and take legal considerations to protect the confidentiality of communications.

[0959] A "generative AI model" refers to a model that uses artificial intelligence to generate text data.

[0960] "Means for generating a response message" refers to technology that uses a generative AI model to create a response message for the caller.

[0961] "Means for sending a response message to the sender" refers to a technique for sending the generated response message to the sender.

[0962] "Means for users to manage system settings" refers to the interface through which users can change and manage the system's operating settings.

[0963] MODE FOR CARRYING OUT THE INVENTION

[0964] This invention is a system that activates only when receiving calls from numbers not registered in the phone book or from anonymous numbers, and uses a generative AI model to generate a response message and send it to the caller. A specific embodiment of this system is described below.

[0965] Hardware and software used

[0966] The server uses a communication API to monitor incoming calls. Specifically, it uses a communication service API to detect incoming calls in real time. The phone book database is managed using MySQL and is used to verify caller IDs. OpenAI's GPT-4 is used as the generative AI model. The interface for users to manage system settings is built using React.

[0967] Program processing

[0968] The server monitors incoming calls through the communication service API. When a call is received, the server checks the phone number against its phone book database. If the number is not registered in the phone book or is blocked, the server activates a generative AI model, sends a prompt, and generates a response message. The generated response message is sent to the caller through the communication service API.

[0969] Specific examples

[0970] If the user receives a call from a number not in their phone book, the server checks the number against the phone book database to ensure it is not registered. The server then sends the following prompt to the generative AI model:

[0971] Example prompt sentence:

[0972] "This number is not in the phone book. Please generate a response message."

[0973] The generative AI model generates a response message like this:

[0974] "Thank you for calling. This number is not in the phone book and cannot be answered. We apologize."

[0975] The server sends this message to the caller via the communication service API, allowing users to prevent accidental calls from acquaintances and to respect legal considerations regarding the confidentiality of communications.

[0976] Managing User Settings

[0977] Users can enable or disable this feature through a system settings screen, which is built using React and includes a checkbox for user consent. When a user changes the settings, the information is sent to the server and stored in a database.

[0978] In this way, the present invention can respond appropriately to calls from numbers not registered in the telephone directory or from anonymous numbers, prevent malfunctions in response to calls from acquaintances, and take legal considerations regarding the secrecy of communications. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0979] Program processing flow

[0980] Step 1: Detecting an incoming call

[0981] The server uses the Communication Services API to monitor incoming calls in real time, and when an incoming call comes in, the Communication Services API sends a notification to the server.

[0982] Input: Call notification (caller ID, call time, etc.)

[0983] Output: Incoming call details

[0984] Specific behavior:

[0985] The server receives a notification from the communication service API and retrieves the details of the incoming call (caller ID, time of call, etc.).

[0986] Step 2: Phone number verification

[0987] The server checks the incoming caller number against a phone book database, which is managed using MySQL.

[0988] Input: Caller ID

[0989] Output: Information on whether the contact is registered in the phone book

[0990] Specific behavior:

[0991] The server queries the MySQL database to see if the caller ID is registered.

[0992] Step 3: Launching the generative AI model

[0993] If the caller ID is not registered in the phone book, the server launches a generative AI model, which uses OpenAI's GPT-4.

[0994] Input: Information that the caller number is not registered in the phone book

[0995] Output: Launch of a generative AI model

[0996] Specific behavior:

[0997] The server sends a prompt to the generative AI model, requesting it to generate a response message.

[0998] Step 4: Generate a response message

[0999] The generative AI model generates a response message based on the prompt sentence.

[1000] Input: prompt statement

[1001] Output: Response message

[1002] Specific behavior:

[1003] The generative AI model receives the following prompt and generates a response message:

[1004] "This number is not in the phone book. Please generate a response message."

[1005] Example of the message generated:

[1006] "Thank you for calling. This number is not in the phone book and cannot be answered. We apologize."

[1007] Step 5: Send a response message

[1008] The server sends the generated response message to the caller via the communication service API.

[1009] Input: Response message

[1010] Output: Send message to caller

[1011] Specific behavior:

[1012] The server uses the communication services API to send the generated message to the caller as an SMS or voice message.

[1013] Step 6: Manage User Settings

[1014] Users can enable or disable this feature through a system settings screen, which is built using React.

[1015] Input: User setting change

[1016] Output: Update configuration information

[1017] Specific behavior:

[1018] The user can enable or disable the feature by checking the checkbox on the settings screen. The server saves this setting in the database and applies it the next time a call is received.

[1019] (Application example 3)

[1020] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1021] In recent years, telephone fraud and nuisance calls have been increasing, and there is a particular need to respond to calls from numbers not registered in the phone book or from anonymous numbers. However, conventional systems have malfunctioned when receiving calls from acquaintances and have insufficient legal consideration for the confidentiality of communications. In addition, there have been problems with users being unable to quickly recognize suspicious calls and take appropriate action.

[1022] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means that utilizes natural language processing technology, and a means that responds to specific keywords. This makes it possible to detect an incoming call from a number not registered in the phone book as a suspicious call and notify the user.

[1023] "Speech recognition means" is a technology that converts voice into a digital signal and analyzes its content.

[1024] "Means for analyzing conversation content using natural language processing technology" refers to technology that uses natural language processing technology to analyze the content of speech and text and understand their meaning.

[1025] "Means for responding to specific keywords" refers to a technology that responds to a preset keyword and performs a specific action.

[1026] "Means of communication" refers to the technology used to send and receive information through messenger apps, etc.

[1027] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology that activates the system only when there is an incoming call from a number not registered in the phone book or from anonymous numbers.

[1028] "Measures to address the need for consent and confidentiality of communications" refers to technologies that enable functions with the user's consent and protect the confidentiality of communications.

[1029] "Means for detecting suspicious calls and notifying the user" refers to technology that detects calls from numbers not registered in the phone book as suspicious calls and notifies the user of this information.

[1030] The system for carrying out this invention is mainly composed of a smartphone and includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a communication means, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for ensuring the consent of the person and confidentiality of communications, and a means for detecting suspicious calls and notifying the user.

[1031] Program processing explanation

[1032] Voice recognition means

[1033] When a call comes in, the application installed on the smartphone performs voice recognition. The voice recognition method converts the voice into a digital signal and analyzes its content. For this, it uses voice recognition technology such as Google Speech-to-Text API.

[1034] Conversation content analysis using natural language processing technology

[1035] The speech data acquired by the speech recognition means is converted into text using natural language processing technology. This text data is then further analyzed to understand the content of the conversation. For example, natural language processing libraries such as NLTK and spaCy are used here.

[1036] How to respond to specific keywords

[1037] The analyzed text data reacts to specific keywords. For example, if keywords such as "settlement money," "accident," "bank," and "transfer" are included, the system will react to them.

[1038] communication means

[1039] If a specific keyword is triggered, the system will send a notification to family and relatives via a messaging app or other communication method, using messaging service APIs and communication service APIs.

[1040] A method to limit activation to calls from numbers not registered in the phone book or from anonymous numbers

[1041] The system will only wake up if there is a call from a number not in the phonebook or from a blocked number. This functionality is implemented using Android's TelephonyManager class.

[1042] Consent of the individual, measures to ensure confidentiality of communications

[1043] The system will obtain user consent to enable the feature and take steps to protect the confidentiality of communications, including a process for obtaining consent through the user interface.

[1044] Detecting suspicious calls and notifying users

[1045] If a call comes from a number that is not registered in the phone book, the system will detect the number as a suspicious call and notify the user. This notification is done using the smartphone's notification function.

[1046] Specific examples

[1047] For example, if a user receives a call from an unknown number, the app will detect the number as suspicious and notify the user, allowing them to take action quickly.

[1048] Prompt Sentence Examples

[1049] "Please create an app that detects a call from a number not registered in the phone book as a suspicious call and notifies the user."

[1050] In this way, specific embodiments for carrying out the invention have been described.

[1051] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1052] Step 1:

[1053] The device detects an incoming call. The input is the incoming number, and the output is a signal indicating that an incoming call has been received. Specifically, the Android TelephonyManager class is used to monitor the phone state.

[1054] Step 2:

[1055] The terminal checks the incoming number against the phone book. The input is the incoming number and the phone book data, and the output is the result of determining whether the incoming number is registered in the phone book. Specifically, it searches the phone book database to see if the incoming number exists.

[1056] Step 3:

[1057] The device checks whether the call is from a number not registered in the phone book or from an anonymous number. The input is the result of the judgment in step 2, and the output is the result of the judgment on whether to start the system. Specifically, if the judgment result is "unregistered," start the system.

[1058] Step 4:

[1059] The device begins speech recognition. The input is the incoming voice and the output is the audio data. Specifically, it uses the Google Speech-to-Text API to convert the voice into a digital signal.

[1060] Step 5:

[1061] The device analyzes the voice data using natural language processing technology. The input is voice data and the output is text data. Specifically, it uses natural language processing libraries such as NLTK and spaCy to convert the voice data into text and analyze the content.

[1062] Step 6:

[1063] The device checks the text data against specific keywords. The input is the text data, and the output is the keyword detection results. Specifically, it checks whether the data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[1064] Step 7:

[1065] If a keyword is detected on the device, a notification is sent to family and relatives using a communication method. The input is the keyword detection result and the notification content, and the output is the notification sending result. Specifically, the notification is sent using a messaging service API or a communication service API.

[1066] Step 8:

[1067] The device detects the call as suspicious and notifies the user. The input is the incoming number and keyword detection results, and the output is a notification to the user. Specifically, the smartphone's notification function is used to notify the user of the suspicious call information.

[1068] In this way, the specific operations, inputs, and outputs in each processing step have been explained.

[1069] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1070] "Example 1"

[1071] One embodiment of the present invention is a system incorporating an emotion engine. This system captures a user's voice and converts it into text using speech recognition technology. The converted text is analyzed using natural language processing technology to determine whether it contains specific keywords. The emotion engine then analyzes the user's voice for emotion and feeds the results back to the conversation content analysis means.

[1072] "Example 2"

[1073] For example, if a user utters keywords such as "settlement money," "accident," "bank," or "transfer" over the phone, the system will detect this and send a notification to family and relatives via pre-registered messenger apps or social media. Furthermore, if the emotion engine detects emotions such as anger or fear from the user's tone of voice or speaking style, these will also be sent as part of the notification.

[1074] "Example 3"

[1075] Additionally, if the emotion engine determines that a user's emotions exceed a certain threshold, for example, that the user is extremely angry or extremely scared, the system will automatically send a response to family and relatives via pre-registered social media apps such as messenger apps, allowing users to quickly notify those around them when they are in danger.

[1076] The processing flow of each embodiment will be described below.

[1077] "Example 1"

[1078] Step 1: Capture the user's voice.

[1079] Step 2: Convert the speech to text using speech recognition technology.

[1080] Step 3: The converted text is analyzed using natural language processing techniques to determine whether it contains specific keywords.

[1081] Step 4: The emotion engine analyzes the emotions from the user's voice and feeds the results back to the conversation content analysis means.

[1082] "Example 2"

[1083] Step 1: If the user utters keywords such as "settlement money," "accident," "bank," or "transfer" over the phone, the system will detect this.

[1084] Step 2: The emotion engine detects emotions from the user's tone of voice and speaking style.

[1085] Step 3: Combine the detected keywords and emotions and send notifications to family and relatives via pre-registered messenger apps or social media.

[1086] "Example 3"

[1087] Step 1: The emotion engine detects when a user's emotion exceeds a certain threshold.

[1088] Step 2: If the emotion engine determines that the person is very angry or very scared, the system will automatically send the reaction result to family and relatives via pre-registered social media apps such as messenger apps.

[1089] Example 1

[1090] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1091] In recent years, fraudulent and suspicious phone calls have been increasing, with elderly people and those less well-informed becoming victims in particular. In order to prevent such fraudulent calls, a system is needed that can analyze the content of phone calls in real time and detect dangerous keywords and changes in emotion. However, current systems are limited to voice recognition and keyword detection, and lack comprehensive measures that include emotion analysis. This makes it difficult to accurately detect possible fraud and respond quickly.

[1092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1093] In this invention, the server includes a voice recognition unit, a conversation content analysis unit that utilizes natural language processing technology, and a unit that responds to specific keywords, which enables the server to analyze the contents of fraudulent or suspicious phone calls in real time and detect dangerous keywords and emotional changes with high accuracy.

[1094] "Speech recognition means" refers to technology that converts voice data into text data.

[1095] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning and context.

[1096] "Conversation content analysis means" refers to a means for analyzing the content of a conversation using natural language processing technology and extracting specific information.

[1097] "Means for responding to specific keywords" refers to means for detecting whether a preset keyword is included in text data.

[1098] "Communications application" refers to software that allows you to send and receive messages and information over the Internet.

[1099] "Information and communication means" refers to technologies and devices for sending and receiving data.

[1100] "Emotion analysis means" refers to a means for analyzing emotions from voice data or text data and outputting the results.

[1101] "Means for providing feedback to the conversation content analysis means" refers to a means for returning the results of the emotion analysis to the conversation content analysis means and reflecting them in the analysis results.

[1102] MODE FOR CARRYING OUT THE INVENTION

[1103] The present invention is realized as an application installed on devices such as smartphones and tablets. This application is a system that detects fraudulent and suspicious phone calls and issues a warning to the user by combining speech recognition technology, natural language processing technology, and emotion analysis technology.

[1104] Hardware and software used

[1105] Hardware: Smartphones, tablets

[1106] Software: Speech recognition technology (e.g., speech recognition API), natural language processing technology (e.g., natural language processing library), sentiment analysis technology (e.g., sentiment analysis engine)

[1107] System Operation

[1108] 1. Audio capture:

[1109] The user speaks into the microphone on their smartphone or tablet.

[1110] The device captures audio through the microphone.

[1111] 2. Speech Recognition:

[1112] The device sends the captured voice data to the voice recognition API.

[1113] The server receives the voice data and performs voice recognition processing.

[1114] The server converts the voice data into text data and returns the converted text to the terminal.

[1115] 3. Text analysis:

[1116] The text data received by the device is passed to a natural language processing library.

[1117] The device analyzes the text data and determines whether it contains specific keywords.

[1118] 4. Emotion analysis:

[1119] The device sends text and voice data to the emotion analysis engine.

[1120] The server analyzes emotions from the voice data and returns the results to the device.

[1121] The terminal feeds back the results of the emotion analysis to the conversation content analysis means.

[1122] 5. Feedback of results:

[1123] The device integrates the results of text analysis and sentiment analysis and notifies the user.

[1124] The terminal displays a warning message to the user.

[1125] Specific examples

[1126] For example, if the user says, "I received confirmation of the transfer from my bank," the process will proceed as follows:

[1127] 1. Audio capture:

[1128] The user speaks into the microphone on their smartphone, "I have received confirmation of the transfer from the bank."

[1129] The device captures the audio.

[1130] 2. Speech Recognition:

[1131] The device sends voice data to the voice recognition API.

[1132] The server converts the voice data into text data and returns it to the terminal.

[1133] The device receives a text message saying, "Bank has confirmed transfer."

[1134] 3. Text analysis:

[1135] The device passes the text data to a natural language processing library.

[1136] The terminal extracts the keywords "bank" and "transfer."

[1137] 4. Emotion analysis:

[1138] The device sends text and voice data to the emotion analysis engine.

[1139] The server analyzes the emotion of "anxiety" and returns it to the device.

[1140] The terminal feeds back the results of the emotion analysis to the conversation content analysis means.

[1141] 5. Feedback of results:

[1142] The device combines the results of text analysis and sentiment analysis and displays a warning message to the user saying, "This may be a scam. Be careful."

[1143] Prompt Sentence Examples

[1144] Examples of prompts to be input to a generative AI model include:

[1145] If a user says, "I received confirmation of a transfer from my bank," explain how the system converts this into text using speech recognition technology, analyzes keywords using natural language processing technology, and analyzes emotions using an emotion engine.

[1146] By inputting this prompt sentence into a generative AI model, a sentence can be generated that explains the system's processing in detail.

[1147] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1148] Step 1:

[1149] Audio Capture

[1150] The user speaks into the microphone on their smartphone or tablet.

[1151] The device captures audio through the microphone.

[1152] Input: User's voice data

[1153] Output: Captured audio data

[1154] What happens: The user says, "I received confirmation of transfer from my bank."

[1155] Step 2:

[1156] Voice Recognition

[1157] The device sends the captured voice data to the voice recognition API.

[1158] The server receives the voice data and performs voice recognition processing.

[1159] The server converts the voice data into text data and returns the converted text to the terminal.

[1160] Input: Captured audio data

[1161] Output: Converted text data

[1162] Specific operation: The audio "The bank has confirmed the transfer" is converted to the text "The bank has confirmed the transfer".

[1163] Step 3:

[1164] Text analytics

[1165] The text data received by the device is passed to a natural language processing library.

[1166] The device analyzes the text data and determines whether it contains specific keywords.

[1167] Input: Converted text data

[1168] Output: Keyword extraction results

[1169] Specific operation: The keywords "bank" and "transfer" are extracted from the text "The bank has confirmed the transfer."

[1170] Step 4:

[1171] Emotion analysis

[1172] The device sends text and voice data to the emotion analysis engine.

[1173] The server analyzes emotions from the voice data and returns the results to the device.

[1174] The terminal feeds back the results of the emotion analysis to the conversation content analysis means.

[1175] Input: Text and audio data

[1176] Output: Sentiment analysis results

[1177] Specific behavior: The emotion "anxiety" is detected from the voice.

[1178] Step 5:

[1179] Feedback of results

[1180] The device integrates the results of text analysis and sentiment analysis and notifies the user.

[1181] The terminal displays a warning message to the user.

[1182] Input: Keyword extraction results and sentiment analysis results

[1183] Output: A warning message to the user.

[1184] Specific behavior: A warning message will appear saying, "This may be a scam. Be careful."

[1185] (Application example 1)

[1186] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1187] In recent years, fraudulent and suspicious phone calls have been increasing, with the number of victims targeting the elderly becoming particularly serious. Conventional countermeasures only respond to specific keywords, making it difficult to completely eliminate the possibility of fraud. Furthermore, because they do not take into account the user's emotions, they may miss signs of fraud. This poses a challenge as users remain at high risk of becoming victims of fraud.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1189] In this invention, the server includes a speech recognition unit, a conversation content analysis unit that utilizes natural language processing technology, a unit that responds to specific keywords, a sentiment analysis unit, and a unit that issues a warning based on the sentiment analysis results. This allows for real-time analysis of the content of calls, and by combining the detection of specific keywords with the sentiment analysis, it is possible to detect the possibility of fraud with high accuracy and issue a warning to the user.

[1190] "Speech recognition means" refers to a device or software that uses speech-to-text technology to analyze the content of a user's speech.

[1191] "Natural language processing technology" is a technology for analyzing text data and understanding its meaning and context.

[1192] The "conversation content analysis means" is a device or software that uses natural language processing technology to analyze the text converted by the speech recognition means and extract specific information.

[1193] A "means for responding to specific keywords" is a device or software that detects whether a predetermined keyword is contained within a text.

[1194] An "emotion analysis means" is a device or software that analyzes a user's emotions from voice or text and outputs the results.

[1195] The "means for issuing a warning based on the result of emotion analysis" refers to a device or software that issues a warning to the user based on the result of the emotion analysis means.

[1196] A "messenger app" is an application that sends and receives messages over the Internet.

[1197] "SNS" is an abbreviation for social networking service, an online platform for users to share information with each other.

[1198] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to a device or software that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[1199] "Means for protecting the privacy of communications with the user's consent" refers to devices or software that protect the privacy of communications with the user's consent.

[1200] A system for carrying out this invention is configured as an application installed on a terminal such as a smartphone, tablet, etc. This application includes a speech recognition means, natural language processing technology, a conversation content analysis means, a means for responding to specific keywords, a sentiment analysis means, and a means for issuing a warning based on the sentiment analysis results.

[1201] Hardware and software used

[1202] Hardware: Smartphone, tablet (with microphone)

[1203] Software: Speech recognition library, transformers library pipeline

[1204] Data processing and calculation

[1205] 1. Speech recognition: Speech is captured through the device's microphone and converted into text using a speech recognition library, thereby obtaining the user's speech as text data.

[1206] 2. Keyword detection: The converted text is checked to see if it contains specific keywords (e.g., "settlement money," "accident," "bank," "transfer"), which helps detect calls that may be fraudulent.

[1207] 3. Sentiment Analysis: Using the transformers library pipeline, the text is analyzed for sentiment. If negative sentiment is detected, it is determined to be highly likely to be fraudulent.

[1208] 4. Warning: Based on the results of sentiment analysis, if there is a high possibility of fraud, a warning is issued to the user. The warning is displayed using the device's notification function.

[1209] Specific examples

[1210] For example, if a user receives a call containing keywords such as "settlement money" or "transfer money," the app will automatically detect this and perform sentiment analysis. If it determines that the call is likely to be fraudulent, it will display a warning to the user.

[1211] Prompt Sentence Examples

[1212] When a user receives a call, the call content is analyzed in real time to determine whether it contains specific keywords (e.g., "settlement money," "accident," "bank," "transfer"). In addition, an emotion engine is used to analyze the emotions of the other party during the call, and if there is a high possibility of fraud, a warning is issued to the user.

[1213] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1214] Step 1:

[1215] The device initiates a call. During the call, audio data is captured in real time through the device's microphone. The input is audio data, and the output is a stream of audio data. Specifically, the device's microphone captures audio and stores the audio data in a buffer.

[1216] Step 2:

[1217] The device uses a speech recognition means to convert the captured voice data into text. The input is a stream of voice data, and the output is text data. Specifically, the device uses a speech recognition library to analyze the voice data and generate the corresponding text.

[1218] Step 3:

[1219] The device then uses natural language processing technology to analyze the converted text data. The input is text data, and the output is the analysis results. Specifically, the text data is tokenized and analyzed for context and meaning.

[1220] Step 4:

[1221] The terminal uses a conversation content analysis means to check whether specific keywords are included. The input is the analyzed text data, and the output is the keyword detection results. Specifically, it searches for the presence of specific keywords (e.g., "settlement money," "accident," "bank," "transfer") in the text.

[1222] Step 5:

[1223] The device uses a sentiment analysis method to analyze the sentiment of text data. The input is text data, and the output is the sentiment analysis result. Specifically, it uses a pipeline from the transformers library to analyze the sentiment of the text (positive, negative, neutral).

[1224] Step 6:

[1225] Based on the sentiment analysis results, the device issues a warning to the user if there is a high possibility of fraud. The input is the sentiment analysis result and keyword detection result, and the output is a warning message. Specifically, if the sentiment analysis result is negative and keywords are detected, a warning is displayed to the user using the device's notification function.

[1226] Step 7:

[1227] After the terminal displays the warning message, the user can take appropriate action. The input is the warning message, and the output is the user's response. Specific actions include the user ending the call or contacting family or relatives.

[1228] Example 2

[1229] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1230] In recent years, there has been an increase in fraudulent and suspicious phone calls, with elderly people and those in socially vulnerable positions often falling victim. In such situations, there is a need for a system that allows family members and relatives to respond quickly. Furthermore, more appropriate responses can be made by providing notifications that take into account the user's emotional state. However, current systems only use voice recognition and keyword detection, and lack comprehensive support including emotion analysis.

[1231] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family members and relatives through an information sharing service such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, a sentiment analysis means, and a means for including the sentiment analysis results in a notification. This enables family members and relatives to respond quickly and appropriately when the user receives a suspicious phone call.

[1232] "Speech recognition means" refers to technology that converts voice data into text data.

[1233] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning and intent.

[1234] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and extracts specific information.

[1235] "Means for responding to specific keywords" refers to technology that detects pre-set keywords from text data.

[1236] "Communication applications" refers to software for sending and receiving messages over the Internet, such as messenger apps and social networking sites.

[1237] "Information sharing service" refers to a platform for sharing information via the Internet.

[1238] "Sentiment analysis means" refers to technology that analyzes user emotions from voice data and text data.

[1239] The "means for including emotion analysis results in notifications" refers to a technique for adding emotion information obtained by emotion analysis means to notification messages.

[1240] "Consent and confidentiality of communications measures" refers to technologies that protect user privacy and meet legal requirements.

[1241] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only under specific conditions.

[1242] MODE FOR CARRYING OUT THE INVENTION

[1243] This invention is a system that sends a notification to family and relatives via a pre-registered communication application or information sharing service when a user utters a specific keyword during a phone call. The notification includes information about a suspicious call, the content of the call (keyword), the caller's phone number, and the user's emotional state.

[1244] Hardware and software used

[1245] Hardware: Smartphones, servers

[1246] Software: speech recognition engines (e.g., Google Speech-to-Text), sentiment analysis engines (e.g., IBM Watson Tone Analyzer), communication applications (e.g., messaging services), information sharing services (e.g., SNS)

[1247] System Overview

[1248] 1. Acquire audio data:

[1249] When a user receives a call, the device will record the call in real time. Specifically, the device's microphone will capture the audio data and temporarily store it in the internal storage.

[1250] 2. Speech Recognition:

[1251] The device sends the recorded voice data to a voice recognition engine (Google Speech-to-Text). The voice recognition engine converts the voice data into text data and sends it back to the device. Specifically, the voice data is sent via the Internet to the voice recognition engine's server, where it is analyzed.

[1252] 3. Keyword detection:

[1253] The server receives the text data sent from the device and detects specific keywords (e.g., settlement amount, accident, bank, transfer). The server analyzes the text data and checks whether the keywords are included. Specifically, the server parses the text data and compares it with a pre-set keyword list.

[1254] 4. Sentiment analysis:

[1255] The server sends the text and voice data to a sentiment analysis engine (IBM Watson Tone Analyzer). The sentiment analysis engine detects the user's emotions (e.g., anger, fear) and sends the results back to the server. Specifically, it analyzes emotions from the tone of the voice data and the content of the text data, and quantifies the type and intensity of the emotion.

[1256] 5. Generate and send notifications:

[1257] The server generates a notification message based on the results of keyword and emotion analysis. The server then sends the notification to family and relatives via pre-registered communication applications or information sharing services. Specifically, the notification message includes information about a suspicious call, its content (keywords), the caller's phone number, and the detected emotion.

[1258] Specific examples

[1259] When a user utters keywords such as "settlement money," "accident," "bank," or "transfer" over the phone, the device records this and sends it to a voice recognition engine. The server analyzes the text data and detects keywords. The server then sends the data to a sentiment analysis engine to analyze the user's emotions. Finally, the server generates a notification message and sends it to family and relatives via messaging services or social networking sites.

[1260] Prompt Sentence Examples

[1261] "When a user utters a specific keyword (e.g., settlement money, accident, bank, transfer) during a call, the device records it and sends it to a voice recognition engine. The server analyzes the text data and detects the keyword. The server then sends the data to a sentiment analysis engine to analyze the user's emotions. Finally, the server generates a notification message and sends it to family and relatives via a messaging service or social networking site."

[1262] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1263] Step 1:

[1264] Acquiring audio data

[1265] When a user receives a call, the device records the call in real time. Specifically, the device's microphone captures the audio data and temporarily stores it in the internal storage. The input is the user's call voice, and the output is the recorded audio data.

[1266] Step 2:

[1267] Voice Recognition

[1268] The device sends the recorded voice data to the voice recognition engine. The voice recognition engine converts the voice data into text data and sends it back to the device. Specifically, the voice data is sent via the Internet to the voice recognition engine's server, where it is analyzed. The input is the recorded voice data, and the output is text data.

[1269] Step 3:

[1270] Keyword detection

[1271] The server receives text data sent from the device and detects specific keywords. The server analyzes the text data to see if it contains any keywords. Specifically, the server parses the text data and compares it with a pre-defined keyword list. The input is the text data, and the output is a list of detected keywords.

[1272] Step 4:

[1273] sentiment analysis

[1274] The server sends the text data and voice data to the sentiment analysis engine. The sentiment analysis engine detects the user's emotions and returns the results to the server. Specifically, it analyzes emotions from the tone of the voice data and the content of the text data, and quantifies the type and intensity of the emotions. The input is text data and voice data, and the output is the sentiment analysis results.

[1275] Step 5:

[1276] Generate and send notifications

[1277] The server generates a notification message based on the results of the keyword and sentiment analysis. The server then sends the notification to family and relatives via a pre-registered communication application or information sharing service. Specifically, the notification message contains information about a suspicious call, its content (keywords), the caller's phone number, and the detected emotion. The input is a list of keywords and the results of sentiment analysis, and the output is a notification message.

[1278] (Application example 2)

[1279] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1280] In recent years, the number of fraudulent and suspicious phone calls has been increasing, with elderly people and those less well-informed becoming victims in particular. In such situations, there is a need for a system that allows family members and relatives to respond quickly. In addition, it is necessary to provide more accurate notifications by not only detecting keywords but also taking into account the user's emotional state.

[1281] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1282] In this invention, the server includes a speech recognition unit, a conversation content analysis unit using natural language processing technology, a unit that responds to specific keywords, a unit that sends the response results to family and relatives via a pre-registered SNS such as a messenger app, a unit that limits activation to calls from numbers not registered in the phone book or from anonymous numbers, a unit that handles the person's consent and confidentiality of communications, a sentiment analysis unit, and a unit that includes the sentiment analysis results in a notification. This makes it possible to quickly send a notification containing keywords and sentiment information to family and relatives when a suspicious call is received.

[1283] "Speech recognition means" is a technology for converting speech into text.

[1284] "Natural language processing technology" is a technology that allows computers to process and understand human language.

[1285] "Conversation content analysis means" is a technology for analyzing text obtained by speech recognition and understanding its content.

[1286] The "means for responding to a specific keyword" is a technology for detecting a preset keyword and performing a specific action based on it.

[1287] A "messenger app" is an application for sending and receiving messages over the Internet.

[1288] "SNS" is an abbreviation for social networking service, an online platform for users to share information with each other.

[1289] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology for activating the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[1290] "Measures to address the need for consent and confidentiality of communications" refers to technologies to protect the privacy of users and ensure the confidentiality of communications.

[1291] "Sentiment analysis means" is a technology for analyzing user emotions from voice and text.

[1292] The "means for including the result of sentiment analysis in the notification" is a technique for including the result obtained by sentiment analysis in the notification message.

[1293] A system for implementing this invention includes a voice recognition means, natural language processing technology, a conversation content analysis means, a means for responding to specific keywords, a means for sending notifications via a messenger app or SNS, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to consent from the individual and confidentiality of communications, a sentiment analysis means, and a means for including the results of the sentiment analysis in notifications.

[1294] Hardware and software used

[1295] Hardware: Smartphone (microphone, internet connection)

[1296] software:

[1297] Google Speech Recognition API: Used to convert speech to text.

[1298] IBM Watson Tone Analyzer: Used to analyze sentiment from text.

[1299] Requests library: Used to send notifications to messenger apps and social media.

[1300] System Operation

[1301] 1. Speech Recognition: When a user receives a call, the smartphone's microphone captures the voice, which is then converted into text using the Google Speech Recognition API.

[1302] 2. Keyword detection: Detect specific keywords (e.g., "settlement money," "accident," "bank," "transfer") from the converted text.

[1303] 3. Sentiment Analysis: Analyze the sentiment of the text using IBM Watson Tone Analyzer, which detects emotions such as anger or fear from the user's tone of voice and speaking style.

[1304] 4. Notification sending: Based on the detected keywords and emotional information, a notification message is created and sent to pre-registered family and relatives via messenger apps or social media.

[1305] Specific examples

[1306] If a user calls and says, "I've had an accident and would like a settlement payment transferred to me," the system will detect the keywords "accident," "settlement payment," and "transfer." It will then use sentiment analysis to detect "fear." A notification message containing this information will be sent to family and relatives.

[1307] Prompt Sentence Examples

[1308] If a user calls and says, "I was in an accident and would like the settlement money transferred," the application detects the keywords "accident," "settlement money," and "transfer," and then uses sentiment analysis to detect "fear." It then notifies the family of this information.

[1309] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1310] Step 1:

[1311] When a user receives a call, the microphone on the device (smartphone) captures the voice. The input is the user's voice, and the output is the captured voice data. This voice data is used in subsequent processing steps.

[1312] Step 2:

[1313] The device sends the captured voice data to the Google Speech Recognition API, which converts the voice into text. The input is voice data, and the output is text data. This text data is the basis for analyzing the conversation content.

[1314] Step 3:

[1315] The terminal detects specific keywords from the converted text data. The input is text data, and the output is a list of detected keywords. Specifically, it checks whether the text data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[1316] Step 4:

[1317] The device sends text data to the IBM Watson Tone Analyzer for sentiment analysis. The input is text data, and the output is the sentiment analysis results. Specifically, the device analyzes the user's emotions (e.g., anger, fear, etc.) from the text data.

[1318] Step 5:

[1319] The device creates a notification message based on the detected keywords and the results of sentiment analysis. The input is a list of keywords and the results of sentiment analysis, and the output is a notification message. Specific behavior is to include content such as "There was a suspicious call. Keyword: [Detected keyword]. Emotion: [Sentiment analysis result]" in the notification message.

[1320] Step 6:

[1321] The device sends the created notification message to pre-registered family and relatives via a messenger app or SNS. The input is the notification message, and the output is the sent notification. The specific operation is to send the notification message using the API of the messenger app or SNS.

[1322] Step 7:

[1323] The server checks the status of the sent notification and logs the success or failure result. The input is the notification sending result and the output is the log data. The specific operation is to check whether the notification was sent successfully and record the result.

[1324] Example 3

[1325] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1326] Conventional telephone systems have difficulty responding appropriately to calls from numbers not registered in the phone book or from anonymous numbers, and lack a means to quickly respond to situations in which users feel anxious or in danger. Furthermore, the lack of functionality to analyze the user's emotions and notify those around them of dangerous situations makes it difficult to ensure the user's safety.

[1327] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1328] In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through a pre-registered information transmission means such as a communication application, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, an emotion analysis means for analyzing the user's emotions, and a means for sending a notification when the user's emotions exceed a specific threshold. This makes it possible to quickly notify people around the user when they are in a dangerous situation.

[1329] "Speech recognition means" is a technology that analyzes voice data and converts it into text data.

[1330] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1331] The "conversation content analysis means" is a technology that analyzes the text data obtained by the voice recognition means and understands the content of the conversation.

[1332] "Means for responding to specific keywords" refers to a technique for taking specific actions based on pre-set keywords.

[1333] "Information transmission means" refers to the technology used to transmit information to other devices or users through communication applications.

[1334] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[1335] "Measures to ensure the consent of the individual and the confidentiality of communications" refers to technologies that enable the system with the user's consent and protect the confidentiality of communications.

[1336] "Emotion analysis means" is a technology that analyzes emotions from a user's voice and text data and determines their emotional state.

[1337] The "means for sending a notification" is a technique for sending a notification to a pre-defined recipient when a specific condition is met.

[1338] MODE FOR CARRYING OUT THE INVENTION

[1339] This invention is a system that is activated only for calls from numbers not registered in the phone book or from anonymous numbers, analyzes the user's emotions, and sends notifications to family and relatives when the emotions exceed a certain threshold. A specific embodiment of this system is described below.

[1340] Hardware and software used

[1341] Hardware: Smartphones, servers

[1342] Software: speech recognition engines, natural language processing engines, sentiment analysis engines, communication applications (e.g. messaging services)

[1343] System Overview

[1344] 1. When a call comes in, the terminal obtains the incoming call information.

[1345] 2. The terminal checks whether the incoming number is registered in the phone book or is a blocked number.

[1346] 3. The terminal sends the incoming call information to the server.

[1347] 4. The server starts the emotion analysis engine based on the received incoming call information.

[1348] 5. The server uses an emotion analysis engine to analyze emotions from the user's voice and text.

[1349] 6. If the user's emotions exceed a certain threshold, the server will send a notification to family and relatives via a pre-registered communication application.

[1350] Specific examples

[1351] Example 1: A user receives an anonymous call. The device detects the call and sends the call information to the server. The server activates an emotion analysis engine and analyzes the emotion from the user's voice. If the server determines that the user is very angry, it sends a notification to the family via LINE saying, "The user is very angry."

[1352] Example 2: A user receives a call from a number that is not registered in the phone book. The device detects the call and sends the call information to a server. The server activates an emotion analysis engine and analyzes the emotion in the user's voice. If the server determines that the user is very scared, it sends a notification to relatives via a messaging service saying, "The user is very scared."

[1353] Prompt Sentence Examples

[1354] Prompt 1: "When an anonymous call comes in, analyze the user's emotions and notify their family via LINE if they are angry."

[1355] Prompt 2: "If the user is scared when they receive a call from a number not in their phone book, notify a relative via messaging service."

[1356] This system makes it possible to quickly notify people around the user when the user is in a dangerous situation. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1357] Step 1:

[1358] When a call comes in, the terminal obtains incoming call information. As input, it receives the incoming number and time of the call. The terminal checks whether the incoming number is not registered in the phone book or is an anonymous number. As output, it generates incoming call information (incoming number, time of the call, whether the call is registered in the phone book). Specifically, the terminal displays "You have received a call" and checks the incoming number against the phone book.

[1359] Step 2:

[1360] The terminal sends the incoming call information to the server. As input, it receives the incoming call information generated in step 1. The terminal sends the incoming call information to the server via the network. As output, it generates a status indicating that transmission is complete. Specifically, the terminal displays "Sending incoming call information to server," and when transmission is complete, it displays "Transmission completed."

[1361] Step 3:

[1362] The server starts the emotion analysis engine based on the received incoming call information. As input, it receives the incoming call information sent from the terminal. The server starts the emotion analysis engine and prepares to acquire the user's voice data. As output, it generates the startup status of the emotion analysis engine. As a specific operation, the server records "Starting emotion analysis engine" in the log.

[1363] Step 4:

[1364] The server uses an emotion analysis engine to analyze emotions from the user's voice and text. As input, it receives the user's voice data. The server inputs the voice data into the emotion analysis engine and analyzes the emotional state. As output, it generates emotion analysis results. Specifically, the server records "emotion analysis in progress" in the log and obtains the analysis results.

[1365] Step 5:

[1366] If the user's emotion exceeds a certain threshold, the server sends a notification to family and relatives through a pre-registered communication application. The server receives the emotion analysis results as input. The server generates notification content based on the emotion analysis results and sends the notification through the communication application. The server generates a notification sending status as output. Specifically, the server logs "Generating notification content" and displays "Notification sent."

[1367] In this way, it is possible for the user to quickly notify those around him when he is in a dangerous situation.

[1368] (Application example 3)

[1369] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1370] Conventional telephone systems have difficulty in responding appropriately to calls from numbers not registered in the phone book or from anonymous numbers, which can put users in dangerous situations. Furthermore, they lack the means to monitor users' emotions in real time and quickly notify family and relatives in case of an emergency. Therefore, an effective system to ensure user safety is needed.

[1371] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, a means for monitoring the user's emotions in real time using an emotion engine, and a means for automatically notifying family and relatives when the user's emotions exceed a specific threshold. This makes it possible to quickly notify people around the user even when the user is in a dangerous situation.

[1372] "Speech recognition means" is a technology that converts speech into text data.

[1373] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1374] The "conversation content analysis means" is a technology for analyzing the text data obtained by the voice recognition means and understanding the content of the conversation.

[1375] "Means for responding to specific keywords" refers to a technique for executing specific actions based on pre-set keywords.

[1376] "Means of sending reaction results to family and relatives via pre-registered messenger apps or other social networking sites" refers to a technology that sends notifications to family and relatives via pre-registered messenger apps or social networking sites when certain conditions are met.

[1377] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[1378] "Measures to ensure the consent of the individual and the confidentiality of communications" refers to technologies that enable the system with the user's consent and protect the confidentiality of communications.

[1379] "Means for monitoring user emotions in real time using an emotion engine" is a technology for detecting and monitoring user emotions in real time.

[1380] "Means for automatically notifying family and relatives when a user's emotions exceed a certain threshold" refers to technology that automatically sends notifications to family and relatives when a user's emotions exceed a set threshold.

[1381] The system for implementing the present invention uses the following hardware and software.

[1382] Hardware

[1383] Smartphone

[1384] software

[1385] EmotionEngine: A library for detecting user emotions in real time

[1386] PhoneCallDetector: A library for detecting calls from numbers not registered in the phonebook or from anonymous numbers.

[1387] smtplib: Python's standard library for sending email

[1388] System Operation

[1389] 1. Voice recognition means:

[1390] Voice recognition software installed on the smartphone converts voice into text data when a call comes in.

[1391] 2. Natural language processing technology:

[1392] The converted text data is analyzed and natural language processing technology is used to understand the content of the conversation.

[1393] 3. Ways to respond to specific keywords:

[1394] Specific keywords (e.g., "settlement money," "accident," "bank," and "transfer") are detected from the analyzed text data, and specific actions are taken based on these.

[1395] 4. How to send your test results to family and relatives via pre-registered social media apps such as messenger apps:

[1396] If certain conditions are met, notifications will be sent to family and relatives via pre-registered messenger apps or social media.

[1397] 5. How to limit activation to calls from numbers not registered in your phone book or from anonymous numbers:

[1398] The system will only be activated for calls from numbers not registered in the phone book or from anonymous numbers.

[1399] 6. Consent of the individual, means of dealing with confidentiality of communications:

[1400] The system is enabled with the user's consent and protects the confidentiality of communications.

[1401] 7. Real-time monitoring of user sentiment using an emotion engine:

[1402] EmotionEngine is used to detect and monitor user emotions in real time.

[1403] 8. A way to automatically notify family and relatives if a user's emotions exceed a certain threshold:

[1404] If a user's emotions exceed a set threshold, notifications are automatically sent to family and relatives.

[1405] Specific examples

[1406] For example, if a user becomes very angry when they receive a call from a number that is not registered in their phone book, an email will be sent to their family saying, "The user is very angry. Please check."

[1407] Prompt Sentence Examples

[1408] Create a Python program that will detect the user's emotions when they receive a call from a number not registered in their phone book using an emotion engine, and send an emergency notification email to their family if their anger or fear exceeds a threshold.

[1409] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1410] Step 1:

[1411] The device detects an incoming call. The input is the incoming call signal, and the output is a flag indicating that an incoming call has occurred. The device uses PhoneCallDetector to detect calls from numbers not registered in the phone book or from anonymous numbers.

[1412] Step 2:

[1413] The terminal uses a voice recognition means to convert the incoming voice into text data. The input is the incoming voice data and the output is text data. The terminal uses voice recognition software to convert the voice into text in real time.

[1414] Step 3:

[1415] The device uses natural language processing technology to analyze text data and understand the content of the conversation. The input is text data and the output is the analysis results. The device uses a natural language processing library to analyze the text data and extract specific keywords.

[1416] Step 4:

[1417] The terminal detects specific keywords from the analysis results using a means to respond to specific keywords. The input is the analysis results, and the output is the keyword detection results. The terminal determines an action based on the specific keywords (e.g., "settlement money," "accident," "bank," "transfer").

[1418] Step 5:

[1419] The device uses an emotion engine to monitor the user's emotions in real time. The input is the user's voice data and text data, and the output is the emotional state. The device uses the Emotion Engine to detect the user's emotions and monitor them in real time.

[1420] Step 6:

[1421] The device automatically notifies family and relatives when the user's emotions exceed a certain threshold. The input is the emotional state, and the output is a notification message. If the device determines that the user is very angry or very scared based on the output of the emotion engine, it will send a notification to family and relatives via a pre-registered messenger app or social media.

[1422] Step 7:

[1423] The terminal activates the system using the user's consent and measures for protecting the confidentiality of communications, and protects the confidentiality of communications. The input is the user's consent information, and the output is the system's activation status. The terminal confirms the user's consent and activates the system, thereby protecting the confidentiality of communications.

[1424] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1425] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1426] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1427] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1428] [Third embodiment]

[1429] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1430] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1431] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1432] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1433] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1434] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1435] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1436] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1437] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1438] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1439] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1440] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1441] "Example 1"

[1442] An embodiment of the present invention is realized as an application installed on a terminal such as a smartphone or tablet. This application captures speech through the terminal's microphone and converts it into text using speech recognition technology. The converted text is analyzed using natural language processing technology to determine whether it contains specific keywords. Keywords that may indicate fraud or suspicious phone calls include "settlement money," "accident," "bank," and "transfer."

[1443] "Example 2"

[1444] If a keyword is detected, the application will send a notification to family and relatives via pre-registered messaging apps or social media. The notification will include information about the suspicious call, its content (keywords), and the caller's phone number, allowing family and relatives to respond quickly.

[1445] "Example 3"

[1446] In addition, this embodiment of the present invention can be set to activate only for calls from numbers not registered in the phone book or from anonymous numbers. This prevents false activation for calls from acquaintances. Furthermore, the function is activated with the consent of the individual, taking legal considerations into account for the confidentiality of communications.

[1447] The processing flow of each embodiment will be described below.

[1448] "Example 1"

[1449] Step 1: A user installs the application of the present invention on a device such as a smartphone or tablet.

[1450] Step 2: The application captures audio through the device's microphone.

[1451] Step 3: The captured audio is converted into text using speech recognition technology.

[1452] Step 4: The converted text is analyzed using natural language processing techniques to determine whether it contains specific keywords.

[1453] "Example 2"

[1454] Step 1: If a specific keyword is detected in the text, the application will send a notification to family and relatives via pre-registered messenger apps or social media.

[1455] Step 2: The notification will include information about the suspicious call, its content (keywords), and the caller's phone number.

[1456] "Example 3"

[1457] Step 1: The application of the present invention can be set to launch only when receiving calls from numbers not registered in the phone book or from numbers with no identification.

[1458] Step 2: Enable the feature with the individual's consent, while also taking legal considerations into account the confidentiality of communications.

[1459] Example 1

[1460] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1461] In recent years, fraudulent and suspicious phone calls have been increasing, with many elderly people and those unfamiliar with technology falling victim to these attacks. There is a need for systems that can detect these fraudulent calls in advance and warn users. However, existing systems have low accuracy in speech recognition and natural language processing, making it difficult to respond in real time. Furthermore, there is a lack of effective methods for detecting fraudulent calls while protecting user privacy.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1463] In this invention, the server includes a speech recognition unit, a conversation content analysis unit that utilizes natural language processing technology, and a unit that responds to specific keywords, which enables fraud and suspicious phone calls to be detected in real time and a warning given to the user.

[1464] "Speech recognition means" refers to technology that converts voice data into text data.

[1465] "Natural language processing technology" refers to technology for analyzing text data and understanding its meaning and context.

[1466] The "conversation content analysis means" refers to a means for analyzing text data converted by the speech recognition means using natural language processing technology.

[1467] "Means that react to specific keywords" refers to means that detect specific words or phrases contained in text data and take action based on them.

[1468] "Information sharing means" refers to the means for notifying other users or systems of analysis results and warnings.

[1469] "Means to limit activation to calls from numbers not registered in your contacts or from anonymous numbers" refers to means to activate the system only under certain conditions.

[1470] "Consent of the individual, measures to address confidentiality of communications" refers to measures to protect user privacy and meet legal requirements.

[1471] "Means for capturing audio through a microphone of a terminal and processing audio data in real time" refers to means for collecting audio using a microphone of a terminal and processing the audio data immediately.

[1472] "Means for converting voice data into text" refers to means for converting voice data into character data using voice recognition technology.

[1473] "Means for tokenizing the converted text and tagging parts of speech" refers to means for dividing text data into words and phrases and assigning part-of-speech information to each.

[1474] "Means for notifying the user when a keyword is detected" refers to means for issuing a warning or notification to the user when a specific keyword is found in text data.

[1475] This invention is realized as an application installed on devices such as smartphones and tablets. This application is a system that combines voice recognition technology and natural language processing technology to detect fraudulent and suspicious phone calls in real time and warn users.

[1476] When a user launches the application, speech is captured through the device's microphone. The captured speech is converted into text using speech recognition technology such as the Google Cloud Speech-to-Text API. The converted text is then analyzed using Python natural language processing libraries such as NLTK and spaCy. Specifically, the text is tokenized and each word is tagged with its part of speech.

[1477] The device then checks whether the analyzed text contains certain keywords (such as "settlement money," "accident," "bank," or "transfer"). If a keyword is detected, the device records the information and notifies the user, via a pop-up message or audio alert.

[1478] For example, if a user receives a call that begins with, "Hello, this is your bank and we'd like to discuss a transfer," the application can capture this speech, convert it to text, analyze the text, and detect if it contains keywords like "bank" or "transfer," thereby alerting the user to a potential scam.

[1479] Example prompt sentence:

[1480] "This application captures voice through the device's microphone and converts it into text using speech recognition technology. The converted text is then analyzed using natural language processing technology to determine whether it contains specific keywords. Keywords include 'settlement money,' 'accident,' 'bank,' and 'transfer.'"

[1481] This system includes a speech recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family and relatives through information sharing means such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the contacts or from blocked numbers, a means for ensuring the individual's consent and confidentiality of communications, a means for capturing voice through the device's microphone and processing the voice data in real time, a means for converting the voice data into text, a means for tokenizing the converted text and tagging it with parts of speech, and a means for notifying the user when a keyword is detected. This makes it possible to detect fraud and suspicious phone calls in real time and warn the user.

[1482] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1483] Step 1:

[1484] A user launches an application on their smartphone or tablet.

[1485] Input: The application is started by a user action.

[1486] Output: Application initialization complete.

[1487] Specific behavior: A user launches an application by tapping its icon on the device's home screen. The application initializes the necessary libraries and APIs for speech recognition and natural language processing (e.g., Google Cloud Speech-to-Text API, Python's NLTK library).

[1488] Step 2:

[1489] The device captures the user's voice in real time through a microphone.

[1490] Input: User's voice.

[1491] Output: Audio data.

[1492] Specific operation: When the user answers the phone and says, "Hello, this is from the bank. I'd like to talk to you about a transfer," the voice is picked up by the device's microphone. The voice data is temporarily stored in the device's memory.

[1493] Step 3:

[1494] The device sends the captured voice data to the Google Cloud Speech-to-Text API.

[1495] Input: Audio data.

[1496] Output: Text data.

[1497] How it works: The device sends the captured voice data to the Google Cloud Speech-to-Text API, which converts the voice data into text, which is then returned to the device as "Hello, this is from your bank. We'd like to discuss a transfer."

[1498] Step 4:

[1499] The text received by the device is analyzed using a Python natural language processing library (NLTK or spaCy).

[1500] Input: Text data.

[1501] Output: Tokenized text and part-of-speech tags.

[1502] Specific operation: The device divides the text it receives into tokens such as "hello," "this is," "from the bank," "but," "transfer," "about," and "we'd like to talk to you about," and tags each word with its part of speech.

[1503] Step 5:

[1504] The device checks whether certain keywords are present in the analyzed text.

[1505] Input: Tokenized text and part-of-speech tags.

[1506] Output: Keyword detection results.

[1507] What it does: The device checks whether the text contains keywords such as "bank" or "transfer." If a keyword is detected, it records the information.

[1508] Step 6:

[1509] The terminal notifies the user that the keyword has been detected.

[1510] Input: Keyword detection results.

[1511] Output: Notification to the user.

[1512] What it does: The device will display a pop-up message saying "Possible fraud. Please be careful." It can also issue a voice alert saying "Possible fraud."

[1513] (Application example 1)

[1514] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1515] In recent years, fraudulent and suspicious phone calls have been increasing, with elderly people and those less well-informed becoming victims in particular. Conventional countermeasures are insufficient, and there is a need for a system that can detect possible fraud in real time and issue a warning to users. There is also a need for a method to automatically detect potentially fraudulent calls and prompt users to take appropriate action.

[1516] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1517] In this invention, the server includes a speech recognition means, a conversation content analysis means using natural language processing technology, and a means for responding to specific keywords, which makes it possible to issue a warning to the user when a keyword that may be fraudulent is detected and provide the user with the option of whether or not to end the call.

[1518] "Speech recognition means" refers to a device or software that converts the contents of a call into text information in real time using speech-to-text technology.

[1519] "Natural language processing technology" is a technology for analyzing text data and understanding meaning and emotions, and is used to analyze conversation content.

[1520] The "conversation content analysis means" is a device or software that utilizes natural language processing technology to analyze the text data converted by the speech recognition means and detect specific keywords and phrases.

[1521] A "means for responding to specific keywords" is a device or software that detects pre-set keywords (e.g., "settlement money," "accident," "bank," "transfer," etc.) from text data and takes action based on the results.

[1522] A "messenger app" is application software that sends and receives messages over the Internet and is used to send notifications to family and relatives.

[1523] "SNS" stands for Social Networking Service, an online platform for users to share information and communicate.

[1524] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to a device or software that activates the system only when an incoming call is received from a number not registered in the user's phone book or from an anonymous number.

[1525] "Means for obtaining consent and ensuring confidentiality of communications" refers to devices or software that obtain a user's consent when using the system in order to protect the user's privacy and ensure the confidentiality of communications.

[1526] "Means for alerting the user when a potentially fraudulent keyword is detected and offering the option of ending the call" refers to a device or software that displays a warning to the user when a specific keyword is detected and offers the user the option of continuing or ending the call.

[1527] As an embodiment of the present invention, a method for installing a fraudulent call detection system on a smartphone is described. The system includes a voice recognition unit, a conversation content analysis unit using natural language processing technology, a reaction unit for specific keywords, and a unit for issuing a warning to the user and providing the user with the option of ending the call.

[1528] System Program

[1529] The system program operates as follows.

[1530] 1. Voice recognition means:

[1531] The system captures the conversation in real time through the smartphone's microphone and converts it into text using speech recognition technology. Specifically, it uses a speech recognition library and Google's speech recognition API.

[1532] 2. Natural language processing technology:

[1533] The transformed text is analyzed using natural language processing techniques, specifically using the transformers library pipeline to perform sentiment analysis on the text.

[1534] 3. Ways to respond to specific keywords:

[1535] The text data is checked for pre-set keywords (e.g., "settlement money," "accident," "bank," "transfer," etc.) to determine whether the relevant keywords are included.

[1536] 4. By providing the user with a warning and the option to end the call:

[1537] If a keyword is detected, the user will be alerted and given the option to continue or end the call.

[1538] Hardware and Software

[1539] Hardware:

[1540] Smartphone (microphone, speaker, display)

[1541] software:

[1542] Speech Recognition Library

[1543] Google Speech Recognition API

[1544] transformers library

[1545] Specific examples

[1546] For example, if a user receives a call and the caller says, "Please transfer the settlement money," the system will detect the keywords "settlement money" and "transfer." In this case, the system will display a warning to the user saying, "This may be a scam. Do you want to end the call?" and ask the user to choose whether to continue or end the call.

[1547] Prompt Sentence Examples

[1548] If a user receives a phone call and the caller says, "Please transfer the settlement money," the application will detect the keywords "settlement money" and "transfer" and alert the user.

[1549] In this way, the fraudulent call detection system can ensure user safety and prevent fraudulent incidents.

[1550] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1551] Step 1:

[1552] The device captures the call content in real time through the smartphone's microphone. The input is voice data, and the output is raw voice data that is passed to the voice recognition means. Specifically, when the user starts a call, the microphone starts capturing the voice.

[1553] Step 2:

[1554] The device converts the captured voice data into text using a voice recognition means. The input is the voice data acquired in step 1, and the output is text data. Specifically, it uses a voice recognition library and calls Google's voice recognition API to convert the voice into text.

[1555] Step 3:

[1556] The device uses natural language processing technology to analyze the converted text data. The input is the text data generated in step 2, and the output is the analysis results on the sentiment and meaning of the text. Specifically, it uses a pipeline from the transformers library to perform sentiment analysis of the text.

[1557] Step 4:

[1558] The terminal checks the text data for pre-defined keywords using a means that responds to specific keywords. The input is the text data obtained in step 3, and the output is a flag indicating whether the keyword was detected. Specifically, it checks whether the text data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[1559] Step 5:

[1560] If a keyword is detected, the terminal will issue a warning to the user and offer the option of whether to end the call. The input is the keyword detection flag obtained in step 4, and the output is the user's choice. Specifically, the terminal will display a warning message on the display and offer the user the option of "Possible fraud. Do you want to end the call?"

[1561] Step 6:

[1562] The user confirms the warning message and chooses whether to continue or end the call. The input is the warning message displayed in step 5, and the output is the user's choice. As a specific operation, if the user selects "End," the call is automatically ended.

[1563] In this way, the fraudulent call detection system can analyze the content of calls in real time, warn users if there is a possibility of fraud, and prompt them to take appropriate action.

[1564] Example 2

[1565] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1566] In modern society, there has been an increase in fraudulent and suspicious phone calls, with elderly people and those unfamiliar with technology being particularly vulnerable to these attacks. To combat this, there is a need for systems that can quickly notify family and relatives and encourage them to take appropriate action. However, existing systems lack the ability to recognize voices, detect keywords, and send notifications in real time, making it difficult to implement effective countermeasures.

[1567] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1568] In this invention, the server includes a voice recognition unit, a conversation content analysis unit using natural language processing technology, a unit that responds to specific keywords, a unit that sends the response results to family and relatives through an information sharing service such as a pre-registered communication application, a unit that limits activation to calls from numbers not registered in the phone book or from anonymous numbers, a unit that handles the individual's consent and confidentiality of communications, a unit that converts voice data into text data, a unit that compares the text data with a keyword list, a unit that creates a notification when a keyword is detected, and a unit that sends the notification in real time. This enables a user to quickly and reliably notify family and relatives when they receive a suspicious phone call and urge them to take appropriate action.

[1569] "Speech recognition means" is a technology for converting voice data into text data.

[1570] "Means for analyzing conversation content using natural language processing technology" is a technology for analyzing text data and understanding meaning and intent.

[1571] "Means for responding to specific keywords" refers to technology that compares text data with a pre-registered keyword list and detects matching keywords.

[1572] "Information sharing services such as communication applications" refers to services for sharing information over the Internet, such as messenger apps and social networking sites.

[1573] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology for controlling the operation of the system based on specific conditions.

[1574] "Measures to address the need for consent and confidentiality of communications" refers to techniques to protect user privacy and meet legal requirements.

[1575] The "means for converting voice data into text data" is a technology for converting voice data into text information using voice recognition technology.

[1576] The "means for checking text data against a keyword list" is a technique for comparing the converted text data with a pre-set keyword list to detect matching keywords.

[1577] The "means for generating a notification when a keyword is detected" refers to a technique for generating a notification message when a particular keyword is detected.

[1578] The "means for sending notifications in real time" refers to a technique for instantly sending generated notification messages to family and relatives.

[1579] This invention is a system for quickly and reliably notifying family and relatives when a user receives a suspicious phone call, and encouraging them to take an appropriate action. This system includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through an information sharing service such as a pre-registered communication application, a means for limiting activation to calls from numbers not registered in the phone book or from numbers withheld, a means for ensuring the consent of the user and confidentiality of communications, a means for converting voice data into text data, a means for comparing the text data with a keyword list, a means for creating a notification when a keyword is detected, and a means for transmitting the notification in real time.

[1580] The server monitors users' phone calls in real time and converts them into text using the Google Cloud Speech-to-Text API, a voice recognition software. The converted text is then compared with a pre-registered list of keywords.

[1581] The device is the user's smartphone or PC, and is responsible for sending the contents of the call to the server. When the call ends, the device sends the text data of the call to the server.

[1582] The server analyzes the received text data to see if it contains any keywords, and if it does, it uses a communication service API to send notifications to family and relatives via pre-registered messenger apps or social media.

[1583] The notification will include information about the suspicious call, its contents (keywords), the caller's phone number, etc. This will enable family members and relatives to respond quickly.

[1584] As a concrete example, consider the case where a user receives a phone call containing the keyword "scam." In this case, the server detects the keyword "scam" and uses the communication service API to send the following notification to the family:

[1585] Example of notification content:

[1586] There was a suspicious phone call.

[1587] Content: Fraud

[1588] Caller's phone number: +81-90-XXXX-XXXX

[1589] Example prompts to input to the generative AI model:

[1590] Create a program that will send a notification to a family member if the user receives a phone call containing the keyword "scam." The notification should include information about the suspicious call, its content (keywords), and the caller's phone number.

[1591] In this way, the system can detect suspicious calls in real time and quickly notify family and relatives to keep users safe.

[1592] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1593] Step 1:

[1594] The user receives a call.

[1595] When a user receives a call on their smartphone, the call starts. The call start event triggers the system to start working. The input is the incoming call, and the output is the call start event.

[1596] Step 2:

[1597] The device records the call.

[1598] The device records the contents of the call in real time. The recording is done using the smartphone's built-in microphone. The recorded data is temporarily stored in the device. The input is the call audio, and the output is the recorded audio data.

[1599] Step 3:

[1600] The device sends the recorded audio data to the server.

[1601] When the call ends, the device sends the recorded voice data to the server. The transmission is performed using the HTTPS protocol, and the data is encrypted. The input is the recorded voice data, and the output is the transmission of the voice data to the server.

[1602] Step 4:

[1603] The server converts the audio data into text.

[1604] The server converts the received voice data into text using the Google Cloud Speech-to-Text API. The converted text data is stored on the server. The input is voice data, and the output is text data.

[1605] Step 5:

[1606] The server matches the text data with a list of keywords.

[1607] The server compares the converted text data with a pre-registered keyword list, which includes words such as "fraud," "suspicious," and "danger." The input is the text data and the keyword list, and the output is the keyword detection results.

[1608] Step 6:

[1609] If the server detects a keyword, it creates a notification.

[1610] If the server detects a keyword, it creates a notification. The notification contains information about a suspicious call, its content (keyword), and the caller's phone number. The input is the keyword detection result, and the output is the notification message.

[1611] Step 7:

[1612] The server sends notifications to family and relatives.

[1613] The server uses the communication service API to send notifications to family and relatives via pre-registered messenger apps and SNS. The notifications are sent in real time, allowing family and relatives to respond quickly. The input is the notification message, and the output is the notification sent to family and relatives.

[1614] (Application example 2)

[1615] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1616] In recent years, there has been an increase in telephone fraud and suspicious phone calls, with elderly people and those less well-informed becoming victims in particular. In order to respond quickly to such suspicious calls, a system that can immediately send notifications to family and relatives is needed. However, current systems lack the functionality to analyze call content in real time, detect specific keywords, and send notifications. Furthermore, there is a need to provide more detailed information by including recordings of calls in the notification content.

[1617] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family members and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the person's consent and confidentiality of communications, a means for sending a notification to family members and relatives based on the detected keywords, and a means for including in the notification the content of the suspicious call (keywords), the caller's phone number, and recorded data of the call. This makes it possible to quickly provide detailed information to family members and relatives when a suspicious call is received.

[1618] "Speech recognition means" refers to technology that converts speech into text data.

[1619] "Natural language processing technology" refers to the technology of processing and analyzing human language using a computer.

[1620] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and extracts specific information.

[1621] "Means for responding to specific keywords" refers to technology that detects pre-set keywords and operates based on the detection results.

[1622] A "messenger app" refers to an application for sending and receiving messages over the Internet.

[1623] "SNS" is an abbreviation for social networking service, and refers to an online platform where users can share information with each other.

[1624] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[1625] "Measures to ensure the individual's consent and confidentiality of communications" refers to technologies for obtaining the individual's consent when using the system and for protecting the confidentiality of communication content.

[1626] "Means for sending notifications to family and relatives based on detected keywords" refers to technology that sends notifications to family and relatives when specific keywords are detected.

[1627] "Means for the notification to include the content of the suspicious call (keywords), the caller's phone number, and a recording of the call" refers to a technology in which the notification sent includes the content of the suspicious call, the caller's phone number, and a recording of the call.

[1628] A system for implementing this invention includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for sending the response results to family and relatives via a pre-registered SNS such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for dealing with the consent of the individual and confidentiality of communications, a means for sending notifications to family and relatives based on detected keywords, and a means for the notification to include the content of the suspicious call (keywords), the caller's phone number, and recorded data of the call.

[1629] System Program

[1630] This system can be implemented using Python. It uses the Google Cloud Speech-to-Text API for speech recognition, the NLTK (Natural Language Toolkit) for natural language processing, and the smtplib library for sending emails.

[1631] Processing Description

[1632] The server first converts the contents of the call into text data using voice recognition. This text data is then analyzed using natural language processing technology to check whether it contains specific keywords. If a specific keyword is detected, the server will use the results to send a notification to family and relatives via pre-registered messenger apps or social media.

[1633] The notification will include the content of the suspicious call (detected keywords), the phone number from which it originated, and a recording of the call, allowing family members or relatives to take action quickly.

[1634] Hardware and software used

[1635] Hardware: Smartphones, servers

[1636] Software: Python, Google Cloud Speech-to-Text API, NLTK, smtplib

[1637] Specific examples

[1638] For example, if a user receives a call saying, "This is a fraudulent call," a speech recognition means converts this speech into text data. Next, natural language processing technology is used to analyze this text data and detect the keyword "fraud." Based on the detected keyword, the server sends a notification to family and relatives. This notification includes the following content: "There has been a suspicious call. The caller's phone number is: 090-XXXX-XXXX. The detected keyword is: fraud."

[1639] Prompt Sentence Examples

[1640] Create a program that checks the content of phone calls and sends a notification if certain keywords are included. The keyword list includes "fraud," "suspicious," "suspicious," and "transfer." The notification will include the content of the suspicious call (keywords), the caller's phone number, and a recording of the call. The notification will be sent by email.

[1641] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1642] Step 1:

[1643] The user receives a call. The voice data of the call is input to the terminal. The terminal collects this voice data in real time.

[1644] Step 2:

[1645] The device uses a speech recognition means to convert the collected voice data into text data. Specifically, it uses the Google Cloud Speech-to-Text API to analyze the voice data and generate corresponding text data. The input is voice data, and the output is text data.

[1646] Step 3:

[1647] The server analyzes the text data using natural language processing technology. Specifically, it uses NLTK (Natural Language Toolkit) to tokenize the text data and check whether it contains specific keywords. The input is the text data, and the output is a list of detected keywords.

[1648] Step 4:

[1649] If a specific keyword is detected, the server creates a notification based on the results. The notification includes the content of the suspicious call (detected keywords), the caller's phone number, and the call recording. The input is a list of detected keywords, the caller's phone number, and the call recording, and the output is a notification message.

[1650] Step 5:

[1651] The server sends the created notification message to family and relatives via pre-registered messenger apps or SNS. Specifically, it sends emails using the smtplib library. The input is the notification message, and the output is the sent notification.

[1652] Step 6:

[1653] Family members and relatives can check the received notification and respond promptly if necessary. The input is the notification message and the output is the response action.

[1654] The above processing steps enable the user to quickly provide detailed information to family and relatives when they receive a suspicious phone call.

[1655] Example 3

[1656] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1657] Conventional telephone answering systems were unable to respond appropriately to calls from numbers not registered in the phone book or from anonymous numbers, and they malfunctioned when receiving calls from acquaintances and lacked legal consideration for the confidentiality of communications. Furthermore, users had insufficient means to manage the system settings, making them inconvenient.

[1658] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a means for transmitting the response results to family and relatives through a pre-registered communication application such as a messenger app, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for responding to the user's consent and confidentiality of communications, a means for generating a response message using a generative AI model, a means for sending the generated response message to the caller, and a means for the user to manage system settings. This makes it possible to respond appropriately to calls from numbers not registered in the phone book or from anonymous numbers, prevent malfunctions when receiving calls from acquaintances, and ensure legal consideration for confidentiality of communications.

[1659] "Speech recognition means" refers to technology that converts speech into text data.

[1660] "Natural language processing technology" refers to technology that processes, understands, and generates human language using a computer.

[1661] "Conversation content analysis means" refers to a technology that analyzes text data obtained by speech recognition means and understands the content of the conversation.

[1662] "Means for responding to specific keywords" refers to a technology for performing specific actions based on pre-set keywords.

[1663] "Communication application" refers to software that sends and receives messages over the Internet, such as a messenger app.

[1664] "Means to limit activation to calls from numbers not registered in the phone book or from anonymous numbers" refers to technology that activates the system only for calls from numbers not registered in the phone book or from anonymous numbers.

[1665] "Means for addressing the individual's consent and confidentiality of communications" refers to technologies that enable systems with the user's consent and take legal considerations to protect the confidentiality of communications.

[1666] A "generative AI model" refers to a model that uses artificial intelligence to generate text data.

[1667] "Means for generating a response message" refers to technology that uses a generative AI model to create a response message for the caller.

[1668] "Means for sending a response message to the sender" refers to a technique for sending the generated response message to the sender.

[1669] "Means for users to manage system settings" refers to the interface through which users can change and manage the system's operating settings.

[1670] MODE FOR CARRYING OUT THE INVENTION

[1671] This invention is a system that activates only when receiving calls from numbers not registered in the phone book or from anonymous numbers, and uses a generative AI model to generate a response message and send it to the caller. A specific embodiment of this system is described below.

[1672] Hardware and software used

[1673] The server uses a communication API to monitor incoming calls. Specifically, it uses a communication service API to detect incoming calls in real time. The phone book database is managed using MySQL and is used to verify caller IDs. OpenAI's GPT-4 is used as the generative AI model. The interface for users to manage system settings is built using React.

[1674] Program processing

[1675] The server monitors incoming calls through the communication service API. When a call is received, the server checks the phone number against its phone book database. If the number is not registered in the phone book or is blocked, the server activates a generative AI model, sends a prompt, and generates a response message. The generated response message is sent to the caller through the communication service API.

[1676] Specific examples

[1677] If the user receives a call from a number not in their phone book, the server checks the number against the phone book database to ensure it is not registered. The server then sends the following prompt to the generative AI model:

[1678] Example prompt sentence:

[1679] "This number is not in the phone book. Please generate a response message."

[1680] The generative AI model generates a response message like this:

[1681] "Thank you for calling. This number is not in the phone book and cannot be answered. We apologize."

[1682] The server sends this message to the caller via the communication service API, allowing users to prevent accidental calls from acquaintances and to respect legal considerations regarding the confidentiality of communications.

[1683] Managing User Settings

[1684] Users can enable or disable this feature through a system settings screen, which is built using React and includes a checkbox for user consent. When a user changes the settings, the information is sent to the server and stored in a database.

[1685] In this way, the present invention can respond appropriately to calls from numbers not registered in the telephone directory or from anonymous numbers, prevent malfunctions in response to calls from acquaintances, and take legal considerations regarding the secrecy of communications. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1686] Program processing flow

[1687] Step 1: Detecting an incoming call

[1688] The server uses the Communication Services API to monitor incoming calls in real time, and when an incoming call comes in, the Communication Services API sends a notification to the server.

[1689] Input: Call notification (caller ID, call time, etc.)

[1690] Output: Incoming call details

[1691] Specific behavior:

[1692] The server receives a notification from the communication service API and retrieves the details of the incoming call (caller ID, time of call, etc.).

[1693] Step 2: Phone number verification

[1694] The server checks the incoming caller number against a phone book database, which is managed using MySQL.

[1695] Input: Caller ID

[1696] Output: Information on whether the contact is registered in the phone book

[1697] Specific behavior:

[1698] The server queries the MySQL database to see if the caller ID is registered.

[1699] Step 3: Launching the generative AI model

[1700] If the caller ID is not registered in the phone book, the server launches a generative AI model, which uses OpenAI's GPT-4.

[1701] Input: Information that the caller number is not registered in the phone book

[1702] Output: Launch of a generative AI model

[1703] Specific behavior:

[1704] The server sends a prompt to the generative AI model, requesting it to generate a response message.

[1705] Step 4: Generate a response message

[1706] The generative AI model generates a response message based on the prompt sentence.

[1707] Input: prompt statement

[1708] Output: Response message

[1709] Specific behavior:

[1710] The generative AI model receives the following prompt and generates a response message:

[1711] "This number is not in the phone book. Please generate a response message."

[1712] Example of the message generated:

[1713] "Thank you for calling. This number is not in the phone book and cannot be answered. We apologize."

[1714] Step 5: Send a response message

[1715] The server sends the generated response message to the caller via the communication service API.

[1716] Input: Response message

[1717] Output: Send message to caller

[1718] Specific behavior:

[1719] The server uses the communication services API to send the generated message to the caller as an SMS or voice message.

[1720] Step 6: Manage User Settings

[1721] Users can enable or disable this feature through a system settings screen, which is built using React.

[1722] Input: User setting change

[1723] Output: Update configuration information

[1724] Specific behavior:

[1725] The user can enable or disable the feature by checking the checkbox on the settings screen. The server saves this setting in the database and applies it the next time a call is received.

[1726] (Application example 3)

[1727] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1728] In recent years, telephone fraud and nuisance calls have been increasing, and there is a particular need to respond to calls from numbers not registered in the phone book or from anonymous numbers. However, conventional systems have malfunctioned when receiving calls from acquaintances and have insufficient legal consideration for the confidentiality of communications. In addition, there have been problems with users being unable to quickly recognize suspicious calls and take appropriate action.

[1729] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a voice recognition means, a conversation content analysis means that utilizes natural language processing technology, and a means that responds to specific keywords. This makes it possible to detect an incoming call from a number not registered in the phone book as a suspicious call and notify the user.

[1730] "Speech recognition means" is a technology that converts voice into a digital signal and analyzes its content.

[1731] "Means for analyzing conversation content using natural language processing technology" refers to technology that uses natural language processing technology to analyze the content of speech and text and understand their meaning.

[1732] "Means for responding to specific keywords" refers to a technology that responds to a preset keyword and performs a specific action.

[1733] "Means of communication" refers to the technology used to send and receive information through messenger apps, etc.

[1734] "Means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers" is a technology that activates the system only when there is an incoming call from a number not registered in the phone book or from anonymous numbers.

[1735] "Measures to address the need for consent and confidentiality of communications" refers to technologies that enable functions with the user's consent and protect the confidentiality of communications.

[1736] "Means for detecting suspicious calls and notifying the user" refers to technology that detects calls from numbers not registered in the phone book as suspicious calls and notifies the user of this information.

[1737] The system for carrying out this invention is mainly composed of a smartphone and includes a voice recognition means, a conversation content analysis means using natural language processing technology, a means for responding to specific keywords, a communication means, a means for limiting activation to calls from numbers not registered in the phone book or from anonymous numbers, a means for ensuring the consent of the person and confidentiality of communications, and a means for detecting suspicious calls and notifying the user.

[1738] Program processing explanation

[1739] Voice recognition means

[1740] When a call comes in, the application installed on the smartphone performs voice recognition. The voice recognition method converts the voice into a digital signal and analyzes its content. For this, it uses voice recognition technology such as Google Speech-to-Text API.

[1741] Conversation content analysis using natural language processing technology

[1742] The speech data acquired by the speech recognition means is converted into text using natural language processing technology. This text data is then further analyzed to understand the content of the conversation. For example, natural language processing libraries such as NLTK and spaCy are used here.

[1743] How to respond to specific keywords

[1744] The analyzed text data reacts to specific keywords. For example, if keywords such as "settlement money," "accident," "bank," and "transfer" are included, the system will react to them.

[1745] communication means

[1746] If a specific keyword is triggered, the system will send a notification to family and relatives via a messaging app or other communication method, using messaging service APIs and communication service APIs.

[1747] A method to limit activation to calls from numbers not registered in the phone book or from anonymous numbers

[1748] The system will only wake up if there is a call from a number not in the phonebook or from a blocked number. This functionality is implemented using Android's TelephonyManager class.

[1749] Consent of the individual, measures to ensure confidentiality of communications

[1750] The system will obtain user consent to enable the feature and take steps to protect the confidentiality of communications, including a process for obtaining consent through the user interface.

[1751] Detecting suspicious calls and notifying users

[1752] If a call comes from a number that is not registered in the phone book, the system will detect the number as a suspicious call and notify the user. This notification is done using the smartphone's notification function.

[1753] Specific examples

[1754] For example, if a user receives a call from an unknown number, the app will detect the number as suspicious and notify the user, allowing them to take action quickly.

[1755] Prompt Sentence Examples

[1756] "Please create an app that detects a call from a number not registered in the phone book as a suspicious call and notifies the user."

[1757] In this way, specific embodiments for carrying out the invention have been described.

[1758] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1759] Step 1:

[1760] The device detects an incoming call. The input is the incoming number, and the output is a signal indicating that an incoming call has been received. Specifically, the Android TelephonyManager class is used to monitor the phone state.

[1761] Step 2:

[1762] The terminal checks the incoming number against the phone book. The input is the incoming number and the phone book data, and the output is the result of determining whether the incoming number is registered in the phone book. Specifically, it searches the phone book database to see if the incoming number exists.

[1763] Step 3:

[1764] The device checks whether the call is from a number not registered in the phone book or from an anonymous number. The input is the result of the judgment in step 2, and the output is the result of the judgment on whether to start the system. Specifically, if the judgment result is "unregistered," start the system.

[1765] Step 4:

[1766] The device begins speech recognition. The input is the incoming voice and the output is the audio data. Specifically, it uses the Google Speech-to-Text API to convert the voice into a digital signal.

[1767] Step 5:

[1768] The device analyzes the voice data using natural language processing technology. The input is voice data and the output is text data. Specifically, it uses natural language processing libraries such as NLTK and spaCy to convert the voice data into text and analyze the content.

[1769] Step 6:

[1770] The device checks the text data against specific keywords. The input is the text data, and the output is the keyword detection results. Specifically, it checks whether the data contains keywords such as "settlement money," "accident," "bank," and "transfer."

[1771] Step 7:

[1772] If a keyword is detected on the device, a notification is sent to family and relatives using a communication method. The input is the keyword detection result and the notification content, and the output is the notification sending result. Specifically, the notification is sent using a messaging service API or a communication service API.

[1773] Step 8:

[1774] The device detects the call as suspicious and notifies the user. The input is the incoming number and keyword detection results, and the output is a notification to the user. Specifically, the smartphone's notification function is used to notify the user of the suspicious call information.

[1775] In this way, the specific operations, inputs, and outputs in each processing step have been explained.

[1776] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1777] "Example 1"

[1778] One embodiment of the present invention is a system incorporating an emotion engine. This system captures a user's voice and converts it into text using speech recognition technology. The converted text is analyzed using natural language processing technology to determine whether it contains specific keywords. The emotion engine then analyzes the user's voice for emotion and feeds the results back to the conversation content analysis means.

[1779] "Example 2"

[1780] For example, if a user utters keywords such as "settlement money," "accident," "bank," or "transfer" over the phone, the system will detect this and send a notification to family and relatives via pre-registered messenger apps or social media. Furthermore, if the emotion engine detects emotions such as anger or fear from the user's tone of voice or speaking style, these will also be sent as part of the notification.

[1781] "Example 3"

[1782] Additionally, if the emotion engine determines that a user's emotions exceed a certain threshold, for example, that the user is extremely angry or extremely scared, the system will automatically send a response to family and relatives via pre-registered social media apps such as messenger apps, allowing users to quickly notify those around them when they are in danger.

[1783] The processing flow of each embodiment will be described below.

[1784] "Example 1"

[1785] Step 1: Capture the user's voice.

[1786] Step 2: Convert the speech to text using speech recognition technology.

[1787] Step 3: The converted text is analyzed using natural la...

Claims

1. A means for recognizing a user's voice; A means for analyzing the contents of the user's conversation based on the recognized voice of the user using a natural language processing technique; a means for taking a predetermined reaction when a specific keyword related to a crime is detected from the conversation content of the user; means for analyzing the user's emotions; a means for transmitting the result of the predetermined reaction and information on the analyzed emotion of the user to a pre-registered contact address; A system including:

2. The means for analyzing the content of the conversation analyzes the content of the conversation of the user based on data obtained by converting the voice of the user into text. The system of claim 1 .

3. The specific keywords include at least one keyword selected from the group consisting of settlement money, accident, bank, and transfer. The system of claim 1 .

Images