System

Abstract

To provide a system capable of quickly and effectively preventing lonely death.SOLUTION: The data collection means includes means for acquiring call records, message history, and application usage data from mobile communication devices, means for acquiring call records from fixed communication devices, means for receiving power and gas consumption data, means for storing and encrypting data, means for complementing missing values of collected data as data pre-processing means, means for detecting and correcting abnormal values of collected data, means for training a generative AI model that learns normal usage patterns using the acquired data as abnormality detection means, means for analyzing new data in real time and detecting abnormality, means for transmitting an alert to a notification destination when an abnormality is detected, means for notifying a plurality of means, and means for providing a manual release function when no abnormality is detected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The increasing number of "lonely deaths" has become a serious problem in recent years, and is particularly prevalent among elderly people who live alone. Current prevention measures for this issue, such as local government visiting services and IoT devices, have limitations and often make it difficult to respond quickly. There is a need for measures to prevent lonely deaths that target the working generation, and new solutions that are suited to the current situation where the use of internet-connected devices is increasing are needed. [Means for solving the problem]

[0005] The present invention provides a system for collecting data on mobile communication devices, fixed communication devices, and electricity and gas consumption, and for preprocessing, anomaly detection, and notification of the collected data. Specifically, the present invention provides a system including the following means.

[0006] 1. The data collection means includes means for acquiring call records, message history, and application usage data from mobile communication devices, means for acquiring call records from fixed communication devices, and means for receiving electricity and gas consumption data in real time from smart meters.

[0007] 2. As a data preprocessing means, it is equipped with a means to complement missing values ​​in the collected data and a means to detect and correct outliers.

[0008] 3. Anomaly detection means include a means to train a generative AI model that learns normal usage patterns using acquired data, and a means to analyze new data in real time to detect anomalies.

[0009] 4. Notification methods include sending an alert to the notification recipient when an abnormality is detected, providing notification via multiple means (telephone, email, SMS), and providing a manual cancellation function when no abnormalities are confirmed.

[0010] This makes it possible to use product usage data to detect abnormalities and notify relevant parties at the optimal time, thereby quickly and effectively preventing lonely deaths.

[0011] "Data Collection Means" means means for obtaining call records, message histories, application usage data, and consumption data from users' mobile communication devices, fixed communication devices, and smart electricity and gas meters.

[0012] A "generative AI model" is an artificial intelligence model that learns normal usage patterns based on collected data and detects anomalies.

[0013] The "data preprocessing means" is a means for complementing missing values ​​in collected data and detecting and correcting outliers.

[0014] The "anomaly detection means" is a means for learning normal usage patterns using acquired data and analyzing new data in real time to detect anomalies.

[0015] The "notification means" is a means for sending an alert to the notification destination when an abnormality is detected, and for notifying using multiple means, and also a means for providing a manual release function when no abnormality is confirmed.

[0016] A "mobile communication device" is a device with communication capabilities that users use on a daily basis, such as a mobile phone, smartphone, or tablet.

[0017] A "fixed communication device" is a communication device used at a fixed location, such as a landline telephone.

[0018] "Consumption Data" means data on electricity and gas usage obtained from smart meters.

[0019] A "smart meter" is a meter that can measure electricity and gas usage in real time and transmit the data.

[0020] "Missing values" refer to values ​​that are not recorded or are unknown in a dataset.

[0021] "Correction" is a process of adjusting abnormal values ​​to fall within an appropriate range.

[0022] An "alert" is a warning message that is sent when an abnormality is detected.

[0023] The "manual cancellation function" is a function that stops notifications when the user confirms that there is no abnormality. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[0046] Data collection methods

[0047] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from the smart meters.

[0048] Data preprocessing measures

[0049] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0050] Anomaly detection means

[0051] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0052] Notification means

[0053] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0054] Specific examples

[0055] Consider a scenario in which Mr. C, a single elderly person, lives alone. The server collects Mr. C's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that Mr. C's normal daily electricity consumption is 5 kWh and his gas usage is 2 cubic meters.

[0056] If one day, Person C does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. The notification method will automatically send a notification to Person C's family and local government that an "abnormality has been detected," allowing the relevant parties to respond quickly. If Person C is simply away from home for some reason and there is no problem, Person C can manually cancel the notification himself / herself.

[0057] This system allows family members living far away, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario.Furthermore, it is extremely convenient in that it does not require users to install special equipment and can efficiently utilize existing infrastructure.

[0058] The processing flow will be explained below.

[0059] Program processing steps

[0060] Data collection

[0061] Step 1:

[0062] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[0063] The server uses the mobile carrier's API to download the user's call and message history.

[0064] The server collects application usage data from users' smartphones and tablets.

[0065] Step 2:

[0066] A server retrieves call records from the fixed communication device.

[0067] The server retrieves call records through the fixed-line carrier's API and stores them in a database.

[0068] Step 3:

[0069] The server receives consumption data in real time from electricity and gas smart meters.

[0070] The server uses the APIs of electricity and gas suppliers to collect consumption data every hour.

[0071] Data Preprocessing

[0072] Step 4:

[0073] The server preprocesses the collected data.

[0074] The server detects missing values ​​in the dataset.

[0075] For example, the average electricity consumption for the past month is calculated and the missing values ​​are filled based on that.

[0076] Step 5:

[0077] The server detects and corrects outliers.

[0078] The server applies statistical techniques to the collected data to identify outliers.

[0079] For example, if call records or power consumption deviate from the normal range, they are corrected to within an appropriate range.

[0080] Anomaly detection

[0081] Step 6:

[0082] The device uses a generative AI model to learn normal usage patterns.

[0083] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[0084] The model is trained on typical call duration, message frequency, and electricity and gas consumption patterns.

[0085] Step 7:

[0086] The server analyzes real-time data and detects anomalies.

[0087] The server inputs the newly collected data into the generative AI model and analyzes it in real time for any abnormalities.

[0088] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[0089] notification

[0090] Step 8:

[0091] The server will notify you when an abnormality is detected.

[0092] The server automatically generates alerts to registered notification recipients (family, local government, property management company).

[0093] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[0094] Step 9:

[0095] The server provides notification in multiple ways.

[0096] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[0097] To reliably notify abnormalities using a plurality of notification means.

[0098] Step 10:

[0099] If the user confirms that there is no abnormality, the notification can be manually canceled.

[0100] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the link or button provided in the tool.

[0101] The server receives a opt-out request from the user to stop further notifications.

[0102] Through the above steps, the present invention can efficiently and quickly detect any abnormalities in the user and notify the relevant parties, thereby preventing the worst-case scenario from occurring.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] It is expected that the rapid detection of lifestyle anomalies from product usage data for single people will help prevent situations such as lonely deaths. However, existing systems have problems with insufficient data correction for missing data and outliers, low accuracy in learning normal usage patterns, and a lack of speed and accuracy in real-time anomaly detection and notification. There is also a need for a system that efficiently utilizes existing infrastructure and can be operated without burdening users.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes means for acquiring call logs, message histories, and application usage data from communication devices, means for acquiring call logs from fixed communication devices, means for receiving consumption data from power supplies and fuel supplies, means for storing and encrypting data, means for completing missing values ​​in the collected data, means for detecting and correcting abnormal values ​​in the collected data, means for training a generative AI model that learns normal usage patterns using the collected data, means for analyzing new data in real time to detect anomalies, means for sending an alert to the communication partner when an anomaly is detected, means for notifying via multiple means, means for providing a manual release function when no anomalies are confirmed, and means for generating a prompt to send a notification when the collected data deviates from the normal pattern for a certain period of time. This improves the accuracy and real-time nature of anomaly detection, enables appropriate notifications to be sent promptly to relevant parties, and prevents worst-case scenarios from occurring.

[0108] The "data collection means" is a means for collecting various data from users' communication devices, fixed communication devices, power supply devices, and fuel supply devices.

[0109] "Communications Device" refers to a user's mobile device or fixed communications equipment from which call logs, message history, and application usage data are collected.

[0110] "Power supply devices and fuel supply devices" refer to devices such as smart meters installed in users' homes that measure the amount of electricity and gas consumed.

[0111] The "data preprocessing means" is a means for complementing missing values ​​in collected data and detecting and correcting outliers.

[0112] A "generative AI model" is an artificial intelligence model that uses previously collected data to learn normal usage patterns and detect abnormal patterns that deviate from the normal range.

[0113] "Real-time anomaly detection means" is a means for analyzing new data in real time and immediately detecting anomalies.

[0114] "Notification means" refers to a means of quickly sending an alert to relevant parties when an abnormality is detected, and includes methods of sending an alert such as telephone, email, and SMS.

[0115] The "manual cancellation function" is a function that allows the user to manually cancel the notification if they confirm that there is no abnormality.

[0116] The "means for generating a prompt" refers to a means for generating a prompt to send appropriate notifications or alerts when collected data deviates from a normal pattern for a certain period of time.

[0117] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[0118] Data collection methods

[0119] The server collects data 24 hours a day from users' communication devices, fixed communication devices, and smart meters for electricity and fuel supply equipment. Call logs, message history, and application usage data are obtained from communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and fuel consumption data is received in real time from smart meters. The hardware used includes: smart meters (common examples, and can be from a variety of manufacturers), mobile communication devices (e.g., various smartphones and tablets), and fixed communication devices (e.g., various landline phones). The software uses a data collection API.

[0120] Data preprocessing measures

[0121] The server preprocesses the collected data. Specifically, it complements missing values ​​and detects and corrects outliers. For example, if there are missing power consumption data, it calculates the average value from past data and complements the missing parts. Also, if abnormally high or low values ​​are detected, they are corrected to within an appropriate range using a normalization algorithm. The software used includes data preprocessing libraries (e.g., Pandas and NumPy).

[0122] Anomaly detection means

[0123] The device uses a generative AI model to learn normal usage patterns. This model is trained based on previously collected data, learning mobile communication device usage time and electricity and gas consumption patterns. As new data is generated in real time, the server inputs this data into the AI ​​model to determine whether there are any anomalies. Specifically, if electricity consumption drops to 0 kWh for a week, this is detected as an anomaly. The software used includes generative AI models (e.g., TensorFlow and PyTorch) and anomaly detection algorithms.

[0124] Notification means

[0125] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local governments, management companies, and other relevant parties. Notifications are sent via phone, email, or SMS, and include details of the abnormality, the need for action, and contact information. Users can also manually cancel notifications if they confirm that there is no problem. Software used includes notification management systems (e.g., Twilio API, Sendgrid).

[0126] Specific examples

[0127] Consider a single elderly person living alone. The server collects the user's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that a user's normal lifestyle pattern involves daily electricity consumption of 5 kWh and gas consumption of 2 cubic meters. If the user does not use their mobile phone at all on one day and their electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. A notification system automatically sends a notice to the user's family and local government that an anomaly has been detected, allowing the relevant parties to respond promptly.

[0128] This system allows relevant parties to quickly identify abnormalities and prevent the worst-case scenario from occurring. Furthermore, it is extremely convenient in that it does not require users to install additional special equipment, and allows the efficient use of existing infrastructure.

[0129] Prompt Sentence Examples

[0130] "Create a prompt to detect an anomaly and send a notification if the user's call records, electricity consumption, or gas usage deviate from normal patterns for more than three days. Specifically, detect an anomaly when electricity consumption is less than 5 kWh or gas usage is less than 2 cubic meters."

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1:

[0133] The server connects to the communication device. The server retrieves call records, message history, and application usage data from the user's communication device. Specifically, it requests data using the communication device's API and stores the received data in a database. The input is the data request from the communication device, and the output is the retrieved usage data.

[0134] Step 2:

[0135] The server connects to the fixed communication device. The server acquires call records from the fixed communication device. Specifically, the server reads the log of the fixed communication device, extracts the call records, and stores them in a database. The input is a data request from the fixed communication device, and the output is the acquired call records.

[0136] Step 3:

[0137] The server connects to the smart meters of the electricity supply equipment and fuel supply equipment. The server obtains electricity and gas consumption data in real time. Specifically, it obtains the data using the smart meter's API and stores it in a database. The input is a data request from the smart meter, and the output is the received consumption data.

[0138] Step 4:

[0139] The server retrieves the collected data from the database. First, it detects missing values ​​and then calculates the average value based on past data to fill in the gaps. Specifically, it finds the missing values ​​in the collected data and fills in the gaps with the average value calculated from past data. The input is the collected data from the database, and the output is the filled data.

[0140] Step 5:

[0141] The server performs range checks on the data, detects abnormally high or low values, and applies a normalization algorithm to correct them. Specifically, it finds data that deviates from the normal range and converts it to fit within a predefined range. The input is the data with missing values ​​imputed, and the output is the data with the outliers corrected.

[0142] Step 6:

[0143] The device trains the generative AI model. The device uses past data as input to train the model and have it learn normal usage patterns. Specifically, a training dataset is input into the AI ​​model, and learning progresses epoch by epoch. The input is past collected data, and the output is a trained generative AI model.

[0144] Step 7:

[0145] The server inputs new data into the generative AI model in real time. The server uses the model to detect anomalies. Specifically, it inputs new data into the model, calculates an anomaly score, and determines it as an anomaly if the anomaly score exceeds a threshold. The input is new data in real time, and the output is the anomaly detection result.

[0146] Step 8:

[0147] As soon as the server detects an anomaly, it automatically sends an alert to the user, distant family members, local government, and management company via notification means. Specifically, based on the anomaly detection results, it uses a notification API to send alerts by phone, email, and SMS. The input is the anomaly detection result, and the output is the sent alert notification.

[0148] Step 9:

[0149] After receiving a notification, the user can manually cancel the notification if necessary. Specifically, the user confirms that there is no abnormality through the smartphone app and sends a cancellation request to the server. When the server receives this cancellation request, it cancels the notification state. The input is the cancellation request from the user, and the output is the canceled notification state.

[0150] (Application example 1)

[0151] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0152] In modern society, problems such as lonely deaths of single people and bachelors are on the rise. To prevent such incidents, a system is needed to quickly detect abnormalities in daily life and notify relevant parties. However, existing systems tend to be slow to respond because they are incomplete in data collection and anomaly detection. Furthermore, manual monitoring and the need to disable notifications place a heavy burden on users. Therefore, efficient and automated security services are needed.

[0153] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0154] In this invention, the server includes a means for acquiring communication history, message history, and application usage data from communication devices, a means for acquiring communication history from fixed communication devices, and a means for receiving energy consumption data. This enables automatic detection of anomalies in the daily lives of single people and bachelors and prompt notification. Furthermore, by performing missing value imputation and outlier correction in data, real-time anomaly detection using a generative AI model that learns normal usage patterns, notification via multiple means, automatic message sending, and continuous monitoring of infrastructure equipment usage, the system reduces the burden on users and provides a highly accurate monitoring and notification system.

[0155] 1. "Telecommunications equipment" is a general term for equipment used by users to communicate, including mobile and fixed communications equipment.

[0156] 2. "Mobile communication equipment" refers to portable communication devices such as mobile phones and smartphones.

[0157] 3. "Fixed communications equipment" means a device used for communication at a fixed location, such as a landline telephone or desktop computer.

[0158] 4. "Communication history" refers to data that includes records of calls made by the user and the history of message exchanges.

[0159] 5. "Message History" refers to the historical data of text messages and emails sent and received by a User.

[0160] 6. "Application Usage Data" means data regarding the usage of applications installed by a User.

[0161] 7. "Energy Consumption Data" means data relating to energy consumption, including electricity and gas.

[0162] 8. "Data preprocessing" refers to the process of filling in missing values ​​in collected data and detecting and correcting outliers.

[0163] 9. "Missing value imputation" refers to the process of appropriately completing missing values ​​in a dataset.

[0164] 10. "Outlier correction" refers to the process of correcting abnormally high or low values ​​in a data set to fall within an appropriate range.

[0165] 11. A “generative AI model” is a model trained for anomaly detection using machine learning or deep learning.

[0166] 12. “Training” refers to the process by which a generative AI model learns normal usage patterns using historical data.

[0167] 13. "Real-time analytics" refers to the rapid processing of new data as it is collected and the immediate detection of anomalies.

[0168] 14. "Automatic Notification" refers to the process by which the server automatically sends alerts to relevant parties when an anomaly is detected.

[0169] 15. "Manual Cancellation Function" refers to the function that allows the user to manually cancel an alert notification when they confirm that there are no abnormalities.

[0170] 16. "Emergency Contacts" means a list of contacts to be notified in the event of an emergency.

[0171] 17. "Infrastructure equipment" refers to equipment that supplies electricity, gas, and other supplies necessary for daily life and business.

[0172] The system of the present invention uses communication devices, fixed communication devices, and energy consumption data to detect abnormalities and notify relevant parties as necessary. Specifically, the system is configured as follows.

[0173] Data collection methods

[0174] The server retrieves communication history, message history, and application usage data from mobile communication devices, including smartphones and other mobile devices. It also retrieves communication history from fixed communication devices and receives real-time electricity and gas consumption data from energy meters. All data is securely stored and encrypted.

[0175] Data preprocessing measures

[0176] Missing values ​​are first interpolated from the collected data. Then, outliers are detected and corrected. For example, if there are missing power consumption data, the data is interpolated using the average value from past data. Furthermore, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range. This process uses data processing libraries such as Pandas and NumPy.

[0177] Anomaly detection means

[0178] The device uses a generative AI model to learn normal usage patterns. This model is trained based on past data. The trained model (for example, a Keras model) is fed new data in real time and determines whether there are any abnormalities. The server will recognize an abnormality, for example, if power consumption drops to 0 kWh for a week. The following prompts can be used to train the "generative AI model."

[0179] Example prompt for a generative AI model:

[0180] text

[0181] Training generative AI models to detect anomalies based on electricity, gas, and smartphone usage data

[0182] train_model(input_data, labels)

[0183] Notification means

[0184] As soon as the server detects an abnormality, it automatically notifies the user and designated emergency contacts. Notification methods include phone, email, and SMS. For example, notifications can be sent using Twilio or SMTP. Information sent includes details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0185] Example system operation

[0186] Consider the case of monitoring the lifestyle of an elderly person living alone. In this case, the server collects mobile communication history, message history, app usage data, and energy consumption data 24 hours a day. A generative AI model that learns normal lifestyle patterns detects abnormal data in real time. When an abnormality is detected, the server automatically sends a notification to emergency contacts, enabling a prompt response. A manual deactivation function allows the user to deactivate notifications themselves.

[0187] The above system will enable efficient monitoring of the lives of single people and bachelors, quickly detect any abnormalities, and notify the relevant parties, thereby making it possible to prevent problems such as lonely deaths.

[0188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0189] Step 1: Data collection

[0190] The server retrieves communication history, message history, and application usage data from communication devices. At the same time, it receives communication history from fixed communication devices and real-time electricity and gas consumption data from energy meters. All of this data is securely stored and encrypted. Input is data from various sensors and devices, and output is data stored in encrypted data storage.

[0191] Step 2: Data Preprocessing

[0192] The server first fills in missing values ​​in the collected data. For example, if there are missing power consumption data, this involves filling in the data using the average value from past data. Next, it detects and corrects outliers. If abnormally high or low values ​​are detected, they are normalized to fit within an appropriate range. The input is data obtained from encrypted data storage, and the output is the data that has been filled and normalized.

[0193] Step 3: Training the generative AI model

[0194] The device trains a generative AI model to learn normal usage patterns using preprocessed data. Specifically, it uses past data to learn normal patterns and creates a model to detect abnormal patterns. The prompt statement is "Train a generative AI model that detects anomalies based on electricity, gas, and smartphone usage data." The input is preprocessed data, and the output is a model that has learned normal patterns.

[0195] Step 4: Detect anomalies in real-time data

[0196] Using the generative AI model, the server analyzes new data in real time and determines whether there are any abnormalities. For example, if power consumption drops to 0 kWh, it will recognize this as an abnormality. The input is new data collected in real time, and the output is a flag indicating whether there are any abnormalities.

[0197] Step 5: Notification of abnormalities

[0198] If an anomaly is detected, the server automatically notifies the user and the designated emergency contacts. Notification methods include phone, email, and SMS. This notification is performed using Twilio and SMTP. The input is the anomaly detection flag and a list of emergency contacts, and the output is the notification message sent.

[0199] Step 6: Manual release function

[0200] If the user or designated emergency contact confirms that there are no abnormalities, the server provides a function to manually cancel the notification. Specifically, the user logs in to the system and uses a GUI to confirm that there are no abnormalities. The input is the user's confirmation operation, and the output is the notification cancellation status.

[0201] The above processing steps provide a system that can efficiently monitor the lives of single people and bachelors, quickly detect abnormalities, and notify relevant parties, thereby preventing problems such as lonely deaths.

[0202] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0203] The present invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, an emotion engine, and a notification means.

[0204] Data collection methods

[0205] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from smart meters.

[0206] Data preprocessing measures

[0207] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0208] Anomaly detection means

[0209] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0210] Emotion Engine

[0211] The emotion engine recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion engine also works in conjunction with anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of unresponsiveness can be recognized as an abnormality.

[0212] Notification means

[0213] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0214] Specific examples

[0215] Consider a scenario in which Mr. D, a single elderly person, lives alone. The server collects Mr. D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. The emotion engine also analyzes Mr. D's emotional state from the content of his calls and messages. For example, it learns that a typical lifestyle pattern for Mr. D is that his daily electricity consumption is 5kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[0216] If one day, Person D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. At the same time, the emotion engine will detect from the content of past messages that Person D has recently been sending many negative messages. The notification method will automatically send a notice to Person D's family and local government that "an abnormality has been detected," allowing the relevant parties to respond quickly. If Person D is simply away from home for some reason and there is no problem, Person D can manually cancel the notification himself / herself.

[0217] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, it offers great convenience in that it does not require users to install special equipment and can efficiently utilize existing infrastructure. Furthermore, by combining it with an emotion engine, more advanced anomaly detection is possible, taking into account the user's psychological state.

[0218] The processing flow will be explained below.

[0219] Program processing steps

[0220] Data collection

[0221] Step 1:

[0222] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[0223] The server uses the mobile carrier's API to automatically download the user's call and message history.

[0224] The server periodically collects application usage data from users' smartphones and tablets.

[0225] Step 2:

[0226] A server retrieves call records from the fixed communication device.

[0227] The server retrieves call records through the fixed line carrier's API and stores them in a secure database.

[0228] Step 3:

[0229] The server receives consumption data in real time from electricity and gas smart meters.

[0230] The server uses the APIs of electricity and gas suppliers to collect, encrypt, and store consumption data every hour.

[0231] Data Preprocessing

[0232] Step 4:

[0233] The server preprocesses the collected data.

[0234] The server detects missing values ​​in the dataset and imputes them using statistical methods.

[0235] For example, if there is a gap in the electricity consumption data, it will be supplemented based on the average usage amount over the past month.

[0236] Step 5:

[0237] The server detects and corrects outliers.

[0238] The server uses statistical techniques for each item in the dataset to identify outliers.

[0239] For example, if the power consumption deviates from the normal range, it is corrected to fall within an appropriate range.

[0240] Anomaly detection

[0241] Step 6:

[0242] The device uses a generative AI model to learn normal usage patterns.

[0243] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[0244] The model learns typical call duration, message sending frequency, and electricity and gas consumption patterns.

[0245] Step 7:

[0246] The server analyzes real-time data and detects anomalies.

[0247] The server inputs the newly collected data into the generative AI model, which analyzes it in real time for any abnormalities.

[0248] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[0249] Emotion analysis

[0250] Step 8:

[0251] The server uses an emotion engine to analyze the user's emotional state.

[0252] The server performs text analysis of call content and message history to identify positive and negative sentiment.

[0253] For example, if the message content is negative, the emotion engine will recognize that.

[0254] Step 9:

[0255] The server links the results of the emotion engine with the anomaly detection means.

[0256] The server feeds back the emotional state obtained from the emotion engine to the generative AI model, improving the accuracy of anomaly detection.

[0257] If negative emotional states persist, they are recognized as psychological abnormalities.

[0258] notification

[0259] Step 10:

[0260] The server will notify you when an abnormality is detected.

[0261] The server automatically generates and sends alerts to registered notification recipients (family, local government, property management company).

[0262] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[0263] Step 11:

[0264] The server provides notification in multiple ways.

[0265] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[0266] To reliably notify abnormalities using a plurality of notification means.

[0267] Step 12:

[0268] If the user confirms that there is no abnormality, the notification can be manually canceled.

[0269] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the provided link or button.

[0270] The server receives a opt-out request from the user to stop further notifications.

[0271] Through each of the above steps, the system can detect abnormalities by integrating users' product usage data and emotional state, and notify relevant parties quickly and reliably, thereby preventing serious problems such as lonely deaths.

[0272] Example 2

[0273] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0274] There is a need to detect abnormalities in the lifestyles of single people early on and prevent serious incidents such as solitary death. In particular, when elderly people and others live alone, it is important to respond to sudden changes in their health condition or living situation, but current systems have the problem of not being able to detect such abnormalities quickly and accurately.

[0275] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, and means for storing and encrypting data. This makes it possible to centrally collect information from a wide range of data sources.

[0276] The data preprocessing means also includes means for complementing missing values ​​in the collected data and means for detecting and correcting abnormal values ​​in the collected data, thereby improving the accuracy of the data and the reliability of anomaly detection.

[0277] Furthermore, the system includes an anomaly detection means that uses acquired data to train a generative AI model that learns normal usage patterns, and a means that analyzes new data in real time to detect anomalies, and an emotion analysis means that analyzes call content and message text acquired from the mobile communication device to determine the user's emotional state, thereby making it possible to detect not only physical anomalies but also psychological anomalies.

[0278] Finally, the notification means includes a means for sending an alert to the notification destination when an abnormality is detected, a means for sending notifications by multiple means, and a means for providing a manual cancellation function when no abnormality is confirmed, which enables a quick and appropriate response.

[0279] "Data collection means" refers to means for acquiring various types of data from mobile communication devices, fixed communication devices, electricity and gas smart meters, etc.

[0280] A "mobile communications device" is a mobile communications device, such as a smartphone or tablet, that provides call logs, message history, application usage data, and the like.

[0281] A "fixed communications device" is a fixed communications device such as a landline telephone or router that provides call records.

[0282] A "smart meter" is a measuring device that measures and provides electricity and gas consumption data in real time.

[0283] "Data preprocessing means" refers to means for complementing missing values ​​and detecting and correcting outliers in acquired data.

[0284] "Missing value imputation" is a process in which, when there are gaps in the collected data, the gaps are filled in using past data or average values.

[0285] "Outlier detection and correction" is a process that detects abnormal values ​​in data and corrects them to an appropriate range using methods such as normalization.

[0286] An "anomaly detection method" is a method that uses acquired data to train a generative AI model that learns normal usage patterns and analyzes new data in real time to detect anomalies.

[0287] A "generative AI model" is a model that uses machine learning algorithms such as deep learning to learn patterns in data and detect anomalies.

[0288] "Real-time analysis" is an analytical method that processes new data instantly and immediately determines whether or not there are any abnormalities.

[0289] The "emotion analysis means" is a means for analyzing call content and message text obtained from a mobile communication device to determine the user's emotional state.

[0290] The "notification means" is a means for sending an alert to a notification destination when an abnormality is detected.

[0291] The "manual cancellation function" is a function that allows the user to manually cancel the notification when the user confirms that there is no abnormality.

[0292] This invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and for preventing situations such as solitary death. This system includes data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means.

[0293] Data collection methods

[0294] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. It obtains call logs, message histories, and application usage data from mobile communication devices, and call logs from fixed communication devices. It also receives real-time electricity and gas consumption data from the smart meters.

[0295] Data preprocessing measures

[0296] The server performs missing value interpolation and outlier detection / correction on the collected data. Specifically, if there are missing power consumption data, it interpolates using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0297] Anomaly detection means

[0298] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0299] Emotion analysis means

[0300] The emotion analysis means recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion analysis means works in conjunction with the anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of no response can be recognized as an abnormality.

[0301] Notification means

[0302] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0303] Specific examples

[0304] As an example of a single elderly person, let's assume a scenario where user D lives alone. The server collects D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. In addition, an emotion analysis tool analyzes D's emotional state from the content of his calls and messages. For example, it learns that D's normal daily electricity consumption is 5 kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[0305] If one day D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. At the same time, the sentiment analysis means will detect from the content of past messages that D has recently been sending many negative messages. The notification means will automatically send a notice to D's family and local government that "an anomaly has been detected," allowing the relevant parties to respond quickly. If D is simply away from home for some reason and there is no problem, D can manually cancel the notification himself / herself.

[0306] An example of a prompt is, "D is a single elderly person living alone. His smartphone regularly sends call records and message history to a server, and his home's smart meter also sends real-time electricity and gas usage data. If he does not engage in these normal activities for several days, design a system that detects this as an abnormality and automatically sends a notification to his family and local government."

[0307] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, users do not need to install special equipment, so they can live safely using their existing living environment and infrastructure. Furthermore, by combining it with emotion analysis methods, even more advanced anomaly detection becomes possible.

[0308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0309] Step 1: Data collection

[0310] The server collects data 24 hours a day. It receives data from mobile communication devices, fixed communication devices, and smart meters as input and stores this data as output. From mobile communication devices, it obtains call records, message history, and application usage data, and from fixed communication devices, it obtains landline call records. From smart meters, it receives electricity and gas consumption data in real time. Specifically, the server periodically obtains data from these devices through APIs.

[0311] Step 2: Data Preprocessing

[0312] The server preprocesses the collected data. It uses the data collected in step 1 as input and generates data with missing values ​​and corrected outliers as output. Specifically, if there is missing data, the server calculates the average value from past data to fill in the gaps. Also, if an abnormally high or low value is detected, it normalizes the value and corrects it to fall within an appropriate range. For example, if there is a gap in Mr. D's power consumption data, it fills in the gaps using the average value from a similar time period in the past.

[0313] Step 3: Anomaly detection

[0314] The device uses the generative AI model to detect anomalies. It uses the preprocessed data from step 2 as input and determines whether an anomaly has been detected as output. Specifically, the device trains the AI ​​model based on past data to learn normal usage patterns. It uses this model to analyze new data in real time and determine whether anomalies exist. For example, if Mr. D's electricity consumption is 0 kWh for three days, it will detect this as an anomaly.

[0315] Step 4: Sentiment Analysis

[0316] The server analyzes the user's emotional state using an emotion analysis method. It uses call content and message text acquired from the mobile communication device as input and determines the user's emotional state as output. Specifically, the server analyzes the call content and message text using a natural language processing algorithm to identify positive or negative emotions. For example, if Mr. D's messages over the past month have contained a lot of negative content, it determines his / her emotional state as negative.

[0317] Step 5: Notification of abnormalities

[0318] If the server detects an abnormality, it notifies the relevant parties. It uses the anomaly detection results and sentiment analysis results from steps 3 and 4 as input, and sends an abnormality notification via multiple means as output. Specifically, the server sends an abnormality alert to the user, family, local government, and property management company via phone, email, and SMS. The notification content includes details of the abnormality, the need for response, contact information, etc. For example, if an abnormality is detected for Mr. D, the server will send an email notification containing details of the abnormality and contact information.

[0319] In this way, the system will be able to detect abnormalities among single people at an early stage and notify relevant parties promptly.

[0320] (Application example 2)

[0321] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0322] Elderly people and single people living alone are isolated, and there is a need to detect and respond early when abnormalities in their health or psychological state occur. However, current support systems for isolated people lack real-time data collection and emotion analysis, making it difficult to detect abnormalities early. In addition, they cannot utilize standard infrastructure or devices, which poses challenges in terms of cost and operation.

[0323] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, means for acquiring data from vital sensors and wearable devices that collect location information, means for storing and encrypting data, as data preprocessing means, means for complementing missing values ​​in the collected data, and means for detecting and correcting abnormal values ​​in the collected data, as anomaly detection means, means for training a generative AI model that learns normal usage patterns using the acquired data, and means for analyzing new data in real time to detect anomalies, as emotion analysis means, means for analyzing the user's emotional state from image and audio data, and means for detecting anomalies based on the emotional state, as notification means, means for sending an alert to a notification destination when an anomaly is detected, means for notifying by multiple means, and means for providing a manual release function when no anomaly is confirmed. This makes it possible to detect anomalies and perform emotion analysis in real time based on collected data, enabling early detection of abnormal situations.

[0324] "Data collection means" refers to means for acquiring various data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices.

[0325] A "mobile communication device" is a device capable of mobile communication, such as a mobile phone or smartphone, that is used to obtain call logs, message history, and application usage data.

[0326] A "fixed communication device" is a telephone facility that is fixedly used in a home or office and is a means for obtaining call records.

[0327] A "smart meter" is a device that measures and transmits electricity and gas consumption data in real time.

[0328] A "wearable device" is a device that can be worn by a user and that acquires vital signs and location information.

[0329] "Means for storing and encrypting data" refers to the means for securely storing collected data and encrypting it to protect it from unauthorized access.

[0330] "Data preprocessing means" refers to means for complementing missing values ​​in collected data and detecting and correcting outliers.

[0331] A "generative AI model" is an artificial intelligence model that is trained to learn normal usage patterns using past data and detect anomalies in new data.

[0332] An "anomaly detection method" is a method for analyzing new data in real time using a generative AI model to detect anomalies.

[0333] The "emotion analysis means" is a means for analyzing the user's emotional state from image and audio data and detecting anomalies based on that state.

[0334] "Notification means" refers to a means for sending an alert when an abnormality is detected, notifying via multiple means, and providing a function for manually canceling notifications when no abnormality is confirmed.

[0335] This invention is a system for ensuring safety in the lives of elderly people and single people living alone. This system uses data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means to collect and analyze various types of user data, and can quickly notify the user when an anomaly is detected.

[0336] System configuration

[0337] Data collection methods

[0338] The server collects data by:

[0339] 1. Obtain call logs, message history, and application usage data from mobile communications devices.

[0340] 2. Obtain call records from fixed communication devices.

[0341] 3. Receive real-time consumption data from electricity and gas smart meters.

[0342] 4. Collect vital data such as heart rate and step count, as well as location information, from wearable devices.

[0343] Data preprocessing measures

[0344] The server completes missing values ​​in the collected data and detects and corrects outliers. For example, missing parts of electricity consumption data are completed with the average value from past data. Abnormally high or low values ​​are normalized.

[0345] Anomaly detection means

[0346] The anomaly detection method uses a generative AI model to learn normal usage patterns. Here, the generative AI model is trained from past data and analyzes new data in real time to detect anomalies. For example, if electricity consumption is 0 kWh for one consecutive week, it will be detected as an anomaly.

[0347] Emotion analysis means

[0348] The emotion analysis means uses the wearable device's camera and microphone to analyze the user's facial expressions and voice. The emotional state is determined from the collected data, and if a negative emotional state or a prolonged period of unresponsiveness is detected, it is recognized as an abnormality.

[0349] Notification means

[0350] As soon as the server detects an abnormality, it sends an alert to the user, distant family members, medical institutions, nursing care services, and other relevant parties. Alerts are sent via multiple means, including phone, email, and SMS. If no abnormalities are confirmed, notifications can be manually canceled.

[0351] Specific examples

[0352] Suppose a user wears smart glasses all day long. The server collects heart rate, step count, and location information through the glasses' sensors, and also uses a camera and microphone to capture the user's facial expressions and voice in real time. For example, if one day the user's heart rate suddenly spikes, their face looks sad, and they begin to stay indoors, the server will detect this as an abnormality. As a result, the server will instantly send an alert to their family or a medical institution saying, "An abnormality has been detected. Urgent investigation is required."

[0353] Prompt Sentence Examples

[0354] Check whether the user's heart rate is within the range of 80 to 100, and detect abnormalities if the number of steps is extremely low, or if the user's facial expression or voice indicates that they are sad. If these conditions are met, the system will automatically begin the process of sending an alert.

[0355] This system allows users to detect anomalies early using existing devices and infrastructure, without the need to install special equipment. In addition, by combining it with emotion analysis, more advanced anomaly detection is possible.

[0356] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0357] Step 1:

[0358] The server collects data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices, providing users' call records, message history, application usage data, electricity and gas consumption data, heart rate, step count, location information, etc. The input is raw data from various devices, and the output is an integrated data set stored on the server.

[0359] Step 2:

[0360] The server preprocesses the collected data. During this process, missing values ​​are filled in and outliers are corrected. For example, if some power consumption data is missing, it is filled in with the historical average value, and if an abnormally high heart rate is detected, it is normalized. The input is the collected raw data, and the output is the clean data that has been filled and corrected.

[0361] Step 3:

[0362] The server uses a generative AI model to train anomaly detection. This AI model is used to learn normal usage patterns based on past data. For example, electricity consumption, heart rate, and water, gas, and utility usage patterns are used as training data. The input is the clean data that has been complemented and corrected, and the output is the trained generative AI model.

[0363] Step 4:

[0364] The server inputs new data into the generative AI model in real time to detect anomalies. For example, if electricity consumption drops to 0 kWh for one consecutive week or if the heart rate rises abnormally, this will be detected as an anomaly. The input is new data acquired in real time, and the output is a judgment result as to whether or not there is an anomaly.

[0365] Step 5:

[0366] As an emotion analysis method, the server analyzes image and audio data acquired from the camera and microphone of the wearable device. This allows it to determine the user's emotions from their facial expressions and voice, and detect, for example, negative emotional states or prolonged periods of unresponsiveness. The input is image and audio data, and the output is the analysis result regarding the user's emotional state.

[0367] Step 6:

[0368] As soon as the server detects an anomaly, it sends an alert using a notification method. This notification is sent by multiple means, including phone, email, and SMS, and includes details of the anomaly and the need for action. The input is the anomaly detection judgment result and emotion analysis result, and the output is an alert notification to the user, distant family members, medical institutions, etc.

[0369] Step 7:

[0370] When a user receives a notification, they can manually dismiss the notification if they confirm that there is no abnormality. This function makes it possible to disable false alerts in the event of a false positive. The input is feedback from the user, and the output is dismissal of the notification.

[0371] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0372] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search<url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0373] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0374] [Second embodiment]

[0375] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0376] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0377] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0378] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0379] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0380] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0381] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0382] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0383] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0384] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0385] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0386] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0387] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[0388] Data collection methods

[0389] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from the smart meters.

[0390] Data preprocessing measures

[0391] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0392] Anomaly detection means

[0393] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0394] Notification means

[0395] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0396] Specific examples

[0397] Consider a scenario in which Mr. C, a single elderly person, lives alone. The server collects Mr. C's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that Mr. C's normal daily electricity consumption is 5 kWh and his gas usage is 2 cubic meters.

[0398] If one day, Person C does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. The notification method will automatically send a notification to Person C's family and local government that an "abnormality has been detected," allowing the relevant parties to respond quickly. If Person C is simply away from home for some reason and there is no problem, Person C can manually cancel the notification himself / herself.

[0399] This system allows family members living far away, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario.Furthermore, it is extremely convenient in that it does not require users to install special equipment and can efficiently utilize existing infrastructure.

[0400] The processing flow will be explained below.

[0401] Program processing steps

[0402] Data collection

[0403] Step 1:

[0404] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[0405] The server uses the mobile carrier's API to download the user's call and message history.

[0406] The server collects application usage data from users' smartphones and tablets.

[0407] Step 2:

[0408] A server retrieves call records from the fixed communication device.

[0409] The server retrieves call records through the fixed-line carrier's API and stores them in a database.

[0410] Step 3:

[0411] The server receives consumption data in real time from electricity and gas smart meters.

[0412] The server uses the APIs of electricity and gas suppliers to collect consumption data every hour.

[0413] Data Preprocessing

[0414] Step 4:

[0415] The server preprocesses the collected data.

[0416] The server detects missing values ​​in the dataset.

[0417] For example, the average electricity consumption for the past month is calculated and the missing values ​​are filled based on that.

[0418] Step 5:

[0419] The server detects and corrects outliers.

[0420] The server applies statistical techniques to the collected data to identify outliers.

[0421] For example, if call records or power consumption deviate from the normal range, they are corrected to within an appropriate range.

[0422] Anomaly detection

[0423] Step 6:

[0424] The device uses a generative AI model to learn normal usage patterns.

[0425] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[0426] The model is trained on typical call duration, message frequency, and electricity and gas consumption patterns.

[0427] Step 7:

[0428] The server analyzes real-time data and detects anomalies.

[0429] The server inputs the newly collected data into the generative AI model and analyzes it in real time for any abnormalities.

[0430] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[0431] notification

[0432] Step 8:

[0433] The server will notify you when an abnormality is detected.

[0434] The server automatically generates alerts to registered notification recipients (family, local government, property management company).

[0435] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[0436] Step 9:

[0437] The server provides notification in multiple ways.

[0438] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[0439] To reliably notify abnormalities using a plurality of notification means.

[0440] Step 10:

[0441] If the user confirms that there is no abnormality, the notification can be manually canceled.

[0442] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the link or button provided in the tool.

[0443] The server receives a opt-out request from the user to stop further notifications.

[0444] Through the above steps, the present invention can efficiently and quickly detect any abnormalities in the user and notify the relevant parties, thereby preventing the worst-case scenario from occurring.

[0445] Example 1

[0446] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0447] It is expected that the rapid detection of lifestyle anomalies from product usage data for single people will help prevent situations such as lonely deaths. However, existing systems have problems with insufficient data correction for missing data and outliers, low accuracy in learning normal usage patterns, and a lack of speed and accuracy in real-time anomaly detection and notification. There is also a need for a system that efficiently utilizes existing infrastructure and can be operated without burdening users.

[0448] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0449] In this invention, the server includes means for acquiring call logs, message histories, and application usage data from communication devices, means for acquiring call logs from fixed communication devices, means for receiving consumption data from power supplies and fuel supplies, means for storing and encrypting data, means for completing missing values ​​in the collected data, means for detecting and correcting abnormal values ​​in the collected data, means for training a generative AI model that learns normal usage patterns using the collected data, means for analyzing new data in real time to detect anomalies, means for sending an alert to the communication partner when an anomaly is detected, means for notifying via multiple means, means for providing a manual release function when no anomalies are confirmed, and means for generating a prompt to send a notification when the collected data deviates from the normal pattern for a certain period of time. This improves the accuracy and real-time nature of anomaly detection, enables appropriate notifications to be sent promptly to relevant parties, and prevents worst-case scenarios from occurring.

[0450] The "data collection means" is a means for collecting various data from users' communication devices, fixed communication devices, power supply devices, and fuel supply devices.

[0451] "Communications Device" refers to a user's mobile device or fixed communications equipment from which call logs, message history, and application usage data are collected.

[0452] "Power supply devices and fuel supply devices" refer to devices such as smart meters installed in users' homes that measure the amount of electricity and gas consumed.

[0453] The "data preprocessing means" is a means for complementing missing values ​​in collected data and detecting and correcting outliers.

[0454] A "generative AI model" is an artificial intelligence model that uses previously collected data to learn normal usage patterns and detect abnormal patterns that deviate from the normal range.

[0455] "Real-time anomaly detection means" is a means for analyzing new data in real time and immediately detecting anomalies.

[0456] "Notification means" refers to a means of quickly sending an alert to relevant parties when an abnormality is detected, and includes methods of sending an alert such as telephone, email, and SMS.

[0457] The "manual cancellation function" is a function that allows the user to manually cancel the notification if they confirm that there is no abnormality.

[0458] The "means for generating a prompt" refers to a means for generating a prompt to send appropriate notifications or alerts when collected data deviates from a normal pattern for a certain period of time.

[0459] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[0460] Data collection methods

[0461] The server collects data 24 hours a day from users' communication devices, fixed communication devices, and smart meters for electricity and fuel supply equipment. Call logs, message history, and application usage data are obtained from communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and fuel consumption data is received in real time from smart meters. The hardware used includes: smart meters (common examples, and can be from a variety of manufacturers), mobile communication devices (e.g., various smartphones and tablets), and fixed communication devices (e.g., various landline phones). The software uses a data collection API.

[0462] Data preprocessing measures

[0463] The server preprocesses the collected data. Specifically, it complements missing values ​​and detects and corrects outliers. For example, if there are missing power consumption data, it calculates the average value from past data and complements the missing parts. Also, if abnormally high or low values ​​are detected, they are corrected to within an appropriate range using a normalization algorithm. The software used includes data preprocessing libraries (e.g., Pandas and NumPy).

[0464] Anomaly detection means

[0465] The device uses a generative AI model to learn normal usage patterns. This model is trained based on previously collected data, learning mobile communication device usage time and electricity and gas consumption patterns. As new data is generated in real time, the server inputs this data into the AI ​​model to determine whether there are any anomalies. Specifically, if electricity consumption drops to 0 kWh for a week, this is detected as an anomaly. The software used includes generative AI models (e.g., TensorFlow and PyTorch) and anomaly detection algorithms.

[0466] Notification means

[0467] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local governments, management companies, and other relevant parties. Notifications are sent via phone, email, or SMS, and include details of the abnormality, the need for action, and contact information. Users can also manually cancel notifications if they confirm that there is no problem. Software used includes notification management systems (e.g., Twilio API, Sendgrid).

[0468] Specific examples

[0469] Consider a single elderly person living alone. The server collects the user's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that a user's normal lifestyle pattern involves daily electricity consumption of 5 kWh and gas consumption of 2 cubic meters. If the user does not use their mobile phone at all on one day and their electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. A notification system automatically sends a notice to the user's family and local government that an anomaly has been detected, allowing the relevant parties to respond promptly.

[0470] This system allows relevant parties to quickly identify abnormalities and prevent the worst-case scenario from occurring. Furthermore, it is extremely convenient in that it does not require users to install additional special equipment, and allows the efficient use of existing infrastructure.

[0471] Prompt Sentence Examples

[0472] "Create a prompt to detect an anomaly and send a notification if the user's call records, electricity consumption, or gas usage deviate from normal patterns for more than three days. Specifically, detect an anomaly when electricity consumption is less than 5 kWh or gas usage is less than 2 cubic meters."

[0473] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0474] Step 1:

[0475] The server connects to the communication device. The server retrieves call records, message history, and application usage data from the user's communication device. Specifically, it requests data using the communication device's API and stores the received data in a database. The input is the data request from the communication device, and the output is the retrieved usage data.

[0476] Step 2:

[0477] The server connects to the fixed communication device. The server acquires call records from the fixed communication device. Specifically, the server reads the log of the fixed communication device, extracts the call records, and stores them in a database. The input is a data request from the fixed communication device, and the output is the acquired call records.

[0478] Step 3:

[0479] The server connects to the smart meters of the electricity supply equipment and fuel supply equipment. The server obtains electricity and gas consumption data in real time. Specifically, it obtains the data using the smart meter's API and stores it in a database. The input is a data request from the smart meter, and the output is the received consumption data.

[0480] Step 4:

[0481] The server retrieves the collected data from the database. First, it detects missing values ​​and then calculates the average value based on past data to fill in the gaps. Specifically, it finds the missing values ​​in the collected data and fills in the gaps with the average value calculated from past data. The input is the collected data from the database, and the output is the filled data.

[0482] Step 5:

[0483] The server performs range checks on the data, detects abnormally high or low values, and applies a normalization algorithm to correct them. Specifically, it finds data that deviates from the normal range and converts it to fit within a predefined range. The input is the data with missing values ​​imputed, and the output is the data with the outliers corrected.

[0484] Step 6:

[0485] The device trains the generative AI model. The device uses past data as input to train the model and have it learn normal usage patterns. Specifically, a training dataset is input into the AI ​​model, and learning progresses epoch by epoch. The input is past collected data, and the output is a trained generative AI model.

[0486] Step 7:

[0487] The server inputs new data into the generative AI model in real time. The server uses the model to detect anomalies. Specifically, it inputs new data into the model, calculates an anomaly score, and determines it as an anomaly if the anomaly score exceeds a threshold. The input is new data in real time, and the output is the anomaly detection result.

[0488] Step 8:

[0489] As soon as the server detects an anomaly, it automatically sends an alert to the user, distant family members, local government, and management company via notification means. Specifically, based on the anomaly detection results, it uses a notification API to send alerts by phone, email, and SMS. The input is the anomaly detection result, and the output is the sent alert notification.

[0490] Step 9:

[0491] After receiving a notification, the user can manually cancel the notification if necessary. Specifically, the user confirms that there is no abnormality through the smartphone app and sends a cancellation request to the server. When the server receives this cancellation request, it cancels the notification state. The input is the cancellation request from the user, and the output is the canceled notification state.

[0492] (Application example 1)

[0493] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0494] In modern society, problems such as lonely deaths of single people and bachelors are on the rise. To prevent such incidents, a system is needed to quickly detect abnormalities in daily life and notify relevant parties. However, existing systems tend to be slow to respond because they are incomplete in data collection and anomaly detection. Furthermore, manual monitoring and the need to disable notifications place a heavy burden on users. Therefore, efficient and automated security services are needed.

[0495] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0496] In this invention, the server includes a means for acquiring communication history, message history, and application usage data from communication devices, a means for acquiring communication history from fixed communication devices, and a means for receiving energy consumption data. This enables automatic detection of anomalies in the daily lives of single people and bachelors and prompt notification. Furthermore, by performing missing value imputation and outlier correction in data, real-time anomaly detection using a generative AI model that learns normal usage patterns, notification via multiple means, automatic message sending, and continuous monitoring of infrastructure equipment usage, the system reduces the burden on users and provides a highly accurate monitoring and notification system.

[0497] 1. "Telecommunications equipment" is a general term for equipment used by users to communicate, including mobile and fixed communications equipment.

[0498] 2. "Mobile communication equipment" refers to portable communication devices such as mobile phones and smartphones.

[0499] 3. "Fixed communications equipment" means a device used for communication at a fixed location, such as a landline telephone or desktop computer.

[0500] 4. "Communication history" refers to data that includes records of calls made by the user and the history of message exchanges.

[0501] 5. "Message History" refers to the historical data of text messages and emails sent and received by a User.

[0502] 6. "Application Usage Data" means data regarding the usage of applications installed by a User.

[0503] 7. "Energy Consumption Data" means data relating to energy consumption, including electricity and gas.

[0504] 8. "Data preprocessing" refers to the process of filling in missing values ​​in collected data and detecting and correcting outliers.

[0505] 9. "Missing value imputation" refers to the process of appropriately completing missing values ​​in a dataset.

[0506] 10. "Outlier correction" refers to the process of correcting abnormally high or low values ​​in a data set to fall within an appropriate range.

[0507] 11. A “generative AI model” is a model trained for anomaly detection using machine learning or deep learning.

[0508] 12. “Training” refers to the process by which a generative AI model learns normal usage patterns using historical data.

[0509] 13. "Real-time analytics" refers to the rapid processing of new data as it is collected and the immediate detection of anomalies.

[0510] 14. "Automatic Notification" refers to the process by which the server automatically sends alerts to relevant parties when an anomaly is detected.

[0511] 15. "Manual Cancellation Function" refers to the function that allows the user to manually cancel an alert notification when they confirm that there are no abnormalities.

[0512] 16. "Emergency Contacts" means a list of contacts to be notified in the event of an emergency.

[0513] 17. "Infrastructure equipment" refers to equipment that supplies electricity, gas, and other supplies necessary for daily life and business.

[0514] The system of the present invention uses communication devices, fixed communication devices, and energy consumption data to detect abnormalities and notify relevant parties as necessary. Specifically, the system is configured as follows.

[0515] Data collection methods

[0516] The server retrieves communication history, message history, and application usage data from mobile communication devices, including smartphones and other mobile devices. It also retrieves communication history from fixed communication devices and receives real-time electricity and gas consumption data from energy meters. All data is securely stored and encrypted.

[0517] Data preprocessing measures

[0518] Missing values ​​are first interpolated from the collected data. Then, outliers are detected and corrected. For example, if there are missing power consumption data, the data is interpolated using the average value from past data. Furthermore, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range. This process uses data processing libraries such as Pandas and NumPy.

[0519] Anomaly detection means

[0520] The device uses a generative AI model to learn normal usage patterns. This model is trained based on past data. The trained model (for example, a Keras model) is fed new data in real time and determines whether there are any abnormalities. The server will recognize an abnormality, for example, if power consumption drops to 0 kWh for a week. The following prompts can be used to train the "generative AI model."

[0521] Example prompt for a generative AI model:

[0522] text

[0523] Training generative AI models to detect anomalies based on electricity, gas, and smartphone usage data

[0524] train_model(input_data, labels)

[0525] Notification means

[0526] As soon as the server detects an abnormality, it automatically notifies the user and designated emergency contacts. Notification methods include phone, email, and SMS. For example, notifications can be sent using Twilio or SMTP. Information sent includes details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0527] Example system operation

[0528] Consider the case of monitoring the lifestyle of an elderly person living alone. In this case, the server collects mobile communication history, message history, app usage data, and energy consumption data 24 hours a day. A generative AI model that learns normal lifestyle patterns detects abnormal data in real time. When an abnormality is detected, the server automatically sends a notification to emergency contacts, enabling a prompt response. A manual deactivation function allows the user to deactivate notifications themselves.

[0529] The above system will enable efficient monitoring of the lives of single people and bachelors, quickly detect any abnormalities, and notify the relevant parties, thereby making it possible to prevent problems such as lonely deaths.

[0530] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0531] Step 1: Data collection

[0532] The server retrieves communication history, message history, and application usage data from communication devices. At the same time, it receives communication history from fixed communication devices and real-time electricity and gas consumption data from energy meters. All of this data is securely stored and encrypted. Input is data from various sensors and devices, and output is data stored in encrypted data storage.

[0533] Step 2: Data Preprocessing

[0534] The server first fills in missing values ​​in the collected data. For example, if there are missing power consumption data, this involves filling in the data using the average value from past data. Next, it detects and corrects outliers. If abnormally high or low values ​​are detected, they are normalized to fit within an appropriate range. The input is data obtained from encrypted data storage, and the output is the data that has been filled and normalized.

[0535] Step 3: Training the generative AI model

[0536] The device trains a generative AI model to learn normal usage patterns using preprocessed data. Specifically, it uses past data to learn normal patterns and creates a model to detect abnormal patterns. The prompt statement is "Train a generative AI model that detects anomalies based on electricity, gas, and smartphone usage data." The input is preprocessed data, and the output is a model that has learned normal patterns.

[0537] Step 4: Detect anomalies in real-time data

[0538] Using the generative AI model, the server analyzes new data in real time and determines whether there are any abnormalities. For example, if power consumption drops to 0 kWh, it will recognize this as an abnormality. The input is new data collected in real time, and the output is a flag indicating whether there are any abnormalities.

[0539] Step 5: Notification of abnormalities

[0540] If an anomaly is detected, the server automatically notifies the user and the designated emergency contacts. Notification methods include phone, email, and SMS. This notification is performed using Twilio and SMTP. The input is the anomaly detection flag and a list of emergency contacts, and the output is the notification message sent.

[0541] Step 6: Manual release function

[0542] If the user or designated emergency contact confirms that there are no abnormalities, the server provides a function to manually cancel the notification. Specifically, the user logs in to the system and uses a GUI to confirm that there are no abnormalities. The input is the user's confirmation operation, and the output is the notification cancellation status.

[0543] The above processing steps provide a system that can efficiently monitor the lives of single people and bachelors, quickly detect abnormalities, and notify relevant parties, thereby preventing problems such as lonely deaths.

[0544] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0545] The present invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, an emotion engine, and a notification means.

[0546] Data collection methods

[0547] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from smart meters.

[0548] Data preprocessing measures

[0549] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0550] Anomaly detection means

[0551] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0552] Emotion Engine

[0553] The emotion engine recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion engine also works in conjunction with anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of unresponsiveness can be recognized as an abnormality.

[0554] Notification means

[0555] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0556] Specific examples

[0557] Consider a scenario in which Mr. D, a single elderly person, lives alone. The server collects Mr. D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. The emotion engine also analyzes Mr. D's emotional state from the content of his calls and messages. For example, it learns that a typical lifestyle pattern for Mr. D is that his daily electricity consumption is 5kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[0558] If one day, Person D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. At the same time, the emotion engine will detect from the content of past messages that Person D has recently been sending many negative messages. The notification method will automatically send a notice to Person D's family and local government that "an abnormality has been detected," allowing the relevant parties to respond quickly. If Person D is simply away from home for some reason and there is no problem, Person D can manually cancel the notification himself / herself.

[0559] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, it offers great convenience in that it does not require users to install special equipment and can efficiently utilize existing infrastructure. Furthermore, by combining it with an emotion engine, more advanced anomaly detection is possible, taking into account the user's psychological state.

[0560] The processing flow will be explained below.

[0561] Program processing steps

[0562] Data collection

[0563] Step 1:

[0564] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[0565] The server uses the mobile carrier's API to automatically download the user's call and message history.

[0566] The server periodically collects application usage data from users' smartphones and tablets.

[0567] Step 2:

[0568] A server retrieves call records from the fixed communication device.

[0569] The server retrieves call records through the fixed line carrier's API and stores them in a secure database.

[0570] Step 3:

[0571] The server receives consumption data in real time from electricity and gas smart meters.

[0572] The server uses the APIs of electricity and gas suppliers to collect, encrypt, and store consumption data every hour.

[0573] Data Preprocessing

[0574] Step 4:

[0575] The server preprocesses the collected data.

[0576] The server detects missing values ​​in the dataset and imputes them using statistical methods.

[0577] For example, if there is a gap in the electricity consumption data, it will be supplemented based on the average usage amount over the past month.

[0578] Step 5:

[0579] The server detects and corrects outliers.

[0580] The server uses statistical techniques for each item in the dataset to identify outliers.

[0581] For example, if the power consumption deviates from the normal range, it is corrected to fall within an appropriate range.

[0582] Anomaly detection

[0583] Step 6:

[0584] The device uses a generative AI model to learn normal usage patterns.

[0585] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[0586] The model learns typical call duration, message sending frequency, and electricity and gas consumption patterns.

[0587] Step 7:

[0588] The server analyzes real-time data and detects anomalies.

[0589] The server inputs the newly collected data into the generative AI model, which analyzes it in real time for any abnormalities.

[0590] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[0591] Emotion analysis

[0592] Step 8:

[0593] The server uses an emotion engine to analyze the user's emotional state.

[0594] The server performs text analysis of call content and message history to identify positive and negative sentiment.

[0595] For example, if the message content is negative, the emotion engine will recognize that.

[0596] Step 9:

[0597] The server links the results of the emotion engine with the anomaly detection means.

[0598] The server feeds back the emotional state obtained from the emotion engine to the generative AI model, improving the accuracy of anomaly detection.

[0599] If negative emotional states persist, they are recognized as psychological abnormalities.

[0600] notification

[0601] Step 10:

[0602] The server will notify you when an abnormality is detected.

[0603] The server automatically generates and sends alerts to registered notification recipients (family, local government, property management company).

[0604] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[0605] Step 11:

[0606] The server provides notification in multiple ways.

[0607] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[0608] To reliably notify abnormalities using a plurality of notification means.

[0609] Step 12:

[0610] If the user confirms that there is no abnormality, the notification can be manually canceled.

[0611] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the provided link or button.

[0612] The server receives a opt-out request from the user to stop further notifications.

[0613] Through each of the above steps, the system can detect abnormalities by integrating users' product usage data and emotional state, and notify relevant parties quickly and reliably, thereby preventing serious problems such as lonely deaths.

[0614] Example 2

[0615] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0616] There is a need to detect abnormalities in the lifestyles of single people early on and prevent serious incidents such as solitary death. In particular, when elderly people and others live alone, it is important to respond to sudden changes in their health condition or living situation, but current systems have the problem of not being able to detect such abnormalities quickly and accurately.

[0617] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, and means for storing and encrypting data. This makes it possible to centrally collect information from a wide range of data sources.

[0618] The data preprocessing means also includes means for complementing missing values ​​in the collected data and means for detecting and correcting abnormal values ​​in the collected data, thereby improving the accuracy of the data and the reliability of anomaly detection.

[0619] Furthermore, the system includes an anomaly detection means that uses acquired data to train a generative AI model that learns normal usage patterns, and a means that analyzes new data in real time to detect anomalies, and an emotion analysis means that analyzes call content and message text acquired from the mobile communication device to determine the user's emotional state, thereby making it possible to detect not only physical anomalies but also psychological anomalies.

[0620] Finally, the notification means includes a means for sending an alert to the notification destination when an abnormality is detected, a means for sending notifications by multiple means, and a means for providing a manual cancellation function when no abnormality is confirmed, which enables a quick and appropriate response.

[0621] "Data collection means" refers to means for acquiring various types of data from mobile communication devices, fixed communication devices, electricity and gas smart meters, etc.

[0622] A "mobile communications device" is a mobile communications device, such as a smartphone or tablet, that provides call logs, message history, application usage data, and the like.

[0623] A "fixed communications device" is a fixed communications device such as a landline telephone or router that provides call records.

[0624] A "smart meter" is a measuring device that measures and provides electricity and gas consumption data in real time.

[0625] "Data preprocessing means" refers to means for complementing missing values ​​and detecting and correcting outliers in acquired data.

[0626] "Missing value imputation" is a process in which, when there are gaps in the collected data, the gaps are filled in using past data or average values.

[0627] "Outlier detection and correction" is a process that detects abnormal values ​​in data and corrects them to an appropriate range using methods such as normalization.

[0628] An "anomaly detection method" is a method that uses acquired data to train a generative AI model that learns normal usage patterns and analyzes new data in real time to detect anomalies.

[0629] A "generative AI model" is a model that uses machine learning algorithms such as deep learning to learn patterns in data and detect anomalies.

[0630] "Real-time analysis" is an analytical method that processes new data instantly and immediately determines whether or not there are any abnormalities.

[0631] The "emotion analysis means" is a means for analyzing call content and message text obtained from a mobile communication device to determine the user's emotional state.

[0632] The "notification means" is a means for sending an alert to a notification destination when an abnormality is detected.

[0633] The "manual cancellation function" is a function that allows the user to manually cancel the notification when the user confirms that there is no abnormality.

[0634] This invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and for preventing situations such as solitary death. This system includes data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means.

[0635] Data collection methods

[0636] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. It obtains call logs, message histories, and application usage data from mobile communication devices, and call logs from fixed communication devices. It also receives real-time electricity and gas consumption data from the smart meters.

[0637] Data preprocessing measures

[0638] The server performs missing value interpolation and outlier detection / correction on the collected data. Specifically, if there are missing power consumption data, it interpolates using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0639] Anomaly detection means

[0640] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0641] Emotion analysis means

[0642] The emotion analysis means recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion analysis means works in conjunction with the anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of no response can be recognized as an abnormality.

[0643] Notification means

[0644] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0645] Specific examples

[0646] As an example of a single elderly person, let's assume a scenario where user D lives alone. The server collects D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. In addition, an emotion analysis tool analyzes D's emotional state from the content of his calls and messages. For example, it learns that D's normal daily electricity consumption is 5 kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[0647] If one day D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. At the same time, the sentiment analysis means will detect from the content of past messages that D has recently been sending many negative messages. The notification means will automatically send a notice to D's family and local government that "an anomaly has been detected," allowing the relevant parties to respond quickly. If D is simply away from home for some reason and there is no problem, D can manually cancel the notification himself / herself.

[0648] An example of a prompt is, "D is a single elderly person living alone. His smartphone regularly sends call records and message history to a server, and his home's smart meter also sends real-time electricity and gas usage data. If he does not engage in these normal activities for several days, design a system that detects this as an abnormality and automatically sends a notification to his family and local government."

[0649] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, users do not need to install special equipment, so they can live safely using their existing living environment and infrastructure. Furthermore, by combining it with emotion analysis methods, even more advanced anomaly detection becomes possible.

[0650] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0651] Step 1: Data collection

[0652] The server collects data 24 hours a day. It receives data from mobile communication devices, fixed communication devices, and smart meters as input and stores this data as output. From mobile communication devices, it obtains call records, message history, and application usage data, and from fixed communication devices, it obtains landline call records. From smart meters, it receives electricity and gas consumption data in real time. Specifically, the server periodically obtains data from these devices through APIs.

[0653] Step 2: Data Preprocessing

[0654] The server preprocesses the collected data. It uses the data collected in step 1 as input and generates data with missing values ​​and corrected outliers as output. Specifically, if there is missing data, the server calculates the average value from past data to fill in the gaps. Also, if an abnormally high or low value is detected, it normalizes the value and corrects it to fall within an appropriate range. For example, if there is a gap in Mr. D's power consumption data, it fills in the gaps using the average value from a similar time period in the past.

[0655] Step 3: Anomaly detection

[0656] The device uses the generative AI model to detect anomalies. It uses the preprocessed data from step 2 as input and determines whether an anomaly has been detected as output. Specifically, the device trains the AI ​​model based on past data to learn normal usage patterns. It uses this model to analyze new data in real time and determine whether anomalies exist. For example, if Mr. D's electricity consumption is 0 kWh for three days, it will detect this as an anomaly.

[0657] Step 4: Sentiment Analysis

[0658] The server analyzes the user's emotional state using an emotion analysis method. It uses call content and message text acquired from the mobile communication device as input and determines the user's emotional state as output. Specifically, the server analyzes the call content and message text using a natural language processing algorithm to identify positive or negative emotions. For example, if Mr. D's messages over the past month have contained a lot of negative content, it determines his / her emotional state as negative.

[0659] Step 5: Notification of abnormalities

[0660] If the server detects an abnormality, it notifies the relevant parties. It uses the anomaly detection results and sentiment analysis results from steps 3 and 4 as input, and sends an abnormality notification via multiple means as output. Specifically, the server sends an abnormality alert to the user, family, local government, and property management company via phone, email, and SMS. The notification content includes details of the abnormality, the need for response, contact information, etc. For example, if an abnormality is detected for Mr. D, the server will send an email notification containing details of the abnormality and contact information.

[0661] In this way, the system will be able to detect abnormalities among single people at an early stage and notify relevant parties promptly.

[0662] (Application example 2)

[0663] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] Elderly people and single people living alone are isolated, and there is a need to detect and respond early when abnormalities in their health or psychological state occur. However, current support systems for isolated people lack real-time data collection and emotion analysis, making it difficult to detect abnormalities early. In addition, they cannot utilize standard infrastructure or devices, which poses challenges in terms of cost and operation.

[0665] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, means for acquiring data from vital sensors and wearable devices that collect location information, means for storing and encrypting data, as data preprocessing means, means for complementing missing values ​​in the collected data, and means for detecting and correcting abnormal values ​​in the collected data, as anomaly detection means, means for training a generative AI model that learns normal usage patterns using the acquired data, and means for analyzing new data in real time to detect anomalies, as emotion analysis means, means for analyzing the user's emotional state from image and audio data, and means for detecting anomalies based on the emotional state, as notification means, means for sending an alert to a notification destination when an anomaly is detected, means for notifying by multiple means, and means for providing a manual release function when no anomaly is confirmed. This makes it possible to detect anomalies and perform emotion analysis in real time based on collected data, enabling early detection of abnormal situations.

[0666] "Data collection means" refers to means for acquiring various data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices.

[0667] A "mobile communication device" is a device capable of mobile communication, such as a mobile phone or smartphone, that is used to obtain call logs, message history, and application usage data.

[0668] A "fixed communication device" is a telephone facility that is fixedly used in a home or office and is a means for obtaining call records.

[0669] A "smart meter" is a device that measures and transmits electricity and gas consumption data in real time.

[0670] A "wearable device" is a device that can be worn by a user and that acquires vital signs and location information.

[0671] "Means for storing and encrypting data" refers to the means for securely storing collected data and encrypting it to protect it from unauthorized access.

[0672] "Data preprocessing means" refers to means for complementing missing values ​​in collected data and detecting and correcting outliers.

[0673] A "generative AI model" is an artificial intelligence model that is trained to learn normal usage patterns using past data and detect anomalies in new data.

[0674] An "anomaly detection method" is a method for analyzing new data in real time using a generative AI model to detect anomalies.

[0675] The "emotion analysis means" is a means for analyzing the user's emotional state from image and audio data and detecting anomalies based on that state.

[0676] "Notification means" refers to a means for sending an alert when an abnormality is detected, notifying via multiple means, and providing a function for manually canceling notifications when no abnormality is confirmed.

[0677] This invention is a system for ensuring safety in the lives of elderly people and single people living alone. This system uses data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means to collect and analyze various types of user data, and can quickly notify the user when an anomaly is detected.

[0678] System configuration

[0679] Data collection methods

[0680] The server collects data by:

[0681] 1. Obtain call logs, message history, and application usage data from mobile communications devices.

[0682] 2. Obtain call records from fixed communication devices.

[0683] 3. Receive real-time consumption data from electricity and gas smart meters.

[0684] 4. Collect vital data such as heart rate and step count, as well as location information, from wearable devices.

[0685] Data preprocessing measures

[0686] The server completes missing values ​​in the collected data and detects and corrects outliers. For example, missing parts of electricity consumption data are completed with the average value from past data. Abnormally high or low values ​​are normalized.

[0687] Anomaly detection means

[0688] The anomaly detection method uses a generative AI model to learn normal usage patterns. Here, the generative AI model is trained from past data and analyzes new data in real time to detect anomalies. For example, if electricity consumption is 0 kWh for one consecutive week, it will be detected as an anomaly.

[0689] Emotion analysis means

[0690] The emotion analysis means uses the wearable device's camera and microphone to analyze the user's facial expressions and voice. The emotional state is determined from the collected data, and if a negative emotional state or a prolonged period of unresponsiveness is detected, it is recognized as an abnormality.

[0691] Notification means

[0692] As soon as the server detects an abnormality, it sends an alert to the user, distant family members, medical institutions, nursing care services, and other relevant parties. Alerts are sent via multiple means, including phone, email, and SMS. If no abnormalities are confirmed, notifications can be manually canceled.

[0693] Specific examples

[0694] Suppose a user wears smart glasses all day long. The server collects heart rate, step count, and location information through the glasses' sensors, and also uses a camera and microphone to capture the user's facial expressions and voice in real time. For example, if one day the user's heart rate suddenly spikes, their face looks sad, and they begin to stay indoors, the server will detect this as an abnormality. As a result, the server will instantly send an alert to their family or a medical institution saying, "An abnormality has been detected. Urgent investigation is required."

[0695] Prompt Sentence Examples

[0696] Check whether the user's heart rate is within the range of 80 to 100, and detect abnormalities if the number of steps is extremely low, or if the user's facial expression or voice indicates that they are sad. If these conditions are met, the system will automatically begin the process of sending an alert.

[0697] This system allows users to detect anomalies early using existing devices and infrastructure, without the need to install special equipment. In addition, by combining it with emotion analysis, more advanced anomaly detection is possible.

[0698] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0699] Step 1:

[0700] The server collects data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices, providing users' call records, message history, application usage data, electricity and gas consumption data, heart rate, step count, location information, etc. The input is raw data from various devices, and the output is an integrated data set stored on the server.

[0701] Step 2:

[0702] The server preprocesses the collected data. During this process, missing values ​​are filled in and outliers are corrected. For example, if some power consumption data is missing, it is filled in with the historical average value, and if an abnormally high heart rate is detected, it is normalized. The input is the collected raw data, and the output is the clean data that has been filled and corrected.

[0703] Step 3:

[0704] The server uses a generative AI model to train anomaly detection. This AI model is used to learn normal usage patterns based on past data. For example, electricity consumption, heart rate, and water, gas, and utility usage patterns are used as training data. The input is the clean data that has been complemented and corrected, and the output is the trained generative AI model.

[0705] Step 4:

[0706] The server inputs new data into the generative AI model in real time to detect anomalies. For example, if electricity consumption drops to 0 kWh for one consecutive week or if the heart rate rises abnormally, this will be detected as an anomaly. The input is new data acquired in real time, and the output is a judgment result as to whether or not there is an anomaly.

[0707] Step 5:

[0708] As an emotion analysis method, the server analyzes image and audio data acquired from the camera and microphone of the wearable device. This allows it to determine the user's emotions from their facial expressions and voice, and detect, for example, negative emotional states or prolonged periods of unresponsiveness. The input is image and audio data, and the output is the analysis result regarding the user's emotional state.

[0709] Step 6:

[0710] As soon as the server detects an anomaly, it sends an alert using a notification method. This notification is sent by multiple means, including phone, email, and SMS, and includes details of the anomaly and the need for action. The input is the anomaly detection judgment result and emotion analysis result, and the output is an alert notification to the user, distant family members, medical institutions, etc.

[0711] Step 7:

[0712] When a user receives a notification, they can manually dismiss the notification if they confirm that there is no abnormality. This function makes it possible to disable false alerts in the event of a false positive. The input is feedback from the user, and the output is dismissal of the notification.

[0713] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0714] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0715] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0716] [Third embodiment]

[0717] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0718] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0719] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0720] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0721] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0722] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0723] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0724] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0725] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0726] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0727] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0728] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0729] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[0730] Data collection methods

[0731] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from the smart meters.

[0732] Data preprocessing measures

[0733] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0734] Anomaly detection means

[0735] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0736] Notification means

[0737] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0738] Specific examples

[0739] Consider a scenario in which Mr. C, a single elderly person, lives alone. The server collects Mr. C's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that Mr. C's normal daily electricity consumption is 5 kWh and his gas usage is 2 cubic meters.

[0740] If one day, Person C does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. The notification method will automatically send a notification to Person C's family and local government that an "abnormality has been detected," allowing the relevant parties to respond quickly. If Person C is simply away from home for some reason and there is no problem, Person C can manually cancel the notification himself / herself.

[0741] This system allows family members living far away, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario.Furthermore, it is extremely convenient in that it does not require users to install special equipment and can efficiently utilize existing infrastructure.

[0742] The processing flow will be explained below.

[0743] Program processing steps

[0744] Data collection

[0745] Step 1:

[0746] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[0747] The server uses the mobile carrier's API to download the user's call and message history.

[0748] The server collects application usage data from users' smartphones and tablets.

[0749] Step 2:

[0750] A server retrieves call records from the fixed communication device.

[0751] The server retrieves call records through the fixed-line carrier's API and stores them in a database.

[0752] Step 3:

[0753] The server receives consumption data in real time from electricity and gas smart meters.

[0754] The server uses the APIs of electricity and gas suppliers to collect consumption data every hour.

[0755] Data Preprocessing

[0756] Step 4:

[0757] The server preprocesses the collected data.

[0758] The server detects missing values ​​in the dataset.

[0759] For example, the average electricity consumption for the past month is calculated and the missing values ​​are filled based on that.

[0760] Step 5:

[0761] The server detects and corrects outliers.

[0762] The server applies statistical techniques to the collected data to identify outliers.

[0763] For example, if call records or power consumption deviate from the normal range, they are corrected to within an appropriate range.

[0764] Anomaly detection

[0765] Step 6:

[0766] The device uses a generative AI model to learn normal usage patterns.

[0767] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[0768] The model is trained on typical call duration, message frequency, and electricity and gas consumption patterns.

[0769] Step 7:

[0770] The server analyzes real-time data and detects anomalies.

[0771] The server inputs the newly collected data into the generative AI model and analyzes it in real time for any abnormalities.

[0772] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[0773] notification

[0774] Step 8:

[0775] The server will notify you when an abnormality is detected.

[0776] The server automatically generates alerts to registered notification recipients (family, local government, property management company).

[0777] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[0778] Step 9:

[0779] The server provides notification in multiple ways.

[0780] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[0781] To reliably notify abnormalities using a plurality of notification means.

[0782] Step 10:

[0783] If the user confirms that there is no abnormality, the notification can be manually canceled.

[0784] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the link or button provided in the tool.

[0785] The server receives a opt-out request from the user to stop further notifications.

[0786] Through the above steps, the present invention can efficiently and quickly detect any abnormalities in the user and notify the relevant parties, thereby preventing the worst-case scenario from occurring.

[0787] Example 1

[0788] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0789] It is expected that the rapid detection of lifestyle anomalies from product usage data for single people will help prevent situations such as lonely deaths. However, existing systems have problems with insufficient data correction for missing data and outliers, low accuracy in learning normal usage patterns, and a lack of speed and accuracy in real-time anomaly detection and notification. There is also a need for a system that efficiently utilizes existing infrastructure and can be operated without burdening users.

[0790] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0791] In this invention, the server includes means for acquiring call logs, message histories, and application usage data from communication devices, means for acquiring call logs from fixed communication devices, means for receiving consumption data from power supplies and fuel supplies, means for storing and encrypting data, means for completing missing values ​​in the collected data, means for detecting and correcting abnormal values ​​in the collected data, means for training a generative AI model that learns normal usage patterns using the collected data, means for analyzing new data in real time to detect anomalies, means for sending an alert to the communication partner when an anomaly is detected, means for notifying via multiple means, means for providing a manual release function when no anomalies are confirmed, and means for generating a prompt to send a notification when the collected data deviates from the normal pattern for a certain period of time. This improves the accuracy and real-time nature of anomaly detection, enables appropriate notifications to be sent promptly to relevant parties, and prevents worst-case scenarios from occurring.

[0792] The "data collection means" is a means for collecting various data from users' communication devices, fixed communication devices, power supply devices, and fuel supply devices.

[0793] "Communications Device" refers to a user's mobile device or fixed communications equipment from which call logs, message history, and application usage data are collected.

[0794] "Power supply devices and fuel supply devices" refer to devices such as smart meters installed in users' homes that measure the amount of electricity and gas consumed.

[0795] The "data preprocessing means" is a means for complementing missing values ​​in collected data and detecting and correcting outliers.

[0796] A "generative AI model" is an artificial intelligence model that uses previously collected data to learn normal usage patterns and detect abnormal patterns that deviate from the normal range.

[0797] "Real-time anomaly detection means" is a means for analyzing new data in real time and immediately detecting anomalies.

[0798] "Notification means" refers to a means of quickly sending an alert to relevant parties when an abnormality is detected, and includes methods of sending an alert such as telephone, email, and SMS.

[0799] The "manual cancellation function" is a function that allows the user to manually cancel the notification if they confirm that there is no abnormality.

[0800] The "means for generating a prompt" refers to a means for generating a prompt to send appropriate notifications or alerts when collected data deviates from a normal pattern for a certain period of time.

[0801] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[0802] Data collection methods

[0803] The server collects data 24 hours a day from users' communication devices, fixed communication devices, and smart meters for electricity and fuel supply equipment. Call logs, message history, and application usage data are obtained from communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and fuel consumption data is received in real time from smart meters. The hardware used includes: smart meters (common examples, and can be from a variety of manufacturers), mobile communication devices (e.g., various smartphones and tablets), and fixed communication devices (e.g., various landline phones). The software uses a data collection API.

[0804] Data preprocessing measures

[0805] The server preprocesses the collected data. Specifically, it complements missing values ​​and detects and corrects outliers. For example, if there are missing power consumption data, it calculates the average value from past data and complements the missing parts. Also, if abnormally high or low values ​​are detected, they are corrected to within an appropriate range using a normalization algorithm. The software used includes data preprocessing libraries (e.g., Pandas and NumPy).

[0806] Anomaly detection means

[0807] The device uses a generative AI model to learn normal usage patterns. This model is trained based on previously collected data, learning mobile communication device usage time and electricity and gas consumption patterns. As new data is generated in real time, the server inputs this data into the AI ​​model to determine whether there are any anomalies. Specifically, if electricity consumption drops to 0 kWh for a week, this is detected as an anomaly. The software used includes generative AI models (e.g., TensorFlow and PyTorch) and anomaly detection algorithms.

[0808] Notification means

[0809] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local governments, management companies, and other relevant parties. Notifications are sent via phone, email, or SMS, and include details of the abnormality, the need for action, and contact information. Users can also manually cancel notifications if they confirm that there is no problem. Software used includes notification management systems (e.g., Twilio API, Sendgrid).

[0810] Specific examples

[0811] Consider a single elderly person living alone. The server collects the user's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that a user's normal lifestyle pattern involves daily electricity consumption of 5 kWh and gas consumption of 2 cubic meters. If the user does not use their mobile phone at all on one day and their electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. A notification system automatically sends a notice to the user's family and local government that an anomaly has been detected, allowing the relevant parties to respond promptly.

[0812] This system allows relevant parties to quickly identify abnormalities and prevent the worst-case scenario from occurring. Furthermore, it is extremely convenient in that it does not require users to install additional special equipment, and allows the efficient use of existing infrastructure.

[0813] Prompt Sentence Examples

[0814] "Create a prompt to detect an anomaly and send a notification if the user's call records, electricity consumption, or gas usage deviate from normal patterns for more than three days. Specifically, detect an anomaly when electricity consumption is less than 5 kWh or gas usage is less than 2 cubic meters."

[0815] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0816] Step 1:

[0817] The server connects to the communication device. The server retrieves call records, message history, and application usage data from the user's communication device. Specifically, it requests data using the communication device's API and stores the received data in a database. The input is the data request from the communication device, and the output is the retrieved usage data.

[0818] Step 2:

[0819] The server connects to the fixed communication device. The server acquires call records from the fixed communication device. Specifically, the server reads the log of the fixed communication device, extracts the call records, and stores them in a database. The input is a data request from the fixed communication device, and the output is the acquired call records.

[0820] Step 3:

[0821] The server connects to the smart meters of the electricity supply equipment and fuel supply equipment. The server obtains electricity and gas consumption data in real time. Specifically, it obtains the data using the smart meter's API and stores it in a database. The input is a data request from the smart meter, and the output is the received consumption data.

[0822] Step 4:

[0823] The server retrieves the collected data from the database. First, it detects missing values ​​and then calculates the average value based on past data to fill in the gaps. Specifically, it finds the missing values ​​in the collected data and fills in the gaps with the average value calculated from past data. The input is the collected data from the database, and the output is the filled data.

[0824] Step 5:

[0825] The server performs range checks on the data, detects abnormally high or low values, and applies a normalization algorithm to correct them. Specifically, it finds data that deviates from the normal range and converts it to fit within a predefined range. The input is the data with missing values ​​imputed, and the output is the data with the outliers corrected.

[0826] Step 6:

[0827] The device trains the generative AI model. The device uses past data as input to train the model and have it learn normal usage patterns. Specifically, a training dataset is input into the AI ​​model, and learning progresses epoch by epoch. The input is past collected data, and the output is a trained generative AI model.

[0828] Step 7:

[0829] The server inputs new data into the generative AI model in real time. The server uses the model to detect anomalies. Specifically, it inputs new data into the model, calculates an anomaly score, and determines it as an anomaly if the anomaly score exceeds a threshold. The input is new data in real time, and the output is the anomaly detection result.

[0830] Step 8:

[0831] As soon as the server detects an anomaly, it automatically sends an alert to the user, distant family members, local government, and management company via notification means. Specifically, based on the anomaly detection results, it uses a notification API to send alerts by phone, email, and SMS. The input is the anomaly detection result, and the output is the sent alert notification.

[0832] Step 9:

[0833] After receiving a notification, the user can manually cancel the notification if necessary. Specifically, the user confirms that there is no abnormality through the smartphone app and sends a cancellation request to the server. When the server receives this cancellation request, it cancels the notification state. The input is the cancellation request from the user, and the output is the canceled notification state.

[0834] (Application example 1)

[0835] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0836] In modern society, problems such as lonely deaths of single people and bachelors are on the rise. To prevent such incidents, a system is needed to quickly detect abnormalities in daily life and notify relevant parties. However, existing systems tend to be slow to respond because they are incomplete in data collection and anomaly detection. Furthermore, manual monitoring and the need to disable notifications place a heavy burden on users. Therefore, efficient and automated security services are needed.

[0837] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0838] In this invention, the server includes a means for acquiring communication history, message history, and application usage data from communication devices, a means for acquiring communication history from fixed communication devices, and a means for receiving energy consumption data. This enables automatic detection of anomalies in the daily lives of single people and bachelors and prompt notification. Furthermore, by performing missing value imputation and outlier correction in data, real-time anomaly detection using a generative AI model that learns normal usage patterns, notification via multiple means, automatic message sending, and continuous monitoring of infrastructure equipment usage, the system reduces the burden on users and provides a highly accurate monitoring and notification system.

[0839] 1. "Telecommunications equipment" is a general term for equipment used by users to communicate, including mobile and fixed communications equipment.

[0840] 2. "Mobile communication equipment" refers to portable communication devices such as mobile phones and smartphones.

[0841] 3. "Fixed communications equipment" means a device used for communication at a fixed location, such as a landline telephone or desktop computer.

[0842] 4. "Communication history" refers to data that includes records of calls made by the user and the history of message exchanges.

[0843] 5. "Message History" refers to the historical data of text messages and emails sent and received by a User.

[0844] 6. "Application Usage Data" means data regarding the usage of applications installed by a User.

[0845] 7. "Energy Consumption Data" means data relating to energy consumption, including electricity and gas.

[0846] 8. "Data preprocessing" refers to the process of filling in missing values ​​in collected data and detecting and correcting outliers.

[0847] 9. "Missing value imputation" refers to the process of appropriately completing missing values ​​in a dataset.

[0848] 10. "Outlier correction" refers to the process of correcting abnormally high or low values ​​in a data set to fall within an appropriate range.

[0849] 11. A “generative AI model” is a model trained for anomaly detection using machine learning or deep learning.

[0850] 12. “Training” refers to the process by which a generative AI model learns normal usage patterns using historical data.

[0851] 13. "Real-time analytics" refers to the rapid processing of new data as it is collected and the immediate detection of anomalies.

[0852] 14. "Automatic Notification" refers to the process by which the server automatically sends alerts to relevant parties when an anomaly is detected.

[0853] 15. "Manual Cancellation Function" refers to the function that allows the user to manually cancel an alert notification when they confirm that there are no abnormalities.

[0854] 16. "Emergency Contacts" means a list of contacts to be notified in the event of an emergency.

[0855] 17. "Infrastructure equipment" refers to equipment that supplies electricity, gas, and other supplies necessary for daily life and business.

[0856] The system of the present invention uses communication devices, fixed communication devices, and energy consumption data to detect abnormalities and notify relevant parties as necessary. Specifically, the system is configured as follows.

[0857] Data collection methods

[0858] The server retrieves communication history, message history, and application usage data from mobile communication devices, including smartphones and other mobile devices. It also retrieves communication history from fixed communication devices and receives real-time electricity and gas consumption data from energy meters. All data is securely stored and encrypted.

[0859] Data preprocessing measures

[0860] Missing values ​​are first interpolated from the collected data. Then, outliers are detected and corrected. For example, if there are missing power consumption data, the data is interpolated using the average value from past data. Furthermore, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range. This process uses data processing libraries such as Pandas and NumPy.

[0861] Anomaly detection means

[0862] The device uses a generative AI model to learn normal usage patterns. This model is trained based on past data. The trained model (for example, a Keras model) is fed new data in real time and determines whether there are any abnormalities. The server will recognize an abnormality, for example, if power consumption drops to 0 kWh for a week. The following prompts can be used to train the "generative AI model."

[0863] Example prompt for a generative AI model:

[0864] text

[0865] Training generative AI models to detect anomalies based on electricity, gas, and smartphone usage data

[0866] train_model(input_data, labels)

[0867] Notification means

[0868] As soon as the server detects an abnormality, it automatically notifies the user and designated emergency contacts. Notification methods include phone, email, and SMS. For example, notifications can be sent using Twilio or SMTP. Information sent includes details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0869] Example system operation

[0870] Consider the case of monitoring the lifestyle of an elderly person living alone. In this case, the server collects mobile communication history, message history, app usage data, and energy consumption data 24 hours a day. A generative AI model that learns normal lifestyle patterns detects abnormal data in real time. When an abnormality is detected, the server automatically sends a notification to emergency contacts, enabling a prompt response. A manual deactivation function allows the user to deactivate notifications themselves.

[0871] The above system will enable efficient monitoring of the lives of single people and bachelors, quickly detect any abnormalities, and notify the relevant parties, thereby making it possible to prevent problems such as lonely deaths.

[0872] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0873] Step 1: Data collection

[0874] The server retrieves communication history, message history, and application usage data from communication devices. At the same time, it receives communication history from fixed communication devices and real-time electricity and gas consumption data from energy meters. All of this data is securely stored and encrypted. Input is data from various sensors and devices, and output is data stored in encrypted data storage.

[0875] Step 2: Data Preprocessing

[0876] The server first fills in missing values ​​in the collected data. For example, if there are missing power consumption data, this involves filling in the data using the average value from past data. Next, it detects and corrects outliers. If abnormally high or low values ​​are detected, they are normalized to fit within an appropriate range. The input is data obtained from encrypted data storage, and the output is the data that has been filled and normalized.

[0877] Step 3: Training the generative AI model

[0878] The device trains a generative AI model to learn normal usage patterns using preprocessed data. Specifically, it uses past data to learn normal patterns and creates a model to detect abnormal patterns. The prompt statement is "Train a generative AI model that detects anomalies based on electricity, gas, and smartphone usage data." The input is preprocessed data, and the output is a model that has learned normal patterns.

[0879] Step 4: Detect anomalies in real-time data

[0880] Using the generative AI model, the server analyzes new data in real time and determines whether there are any abnormalities. For example, if power consumption drops to 0 kWh, it will recognize this as an abnormality. The input is new data collected in real time, and the output is a flag indicating whether there are any abnormalities.

[0881] Step 5: Notification of abnormalities

[0882] If an anomaly is detected, the server automatically notifies the user and the designated emergency contacts. Notification methods include phone, email, and SMS. This notification is performed using Twilio and SMTP. The input is the anomaly detection flag and a list of emergency contacts, and the output is the notification message sent.

[0883] Step 6: Manual release function

[0884] If the user or designated emergency contact confirms that there are no abnormalities, the server provides a function to manually cancel the notification. Specifically, the user logs in to the system and uses a GUI to confirm that there are no abnormalities. The input is the user's confirmation operation, and the output is the notification cancellation status.

[0885] The above processing steps provide a system that can efficiently monitor the lives of single people and bachelors, quickly detect abnormalities, and notify relevant parties, thereby preventing problems such as lonely deaths.

[0886] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0887] The present invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, an emotion engine, and a notification means.

[0888] Data collection methods

[0889] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from smart meters.

[0890] Data preprocessing measures

[0891] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0892] Anomaly detection means

[0893] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0894] Emotion Engine

[0895] The emotion engine recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion engine also works in conjunction with anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of unresponsiveness can be recognized as an abnormality.

[0896] Notification means

[0897] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0898] Specific examples

[0899] Consider a scenario in which Mr. D, a single elderly person, lives alone. The server collects Mr. D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. The emotion engine also analyzes Mr. D's emotional state from the content of his calls and messages. For example, it learns that a typical lifestyle pattern for Mr. D is that his daily electricity consumption is 5kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[0900] If one day, Person D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. At the same time, the emotion engine will detect from the content of past messages that Person D has recently been sending many negative messages. The notification method will automatically send a notice to Person D's family and local government that "an abnormality has been detected," allowing the relevant parties to respond quickly. If Person D is simply away from home for some reason and there is no problem, Person D can manually cancel the notification himself / herself.

[0901] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, it offers great convenience in that it does not require users to install special equipment and can efficiently utilize existing infrastructure. Furthermore, by combining it with an emotion engine, more advanced anomaly detection is possible, taking into account the user's psychological state.

[0902] The processing flow will be explained below.

[0903] Program processing steps

[0904] Data collection

[0905] Step 1:

[0906] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[0907] The server uses the mobile carrier's API to automatically download the user's call and message history.

[0908] The server periodically collects application usage data from users' smartphones and tablets.

[0909] Step 2:

[0910] A server retrieves call records from the fixed communication device.

[0911] The server retrieves call records through the fixed line carrier's API and stores them in a secure database.

[0912] Step 3:

[0913] The server receives consumption data in real time from electricity and gas smart meters.

[0914] The server uses the APIs of electricity and gas suppliers to collect, encrypt, and store consumption data every hour.

[0915] Data Preprocessing

[0916] Step 4:

[0917] The server preprocesses the collected data.

[0918] The server detects missing values ​​in the dataset and imputes them using statistical methods.

[0919] For example, if there is a gap in the electricity consumption data, it will be supplemented based on the average usage amount over the past month.

[0920] Step 5:

[0921] The server detects and corrects outliers.

[0922] The server uses statistical techniques for each item in the dataset to identify outliers.

[0923] For example, if the power consumption deviates from the normal range, it is corrected to fall within an appropriate range.

[0924] Anomaly detection

[0925] Step 6:

[0926] The device uses a generative AI model to learn normal usage patterns.

[0927] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[0928] The model learns typical call duration, message sending frequency, and electricity and gas consumption patterns.

[0929] Step 7:

[0930] The server analyzes real-time data and detects anomalies.

[0931] The server inputs the newly collected data into the generative AI model, which analyzes it in real time for any abnormalities.

[0932] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[0933] Emotion analysis

[0934] Step 8:

[0935] The server uses an emotion engine to analyze the user's emotional state.

[0936] The server performs text analysis of call content and message history to identify positive and negative sentiment.

[0937] For example, if the message content is negative, the emotion engine will recognize that.

[0938] Step 9:

[0939] The server links the results of the emotion engine with the anomaly detection means.

[0940] The server feeds back the emotional state obtained from the emotion engine to the generative AI model, improving the accuracy of anomaly detection.

[0941] If negative emotional states persist, they are recognized as psychological abnormalities.

[0942] notification

[0943] Step 10:

[0944] The server will notify you when an abnormality is detected.

[0945] The server automatically generates and sends alerts to registered notification recipients (family, local government, property management company).

[0946] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[0947] Step 11:

[0948] The server provides notification in multiple ways.

[0949] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[0950] To reliably notify abnormalities using a plurality of notification means.

[0951] Step 12:

[0952] If the user confirms that there is no abnormality, the notification can be manually canceled.

[0953] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the provided link or button.

[0954] The server receives a opt-out request from the user to stop further notifications.

[0955] Through each of the above steps, the system can detect abnormalities by integrating users' product usage data and emotional state, and notify relevant parties quickly and reliably, thereby preventing serious problems such as lonely deaths.

[0956] Example 2

[0957] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0958] There is a need to detect abnormalities in the lifestyles of single people early on and prevent serious incidents such as solitary death. In particular, when elderly people and others live alone, it is important to respond to sudden changes in their health condition or living situation, but current systems have the problem of not being able to detect such abnormalities quickly and accurately.

[0959] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, and means for storing and encrypting data. This makes it possible to centrally collect information from a wide range of data sources.

[0960] The data preprocessing means also includes means for complementing missing values ​​in the collected data and means for detecting and correcting abnormal values ​​in the collected data, thereby improving the accuracy of the data and the reliability of anomaly detection.

[0961] Furthermore, the system includes an anomaly detection means that uses acquired data to train a generative AI model that learns normal usage patterns, and a means that analyzes new data in real time to detect anomalies, and an emotion analysis means that analyzes call content and message text acquired from the mobile communication device to determine the user's emotional state, thereby making it possible to detect not only physical anomalies but also psychological anomalies.

[0962] Finally, the notification means includes a means for sending an alert to the notification destination when an abnormality is detected, a means for sending notifications by multiple means, and a means for providing a manual cancellation function when no abnormality is confirmed, which enables a quick and appropriate response.

[0963] "Data collection means" refers to means for acquiring various types of data from mobile communication devices, fixed communication devices, electricity and gas smart meters, etc.

[0964] A "mobile communications device" is a mobile communications device, such as a smartphone or tablet, that provides call logs, message history, application usage data, and the like.

[0965] A "fixed communications device" is a fixed communications device such as a landline telephone or router that provides call records.

[0966] A "smart meter" is a measuring device that measures and provides electricity and gas consumption data in real time.

[0967] "Data preprocessing means" refers to means for complementing missing values ​​and detecting and correcting outliers in acquired data.

[0968] "Missing value imputation" is a process in which, when there are gaps in the collected data, the gaps are filled in using past data or average values.

[0969] "Outlier detection and correction" is a process that detects abnormal values ​​in data and corrects them to an appropriate range using methods such as normalization.

[0970] An "anomaly detection method" is a method that uses acquired data to train a generative AI model that learns normal usage patterns and analyzes new data in real time to detect anomalies.

[0971] A "generative AI model" is a model that uses machine learning algorithms such as deep learning to learn patterns in data and detect anomalies.

[0972] "Real-time analysis" is an analytical method that processes new data instantly and immediately determines whether or not there are any abnormalities.

[0973] The "emotion analysis means" is a means for analyzing call content and message text obtained from a mobile communication device to determine the user's emotional state.

[0974] The "notification means" is a means for sending an alert to a notification destination when an abnormality is detected.

[0975] The "manual cancellation function" is a function that allows the user to manually cancel the notification when the user confirms that there is no abnormality.

[0976] This invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and for preventing situations such as solitary death. This system includes data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means.

[0977] Data collection methods

[0978] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. It obtains call logs, message histories, and application usage data from mobile communication devices, and call logs from fixed communication devices. It also receives real-time electricity and gas consumption data from the smart meters.

[0979] Data preprocessing measures

[0980] The server performs missing value interpolation and outlier detection / correction on the collected data. Specifically, if there are missing power consumption data, it interpolates using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[0981] Anomaly detection means

[0982] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[0983] Emotion analysis means

[0984] The emotion analysis means recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion analysis means works in conjunction with the anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of no response can be recognized as an abnormality.

[0985] Notification means

[0986] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[0987] Specific examples

[0988] As an example of a single elderly person, let's assume a scenario where user D lives alone. The server collects D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. In addition, an emotion analysis tool analyzes D's emotional state from the content of his calls and messages. For example, it learns that D's normal daily electricity consumption is 5 kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[0989] If one day D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. At the same time, the sentiment analysis means will detect from the content of past messages that D has recently been sending many negative messages. The notification means will automatically send a notice to D's family and local government that "an anomaly has been detected," allowing the relevant parties to respond quickly. If D is simply away from home for some reason and there is no problem, D can manually cancel the notification himself / herself.

[0990] An example of a prompt is, "D is a single elderly person living alone. His smartphone regularly sends call records and message history to a server, and his home's smart meter also sends real-time electricity and gas usage data. If he does not engage in these normal activities for several days, design a system that detects this as an abnormality and automatically sends a notification to his family and local government."

[0991] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, users do not need to install special equipment, so they can live safely using their existing living environment and infrastructure. Furthermore, by combining it with emotion analysis methods, even more advanced anomaly detection becomes possible.

[0992] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0993] Step 1: Data collection

[0994] The server collects data 24 hours a day. It receives data from mobile communication devices, fixed communication devices, and smart meters as input and stores this data as output. From mobile communication devices, it obtains call records, message history, and application usage data, and from fixed communication devices, it obtains landline call records. From smart meters, it receives electricity and gas consumption data in real time. Specifically, the server periodically obtains data from these devices through APIs.

[0995] Step 2: Data Preprocessing

[0996] The server preprocesses the collected data. It uses the data collected in step 1 as input and generates data with missing values ​​and corrected outliers as output. Specifically, if there is missing data, the server calculates the average value from past data to fill in the gaps. Also, if an abnormally high or low value is detected, it normalizes the value and corrects it to fall within an appropriate range. For example, if there is a gap in Mr. D's power consumption data, it fills in the gaps using the average value from a similar time period in the past.

[0997] Step 3: Anomaly detection

[0998] The device uses the generative AI model to detect anomalies. It uses the preprocessed data from step 2 as input and determines whether an anomaly has been detected as output. Specifically, the device trains the AI ​​model based on past data to learn normal usage patterns. It uses this model to analyze new data in real time and determine whether anomalies exist. For example, if Mr. D's electricity consumption is 0 kWh for three days, it will detect this as an anomaly.

[0999] Step 4: Sentiment Analysis

[1000] The server analyzes the user's emotional state using an emotion analysis method. It uses call content and message text acquired from the mobile communication device as input and determines the user's emotional state as output. Specifically, the server analyzes the call content and message text using a natural language processing algorithm to identify positive or negative emotions. For example, if Mr. D's messages over the past month have contained a lot of negative content, it determines his / her emotional state as negative.

[1001] Step 5: Notification of abnormalities

[1002] If the server detects an abnormality, it notifies the relevant parties. It uses the anomaly detection results and sentiment analysis results from steps 3 and 4 as input, and sends an abnormality notification via multiple means as output. Specifically, the server sends an abnormality alert to the user, family, local government, and property management company via phone, email, and SMS. The notification content includes details of the abnormality, the need for response, contact information, etc. For example, if an abnormality is detected for Mr. D, the server will send an email notification containing details of the abnormality and contact information.

[1003] In this way, the system will be able to detect abnormalities among single people at an early stage and notify relevant parties promptly.

[1004] (Application example 2)

[1005] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1006] Elderly people and single people living alone are isolated, and there is a need to detect and respond early when abnormalities in their health or psychological state occur. However, current support systems for isolated people lack real-time data collection and emotion analysis, making it difficult to detect abnormalities early. In addition, they cannot utilize standard infrastructure or devices, which poses challenges in terms of cost and operation.

[1007] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, means for acquiring data from vital sensors and wearable devices that collect location information, means for storing and encrypting data, as data preprocessing means, means for complementing missing values ​​in the collected data, and means for detecting and correcting abnormal values ​​in the collected data, as anomaly detection means, means for training a generative AI model that learns normal usage patterns using the acquired data, and means for analyzing new data in real time to detect anomalies, as emotion analysis means, means for analyzing the user's emotional state from image and audio data, and means for detecting anomalies based on the emotional state, as notification means, means for sending an alert to a notification destination when an anomaly is detected, means for notifying by multiple means, and means for providing a manual release function when no anomaly is confirmed. This makes it possible to detect anomalies and perform emotion analysis in real time based on collected data, enabling early detection of abnormal situations.

[1008] "Data collection means" refers to means for acquiring various data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices.

[1009] A "mobile communication device" is a device capable of mobile communication, such as a mobile phone or smartphone, that is used to obtain call logs, message history, and application usage data.

[1010] A "fixed communication device" is a telephone facility that is fixedly used in a home or office and is a means for obtaining call records.

[1011] A "smart meter" is a device that measures and transmits electricity and gas consumption data in real time.

[1012] A "wearable device" is a device that can be worn by a user and that acquires vital signs and location information.

[1013] "Means for storing and encrypting data" refers to the means for securely storing collected data and encrypting it to protect it from unauthorized access.

[1014] "Data preprocessing means" refers to means for complementing missing values ​​in collected data and detecting and correcting outliers.

[1015] A "generative AI model" is an artificial intelligence model that is trained to learn normal usage patterns using past data and detect anomalies in new data.

[1016] An "anomaly detection method" is a method for analyzing new data in real time using a generative AI model to detect anomalies.

[1017] The "emotion analysis means" is a means for analyzing the user's emotional state from image and audio data and detecting anomalies based on that state.

[1018] "Notification means" refers to a means for sending an alert when an abnormality is detected, notifying via multiple means, and providing a function for manually canceling notifications when no abnormality is confirmed.

[1019] This invention is a system for ensuring safety in the lives of elderly people and single people living alone. This system uses data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means to collect and analyze various types of user data, and can quickly notify the user when an anomaly is detected.

[1020] System configuration

[1021] Data collection methods

[1022] The server collects data by:

[1023] 1. Obtain call logs, message history, and application usage data from mobile communications devices.

[1024] 2. Obtain call records from fixed communication devices.

[1025] 3. Receive real-time consumption data from electricity and gas smart meters.

[1026] 4. Collect vital data such as heart rate and step count, as well as location information, from wearable devices.

[1027] Data preprocessing measures

[1028] The server completes missing values ​​in the collected data and detects and corrects outliers. For example, missing parts of electricity consumption data are completed with the average value from past data. Abnormally high or low values ​​are normalized.

[1029] Anomaly detection means

[1030] The anomaly detection method uses a generative AI model to learn normal usage patterns. Here, the generative AI model is trained from past data and analyzes new data in real time to detect anomalies. For example, if electricity consumption is 0 kWh for one consecutive week, it will be detected as an anomaly.

[1031] Emotion analysis means

[1032] The emotion analysis means uses the wearable device's camera and microphone to analyze the user's facial expressions and voice. The emotional state is determined from the collected data, and if a negative emotional state or a prolonged period of unresponsiveness is detected, it is recognized as an abnormality.

[1033] Notification means

[1034] As soon as the server detects an abnormality, it sends an alert to the user, distant family members, medical institutions, nursing care services, and other relevant parties. Alerts are sent via multiple means, including phone, email, and SMS. If no abnormalities are confirmed, notifications can be manually canceled.

[1035] Specific examples

[1036] Suppose a user wears smart glasses all day long. The server collects heart rate, step count, and location information through the glasses' sensors, and also uses a camera and microphone to capture the user's facial expressions and voice in real time. For example, if one day the user's heart rate suddenly spikes, their face looks sad, and they begin to stay indoors, the server will detect this as an abnormality. As a result, the server will instantly send an alert to their family or a medical institution saying, "An abnormality has been detected. Urgent investigation is required."

[1037] Prompt Sentence Examples

[1038] Check whether the user's heart rate is within the range of 80 to 100, and detect abnormalities if the number of steps is extremely low, or if the user's facial expression or voice indicates that they are sad. If these conditions are met, the system will automatically begin the process of sending an alert.

[1039] This system allows users to detect anomalies early using existing devices and infrastructure, without the need to install special equipment. In addition, by combining it with emotion analysis, more advanced anomaly detection is possible.

[1040] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1041] Step 1:

[1042] The server collects data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices, providing users' call records, message history, application usage data, electricity and gas consumption data, heart rate, step count, location information, etc. The input is raw data from various devices, and the output is an integrated data set stored on the server.

[1043] Step 2:

[1044] The server preprocesses the collected data. During this process, missing values ​​are filled in and outliers are corrected. For example, if some power consumption data is missing, it is filled in with the historical average value, and if an abnormally high heart rate is detected, it is normalized. The input is the collected raw data, and the output is the clean data that has been filled and corrected.

[1045] Step 3:

[1046] The server uses a generative AI model to train anomaly detection. This AI model is used to learn normal usage patterns based on past data. For example, electricity consumption, heart rate, and water, gas, and utility usage patterns are used as training data. The input is the clean data that has been complemented and corrected, and the output is the trained generative AI model.

[1047] Step 4:

[1048] The server inputs new data into the generative AI model in real time to detect anomalies. For example, if electricity consumption drops to 0 kWh for one consecutive week or if the heart rate rises abnormally, this will be detected as an anomaly. The input is new data acquired in real time, and the output is a judgment result as to whether or not there is an anomaly.

[1049] Step 5:

[1050] As an emotion analysis method, the server analyzes image and audio data acquired from the camera and microphone of the wearable device. This allows it to determine the user's emotions from their facial expressions and voice, and detect, for example, negative emotional states or prolonged periods of unresponsiveness. The input is image and audio data, and the output is the analysis result regarding the user's emotional state.

[1051] Step 6:

[1052] As soon as the server detects an anomaly, it sends an alert using a notification method. This notification is sent by multiple means, including phone, email, and SMS, and includes details of the anomaly and the need for action. The input is the anomaly detection judgment result and emotion analysis result, and the output is an alert notification to the user, distant family members, medical institutions, etc.

[1053] Step 7:

[1054] When a user receives a notification, they can manually dismiss the notification if they confirm that there is no abnormality. This function makes it possible to disable false alerts in the event of a false positive. The input is feedback from the user, and the output is dismissal of the notification.

[1055] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1056] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1057] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1058] [Fourth embodiment]

[1059] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1060] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1061] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1062] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1063] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1064] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1065] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1066] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1067] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1068] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1069] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1070] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1071] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1072] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[1073] Data collection methods

[1074] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from the smart meters.

[1075] Data preprocessing measures

[1076] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[1077] Anomaly detection means

[1078] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[1079] Notification means

[1080] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[1081] Specific examples

[1082] Consider a scenario in which Mr. C, a single elderly person, lives alone. The server collects Mr. C's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that Mr. C's normal daily electricity consumption is 5 kWh and his gas usage is 2 cubic meters.

[1083] If one day, Person C does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. The notification method will automatically send a notification to Person C's family and local government that an "abnormality has been detected," allowing the relevant parties to respond quickly. If Person C is simply away from home for some reason and there is no problem, Person C can manually cancel the notification himself / herself.

[1084] This system allows family members living far away, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario.Furthermore, it is extremely convenient in that it does not require users to install special equipment and can efficiently utilize existing infrastructure.

[1085] The processing flow will be explained below.

[1086] Program processing steps

[1087] Data collection

[1088] Step 1:

[1089] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[1090] The server uses the mobile carrier's API to download the user's call and message history.

[1091] The server collects application usage data from users' smartphones and tablets.

[1092] Step 2:

[1093] A server retrieves call records from the fixed communication device.

[1094] The server retrieves call records through the fixed-line carrier's API and stores them in a database.

[1095] Step 3:

[1096] The server receives consumption data in real time from electricity and gas smart meters.

[1097] The server uses the APIs of electricity and gas suppliers to collect consumption data every hour.

[1098] Data Preprocessing

[1099] Step 4:

[1100] The server preprocesses the collected data.

[1101] The server detects missing values ​​in the dataset.

[1102] For example, the average electricity consumption for the past month is calculated and the missing values ​​are filled based on that.

[1103] Step 5:

[1104] The server detects and corrects outliers.

[1105] The server applies statistical techniques to the collected data to identify outliers.

[1106] For example, if call records or power consumption deviate from the normal range, they are corrected to within an appropriate range.

[1107] Anomaly detection

[1108] Step 6:

[1109] The device uses a generative AI model to learn normal usage patterns.

[1110] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[1111] The model is trained on typical call duration, message frequency, and electricity and gas consumption patterns.

[1112] Step 7:

[1113] The server analyzes real-time data and detects anomalies.

[1114] The server inputs the newly collected data into the generative AI model and analyzes it in real time for any abnormalities.

[1115] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[1116] notification

[1117] Step 8:

[1118] The server will notify you when an abnormality is detected.

[1119] The server automatically generates alerts to registered notification recipients (family, local government, property management company).

[1120] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[1121] Step 9:

[1122] The server provides notification in multiple ways.

[1123] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[1124] To reliably notify abnormalities using a plurality of notification means.

[1125] Step 10:

[1126] If the user confirms that there is no abnormality, the notification can be manually canceled.

[1127] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the link or button provided in the tool.

[1128] The server receives a opt-out request from the user to stop further notifications.

[1129] Through the above steps, the present invention can efficiently and quickly detect any abnormalities in the user and notify the relevant parties, thereby preventing the worst-case scenario from occurring.

[1130] Example 1

[1131] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1132] It is expected that the rapid detection of lifestyle anomalies from product usage data for single people will help prevent situations such as lonely deaths. However, existing systems have problems with insufficient data correction for missing data and outliers, low accuracy in learning normal usage patterns, and a lack of speed and accuracy in real-time anomaly detection and notification. There is also a need for a system that efficiently utilizes existing infrastructure and can be operated without burdening users.

[1133] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1134] In this invention, the server includes means for acquiring call logs, message histories, and application usage data from communication devices, means for acquiring call logs from fixed communication devices, means for receiving consumption data from power supplies and fuel supplies, means for storing and encrypting data, means for completing missing values ​​in the collected data, means for detecting and correcting abnormal values ​​in the collected data, means for training a generative AI model that learns normal usage patterns using the collected data, means for analyzing new data in real time to detect anomalies, means for sending an alert to the communication partner when an anomaly is detected, means for notifying via multiple means, means for providing a manual release function when no anomalies are confirmed, and means for generating a prompt to send a notification when the collected data deviates from the normal pattern for a certain period of time. This improves the accuracy and real-time nature of anomaly detection, enables appropriate notifications to be sent promptly to relevant parties, and prevents worst-case scenarios from occurring.

[1135] The "data collection means" is a means for collecting various data from users' communication devices, fixed communication devices, power supply devices, and fuel supply devices.

[1136] "Communications Device" refers to a user's mobile device or fixed communications equipment from which call logs, message history, and application usage data are collected.

[1137] "Power supply devices and fuel supply devices" refer to devices such as smart meters installed in users' homes that measure the amount of electricity and gas consumed.

[1138] The "data preprocessing means" is a means for complementing missing values ​​in collected data and detecting and correcting outliers.

[1139] A "generative AI model" is an artificial intelligence model that uses previously collected data to learn normal usage patterns and detect abnormal patterns that deviate from the normal range.

[1140] "Real-time anomaly detection means" is a means for analyzing new data in real time and immediately detecting anomalies.

[1141] "Notification means" refers to a means of quickly sending an alert to relevant parties when an abnormality is detected, and includes methods of sending an alert such as telephone, email, and SMS.

[1142] The "manual cancellation function" is a function that allows the user to manually cancel the notification if they confirm that there is no abnormality.

[1143] The "means for generating a prompt" refers to a means for generating a prompt to send appropriate notifications or alerts when collected data deviates from a normal pattern for a certain period of time.

[1144] The present invention is a system for detecting anomalies using product usage data in the lives of single people and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, and a notification means.

[1145] Data collection methods

[1146] The server collects data 24 hours a day from users' communication devices, fixed communication devices, and smart meters for electricity and fuel supply equipment. Call logs, message history, and application usage data are obtained from communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and fuel consumption data is received in real time from smart meters. The hardware used includes: smart meters (common examples, and can be from a variety of manufacturers), mobile communication devices (e.g., various smartphones and tablets), and fixed communication devices (e.g., various landline phones). The software uses a data collection API.

[1147] Data preprocessing measures

[1148] The server preprocesses the collected data. Specifically, it complements missing values ​​and detects and corrects outliers. For example, if there are missing power consumption data, it calculates the average value from past data and complements the missing parts. Also, if abnormally high or low values ​​are detected, they are corrected to within an appropriate range using a normalization algorithm. The software used includes data preprocessing libraries (e.g., Pandas and NumPy).

[1149] Anomaly detection means

[1150] The device uses a generative AI model to learn normal usage patterns. This model is trained based on previously collected data, learning mobile communication device usage time and electricity and gas consumption patterns. As new data is generated in real time, the server inputs this data into the AI ​​model to determine whether there are any anomalies. Specifically, if electricity consumption drops to 0 kWh for a week, this is detected as an anomaly. The software used includes generative AI models (e.g., TensorFlow and PyTorch) and anomaly detection algorithms.

[1151] Notification means

[1152] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local governments, management companies, and other relevant parties. Notifications are sent via phone, email, or SMS, and include details of the abnormality, the need for action, and contact information. Users can also manually cancel notifications if they confirm that there is no problem. Software used includes notification management systems (e.g., Twilio API, Sendgrid).

[1153] Specific examples

[1154] Consider a single elderly person living alone. The server collects the user's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. For example, it learns that a user's normal lifestyle pattern involves daily electricity consumption of 5 kWh and gas consumption of 2 cubic meters. If the user does not use their mobile phone at all on one day and their electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. A notification system automatically sends a notice to the user's family and local government that an anomaly has been detected, allowing the relevant parties to respond promptly.

[1155] This system allows relevant parties to quickly identify abnormalities and prevent the worst-case scenario from occurring. Furthermore, it is extremely convenient in that it does not require users to install additional special equipment, and allows the efficient use of existing infrastructure.

[1156] Prompt Sentence Examples

[1157] "Create a prompt to detect an anomaly and send a notification if the user's call records, electricity consumption, or gas usage deviate from normal patterns for more than three days. Specifically, detect an anomaly when electricity consumption is less than 5 kWh or gas usage is less than 2 cubic meters."

[1158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1159] Step 1:

[1160] The server connects to the communication device. The server retrieves call records, message history, and application usage data from the user's communication device. Specifically, it requests data using the communication device's API and stores the received data in a database. The input is the data request from the communication device, and the output is the retrieved usage data.

[1161] Step 2:

[1162] The server connects to the fixed communication device. The server acquires call records from the fixed communication device. Specifically, the server reads the log of the fixed communication device, extracts the call records, and stores them in a database. The input is a data request from the fixed communication device, and the output is the acquired call records.

[1163] Step 3:

[1164] The server connects to the smart meters of the electricity supply equipment and fuel supply equipment. The server obtains electricity and gas consumption data in real time. Specifically, it obtains the data using the smart meter's API and stores it in a database. The input is a data request from the smart meter, and the output is the received consumption data.

[1165] Step 4:

[1166] The server retrieves the collected data from the database. First, it detects missing values ​​and then calculates the average value based on past data to fill in the gaps. Specifically, it finds the missing values ​​in the collected data and fills in the gaps with the average value calculated from past data. The input is the collected data from the database, and the output is the filled data.

[1167] Step 5:

[1168] The server performs range checks on the data, detects abnormally high or low values, and applies a normalization algorithm to correct them. Specifically, it finds data that deviates from the normal range and converts it to fit within a predefined range. The input is the data with missing values ​​imputed, and the output is the data with the outliers corrected.

[1169] Step 6:

[1170] The device trains the generative AI model. The device uses past data as input to train the model and have it learn normal usage patterns. Specifically, a training dataset is input into the AI ​​model, and learning progresses epoch by epoch. The input is past collected data, and the output is a trained generative AI model.

[1171] Step 7:

[1172] The server inputs new data into the generative AI model in real time. The server uses the model to detect anomalies. Specifically, it inputs new data into the model, calculates an anomaly score, and determines it as an anomaly if the anomaly score exceeds a threshold. The input is new data in real time, and the output is the anomaly detection result.

[1173] Step 8:

[1174] As soon as the server detects an anomaly, it automatically sends an alert to the user, distant family members, local government, and management company via notification means. Specifically, based on the anomaly detection results, it uses a notification API to send alerts by phone, email, and SMS. The input is the anomaly detection result, and the output is the sent alert notification.

[1175] Step 9:

[1176] After receiving a notification, the user can manually cancel the notification if necessary. Specifically, the user confirms that there is no abnormality through the smartphone app and sends a cancellation request to the server. When the server receives this cancellation request, it cancels the notification state. The input is the cancellation request from the user, and the output is the canceled notification state.

[1177] (Application example 1)

[1178] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1179] In modern society, problems such as lonely deaths of single people and bachelors are on the rise. To prevent such incidents, a system is needed to quickly detect abnormalities in daily life and notify relevant parties. However, existing systems tend to be slow to respond because they are incomplete in data collection and anomaly detection. Furthermore, manual monitoring and the need to disable notifications place a heavy burden on users. Therefore, efficient and automated security services are needed.

[1180] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1181] In this invention, the server includes a means for acquiring communication history, message history, and application usage data from communication devices, a means for acquiring communication history from fixed communication devices, and a means for receiving energy consumption data. This enables automatic detection of anomalies in the daily lives of single people and bachelors and prompt notification. Furthermore, by performing missing value imputation and outlier correction in data, real-time anomaly detection using a generative AI model that learns normal usage patterns, notification via multiple means, automatic message sending, and continuous monitoring of infrastructure equipment usage, the system reduces the burden on users and provides a highly accurate monitoring and notification system.

[1182] 1. "Telecommunications equipment" is a general term for equipment used by users to communicate, including mobile and fixed communications equipment.

[1183] 2. "Mobile communication equipment" refers to portable communication devices such as mobile phones and smartphones.

[1184] 3. "Fixed communications equipment" means a device used for communication at a fixed location, such as a landline telephone or desktop computer.

[1185] 4. "Communication history" refers to data that includes records of calls made by the user and the history of message exchanges.

[1186] 5. "Message History" refers to the historical data of text messages and emails sent and received by a User.

[1187] 6. "Application Usage Data" means data regarding the usage of applications installed by a User.

[1188] 7. "Energy Consumption Data" means data relating to energy consumption, including electricity and gas.

[1189] 8. "Data preprocessing" refers to the process of filling in missing values ​​in collected data and detecting and correcting outliers.

[1190] 9. "Missing value imputation" refers to the process of appropriately completing missing values ​​in a dataset.

[1191] 10. "Outlier correction" refers to the process of correcting abnormally high or low values ​​in a data set to fall within an appropriate range.

[1192] 11. A “generative AI model” is a model trained for anomaly detection using machine learning or deep learning.

[1193] 12. “Training” refers to the process by which a generative AI model learns normal usage patterns using historical data.

[1194] 13. "Real-time analytics" refers to the rapid processing of new data as it is collected and the immediate detection of anomalies.

[1195] 14. "Automatic Notification" refers to the process by which the server automatically sends alerts to relevant parties when an anomaly is detected.

[1196] 15. "Manual Cancellation Function" refers to the function that allows the user to manually cancel an alert notification when they confirm that there are no abnormalities.

[1197] 16. "Emergency Contacts" means a list of contacts to be notified in the event of an emergency.

[1198] 17. "Infrastructure equipment" refers to equipment that supplies electricity, gas, and other supplies necessary for daily life and business.

[1199] The system of the present invention uses communication devices, fixed communication devices, and energy consumption data to detect abnormalities and notify relevant parties as necessary. Specifically, the system is configured as follows.

[1200] Data collection methods

[1201] The server retrieves communication history, message history, and application usage data from mobile communication devices, including smartphones and other mobile devices. It also retrieves communication history from fixed communication devices and receives real-time electricity and gas consumption data from energy meters. All data is securely stored and encrypted.

[1202] Data preprocessing measures

[1203] Missing values ​​are first interpolated from the collected data. Then, outliers are detected and corrected. For example, if there are missing power consumption data, the data is interpolated using the average value from past data. Furthermore, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range. This process uses data processing libraries such as Pandas and NumPy.

[1204] Anomaly detection means

[1205] The device uses a generative AI model to learn normal usage patterns. This model is trained based on past data. The trained model (for example, a Keras model) is fed new data in real time and determines whether there are any abnormalities. The server will recognize an abnormality, for example, if power consumption drops to 0 kWh for a week. The following prompts can be used to train the "generative AI model."

[1206] Example prompt for a generative AI model:

[1207] text

[1208] Training generative AI models to detect anomalies based on electricity, gas, and smartphone usage data

[1209] train_model(input_data, labels)

[1210] Notification means

[1211] As soon as the server detects an abnormality, it automatically notifies the user and designated emergency contacts. Notification methods include phone, email, and SMS. For example, notifications can be sent using Twilio or SMTP. Information sent includes details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[1212] Example system operation

[1213] Consider the case of monitoring the lifestyle of an elderly person living alone. In this case, the server collects mobile communication history, message history, app usage data, and energy consumption data 24 hours a day. A generative AI model that learns normal lifestyle patterns detects abnormal data in real time. When an abnormality is detected, the server automatically sends a notification to emergency contacts, enabling a prompt response. A manual deactivation function allows the user to deactivate notifications themselves.

[1214] The above system will enable efficient monitoring of the lives of single people and bachelors, quickly detect any abnormalities, and notify the relevant parties, thereby making it possible to prevent problems such as lonely deaths.

[1215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1216] Step 1: Data collection

[1217] The server retrieves communication history, message history, and application usage data from communication devices. At the same time, it receives communication history from fixed communication devices and real-time electricity and gas consumption data from energy meters. All of this data is securely stored and encrypted. Input is data from various sensors and devices, and output is data stored in encrypted data storage.

[1218] Step 2: Data Preprocessing

[1219] The server first fills in missing values ​​in the collected data. For example, if there are missing power consumption data, this involves filling in the data using the average value from past data. Next, it detects and corrects outliers. If abnormally high or low values ​​are detected, they are normalized to fit within an appropriate range. The input is data obtained from encrypted data storage, and the output is the data that has been filled and normalized.

[1220] Step 3: Training the generative AI model

[1221] The device trains a generative AI model to learn normal usage patterns using preprocessed data. Specifically, it uses past data to learn normal patterns and creates a model to detect abnormal patterns. The prompt statement is "Train a generative AI model that detects anomalies based on electricity, gas, and smartphone usage data." The input is preprocessed data, and the output is a model that has learned normal patterns.

[1222] Step 4: Detect anomalies in real-time data

[1223] Using the generative AI model, the server analyzes new data in real time and determines whether there are any abnormalities. For example, if power consumption drops to 0 kWh, it will recognize this as an abnormality. The input is new data collected in real time, and the output is a flag indicating whether there are any abnormalities.

[1224] Step 5: Notification of abnormalities

[1225] If an anomaly is detected, the server automatically notifies the user and the designated emergency contacts. Notification methods include phone, email, and SMS. This notification is performed using Twilio and SMTP. The input is the anomaly detection flag and a list of emergency contacts, and the output is the notification message sent.

[1226] Step 6: Manual release function

[1227] If the user or designated emergency contact confirms that there are no abnormalities, the server provides a function to manually cancel the notification. Specifically, the user logs in to the system and uses a GUI to confirm that there are no abnormalities. The input is the user's confirmation operation, and the output is the notification cancellation status.

[1228] The above processing steps provide a system that can efficiently monitor the lives of single people and bachelors, quickly detect abnormalities, and notify relevant parties, thereby preventing problems such as lonely deaths.

[1229] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1230] The present invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and preventing situations such as lonely deaths. This system includes a data collection means, a data preprocessing means, an anomaly detection means, an emotion engine, and a notification means.

[1231] Data collection methods

[1232] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as electricity and gas smart meters. Call logs, message histories, and application usage data are obtained from mobile communication devices, and call logs are obtained from fixed communication devices. In addition, electricity and gas consumption data is received in real time from smart meters.

[1233] Data preprocessing measures

[1234] The server complements missing values ​​in collected data and detects and corrects outliers. For example, if there are missing power consumption data, it complements them using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[1235] Anomaly detection means

[1236] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[1237] Emotion Engine

[1238] The emotion engine recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion engine also works in conjunction with anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of unresponsiveness can be recognized as an abnormality.

[1239] Notification means

[1240] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[1241] Specific examples

[1242] Consider a scenario in which Mr. D, a single elderly person, lives alone. The server collects Mr. D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. The emotion engine also analyzes Mr. D's emotional state from the content of his calls and messages. For example, it learns that a typical lifestyle pattern for Mr. D is that his daily electricity consumption is 5kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[1243] If one day, Person D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an abnormality. At the same time, the emotion engine will detect from the content of past messages that Person D has recently been sending many negative messages. The notification method will automatically send a notice to Person D's family and local government that "an abnormality has been detected," allowing the relevant parties to respond quickly. If Person D is simply away from home for some reason and there is no problem, Person D can manually cancel the notification himself / herself.

[1244] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, it offers great convenience in that it does not require users to install special equipment and can efficiently utilize existing infrastructure. Furthermore, by combining it with an emotion engine, more advanced anomaly detection is possible, taking into account the user's psychological state.

[1245] The processing flow will be explained below.

[1246] Program processing steps

[1247] Data collection

[1248] Step 1:

[1249] The server collects call records, message history, and application usage data from mobile communication devices 24 hours a day.

[1250] The server uses the mobile carrier's API to automatically download the user's call and message history.

[1251] The server periodically collects application usage data from users' smartphones and tablets.

[1252] Step 2:

[1253] A server retrieves call records from the fixed communication device.

[1254] The server retrieves call records through the fixed line carrier's API and stores them in a secure database.

[1255] Step 3:

[1256] The server receives consumption data in real time from electricity and gas smart meters.

[1257] The server uses the APIs of electricity and gas suppliers to collect, encrypt, and store consumption data every hour.

[1258] Data Preprocessing

[1259] Step 4:

[1260] The server preprocesses the collected data.

[1261] The server detects missing values ​​in the dataset and imputes them using statistical methods.

[1262] For example, if there is a gap in the electricity consumption data, it will be supplemented based on the average usage amount over the past month.

[1263] Step 5:

[1264] The server detects and corrects outliers.

[1265] The server uses statistical techniques for each item in the dataset to identify outliers.

[1266] For example, if the power consumption deviates from the normal range, it is corrected to fall within an appropriate range.

[1267] Anomaly detection

[1268] Step 6:

[1269] The device uses a generative AI model to learn normal usage patterns.

[1270] The device uses past user data to train a generative AI model (e.g., an LSTM model).

[1271] The model learns typical call duration, message sending frequency, and electricity and gas consumption patterns.

[1272] Step 7:

[1273] The server analyzes real-time data and detects anomalies.

[1274] The server inputs the newly collected data into the generative AI model, which analyzes it in real time for any abnormalities.

[1275] For example, if power consumption is zero for three consecutive days, it will be detected as an abnormality.

[1276] Emotion analysis

[1277] Step 8:

[1278] The server uses an emotion engine to analyze the user's emotional state.

[1279] The server performs text analysis of call content and message history to identify positive and negative sentiment.

[1280] For example, if the message content is negative, the emotion engine will recognize that.

[1281] Step 9:

[1282] The server links the results of the emotion engine with the anomaly detection means.

[1283] The server feeds back the emotional state obtained from the emotion engine to the generative AI model, improving the accuracy of anomaly detection.

[1284] If negative emotional states persist, they are recognized as psychological abnormalities.

[1285] notification

[1286] Step 10:

[1287] The server will notify you when an abnormality is detected.

[1288] The server automatically generates and sends alerts to registered notification recipients (family, local government, property management company).

[1289] The notification will include details of the abnormality, instructions on how to respond, and emergency contact information.

[1290] Step 11:

[1291] The server provides notification in multiple ways.

[1292] The server will first send a notification via SMS, and if there is no response, it will also notify by phone or email.

[1293] To reliably notify abnormalities using a plurality of notification means.

[1294] Step 12:

[1295] If the user confirms that there is no abnormality, the notification can be manually canceled.

[1296] If the user receives a notification and confirms that there is no problem, they can dismiss the notification by clicking the provided link or button.

[1297] The server receives a opt-out request from the user to stop further notifications.

[1298] Through each of the above steps, the system can detect abnormalities by integrating users' product usage data and emotional state, and notify relevant parties quickly and reliably, thereby preventing serious problems such as lonely deaths.

[1299] Example 2

[1300] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1301] There is a need to detect abnormalities in the lifestyles of single people early on and prevent serious incidents such as solitary death. In particular, when elderly people and others live alone, it is important to respond to sudden changes in their health condition or living situation, but current systems have the problem of not being able to detect such abnormalities quickly and accurately.

[1302] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, and means for storing and encrypting data. This makes it possible to centrally collect information from a wide range of data sources.

[1303] The data preprocessing means also includes means for complementing missing values ​​in the collected data and means for detecting and correcting abnormal values ​​in the collected data, thereby improving the accuracy of the data and the reliability of anomaly detection.

[1304] Furthermore, the system includes an anomaly detection means that uses acquired data to train a generative AI model that learns normal usage patterns, and a means that analyzes new data in real time to detect anomalies, and an emotion analysis means that analyzes call content and message text acquired from the mobile communication device to determine the user's emotional state, thereby making it possible to detect not only physical anomalies but also psychological anomalies.

[1305] Finally, the notification means includes a means for sending an alert to the notification destination when an abnormality is detected, a means for sending notifications by multiple means, and a means for providing a manual cancellation function when no abnormality is confirmed, which enables a quick and appropriate response.

[1306] "Data collection means" refers to means for acquiring various types of data from mobile communication devices, fixed communication devices, electricity and gas smart meters, etc.

[1307] A "mobile communications device" is a mobile communications device, such as a smartphone or tablet, that provides call logs, message history, application usage data, and the like.

[1308] A "fixed communications device" is a fixed communications device such as a landline telephone or router that provides call records.

[1309] A "smart meter" is a measuring device that measures and provides electricity and gas consumption data in real time.

[1310] "Data preprocessing means" refers to means for complementing missing values ​​and detecting and correcting outliers in acquired data.

[1311] "Missing value imputation" is a process in which, when there are gaps in the collected data, the gaps are filled in using past data or average values.

[1312] "Outlier detection and correction" is a process that detects abnormal values ​​in data and corrects them to an appropriate range using methods such as normalization.

[1313] An "anomaly detection method" is a method that uses acquired data to train a generative AI model that learns normal usage patterns and analyzes new data in real time to detect anomalies.

[1314] A "generative AI model" is a model that uses machine learning algorithms such as deep learning to learn patterns in data and detect anomalies.

[1315] "Real-time analysis" is an analytical method that processes new data instantly and immediately determines whether or not there are any abnormalities.

[1316] The "emotion analysis means" is a means for analyzing call content and message text obtained from a mobile communication device to determine the user's emotional state.

[1317] The "notification means" is a means for sending an alert to a notification destination when an abnormality is detected.

[1318] The "manual cancellation function" is a function that allows the user to manually cancel the notification when the user confirms that there is no abnormality.

[1319] This invention is a system for detecting anomalies using product usage data and the emotional state of users in the lives of single people, and for preventing situations such as solitary death. This system includes data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means.

[1320] Data collection methods

[1321] The server collects data 24 / 7 from users' mobile and fixed communication devices, as well as from their electricity and gas smart meters. It obtains call logs, message histories, and application usage data from mobile communication devices, and call logs from fixed communication devices. It also receives real-time electricity and gas consumption data from the smart meters.

[1322] Data preprocessing measures

[1323] The server performs missing value interpolation and outlier detection / correction on the collected data. Specifically, if there are missing power consumption data, it interpolates using the average value from past data. Also, if abnormally high or low values ​​are detected, they are normalized to bring them within an appropriate range.

[1324] Anomaly detection means

[1325] The device uses a generative AI model to learn normal usage patterns. It is trained based on past data and learns mobile usage time and electricity and gas consumption patterns. The server inputs new data into the AI ​​model in real time and determines whether there are any abnormalities. For example, if electricity consumption drops to 0 kWh for a week, it will detect this as an abnormality.

[1326] Emotion analysis means

[1327] The emotion analysis means recognizes and analyzes the user's emotional state. It analyzes call content, message text, and application usage patterns obtained from the mobile communication device to determine the user's emotional state. The emotion analysis means works in conjunction with the anomaly detection means to detect psychological abnormalities in the user. For example, negative messages or a long period of no response can be recognized as an abnormality.

[1328] Notification means

[1329] As soon as the server detects an abnormality, it automatically sends an alert to the user, distant family members, local government, and property management companies. Notifications are sent by multiple means, including phone, email, and SMS, and include details of the abnormality, the need for action, and contact information. In addition, if the user confirms that there are no abnormalities, they are also provided with a function to manually cancel notifications.

[1330] Specific examples

[1331] As an example of a single elderly person, let's assume a scenario where user D lives alone. The server collects D's mobile usage data (call duration, number of messages sent), landline call records, and electricity and gas consumption data 24 hours a day. In addition, an emotion analysis tool analyzes D's emotional state from the content of his calls and messages. For example, it learns that D's normal daily electricity consumption is 5 kWh, his gas usage is 2 cubic meters, and the content of his calls and messages is positive.

[1332] If one day D does not use his / her mobile phone at all and his / her electricity and gas consumption remains abnormally low for three consecutive days, the server will detect this as an anomaly. At the same time, the sentiment analysis means will detect from the content of past messages that D has recently been sending many negative messages. The notification means will automatically send a notice to D's family and local government that "an anomaly has been detected," allowing the relevant parties to respond quickly. If D is simply away from home for some reason and there is no problem, D can manually cancel the notification himself / herself.

[1333] An example of a prompt is, "D is a single elderly person living alone. His smartphone regularly sends call records and message history to a server, and his home's smart meter also sends real-time electricity and gas usage data. If he does not engage in these normal activities for several days, design a system that detects this as an abnormality and automatically sends a notification to his family and local government."

[1334] This system allows distant family members, local governments, and property management companies to quickly identify abnormalities and prevent the worst-case scenario. Furthermore, users do not need to install special equipment, so they can live safely using their existing living environment and infrastructure. Furthermore, by combining it with emotion analysis methods, even more advanced anomaly detection becomes possible.

[1335] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1336] Step 1: Data collection

[1337] The server collects data 24 hours a day. It receives data from mobile communication devices, fixed communication devices, and smart meters as input and stores this data as output. From mobile communication devices, it obtains call records, message history, and application usage data, and from fixed communication devices, it obtains landline call records. From smart meters, it receives electricity and gas consumption data in real time. Specifically, the server periodically obtains data from these devices through APIs.

[1338] Step 2: Data Preprocessing

[1339] The server preprocesses the collected data. It uses the data collected in step 1 as input and generates data with missing values ​​and corrected outliers as output. Specifically, if there is missing data, the server calculates the average value from past data to fill in the gaps. Also, if an abnormally high or low value is detected, it normalizes the value and corrects it to fall within an appropriate range. For example, if there is a gap in Mr. D's power consumption data, it fills in the gaps using the average value from a similar time period in the past.

[1340] Step 3: Anomaly detection

[1341] The device uses the generative AI model to detect anomalies. It uses the preprocessed data from step 2 as input and determines whether an anomaly has been detected as output. Specifically, the device trains the AI ​​model based on past data to learn normal usage patterns. It uses this model to analyze new data in real time and determine whether anomalies exist. For example, if Mr. D's electricity consumption is 0 kWh for three days, it will detect this as an anomaly.

[1342] Step 4: Sentiment Analysis

[1343] The server analyzes the user's emotional state using an emotion analysis method. It uses call content and message text acquired from the mobile communication device as input and determines the user's emotional state as output. Specifically, the server analyzes the call content and message text using a natural language processing algorithm to identify positive or negative emotions. For example, if Mr. D's messages over the past month have contained a lot of negative content, it determines his / her emotional state as negative.

[1344] Step 5: Notification of abnormalities

[1345] If the server detects an abnormality, it notifies the relevant parties. It uses the anomaly detection results and sentiment analysis results from steps 3 and 4 as input, and sends an abnormality notification via multiple means as output. Specifically, the server sends an abnormality alert to the user, family, local government, and property management company via phone, email, and SMS. The notification content includes details of the abnormality, the need for response, contact information, etc. For example, if an abnormality is detected for Mr. D, the server will send an email notification containing details of the abnormality and contact information.

[1346] In this way, the system will be able to detect abnormalities among single people at an early stage and notify relevant parties promptly.

[1347] (Application example 2)

[1348] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1349] Elderly people and single people living alone are isolated, and there is a need to detect and respond early when abnormalities in their health or psychological state occur. However, current support systems for isolated people lack real-time data collection and emotion analysis, making it difficult to detect abnormalities early. In addition, they cannot utilize standard infrastructure or devices, which poses challenges in terms of cost and operation.

[1350] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes, as data collection means, means for acquiring call logs, message history, and application usage data from mobile communication devices, means for acquiring call logs from fixed communication devices, means for receiving electricity and gas consumption data, means for acquiring data from vital sensors and wearable devices that collect location information, means for storing and encrypting data, as data preprocessing means, means for complementing missing values ​​in the collected data, and means for detecting and correcting abnormal values ​​in the collected data, as anomaly detection means, means for training a generative AI model that learns normal usage patterns using the acquired data, and means for analyzing new data in real time to detect anomalies, as emotion analysis means, means for analyzing the user's emotional state from image and audio data, and means for detecting anomalies based on the emotional state, as notification means, means for sending an alert to a notification destination when an anomaly is detected, means for notifying by multiple means, and means for providing a manual release function when no anomaly is confirmed. This makes it possible to detect anomalies and perform emotion analysis in real time based on collected data, enabling early detection of abnormal situations.

[1351] "Data collection means" refers to means for acquiring various data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices.

[1352] A "mobile communication device" is a device capable of mobile communication, such as a mobile phone or smartphone, that is used to obtain call logs, message history, and application usage data.

[1353] A "fixed communication device" is a telephone facility that is fixedly used in a home or office and is a means for obtaining call records.

[1354] A "smart meter" is a device that measures and transmits electricity and gas consumption data in real time.

[1355] A "wearable device" is a device that can be worn by a user and that acquires vital signs and location information.

[1356] "Means for storing and encrypting data" refers to the means for securely storing collected data and encrypting it to protect it from unauthorized access.

[1357] "Data preprocessing means" refers to means for complementing missing values ​​in collected data and detecting and correcting outliers.

[1358] A "generative AI model" is an artificial intelligence model that is trained to learn normal usage patterns using past data and detect anomalies in new data.

[1359] An "anomaly detection method" is a method for analyzing new data in real time using a generative AI model to detect anomalies.

[1360] The "emotion analysis means" is a means for analyzing the user's emotional state from image and audio data and detecting anomalies based on that state.

[1361] "Notification means" refers to a means for sending an alert when an abnormality is detected, notifying via multiple means, and providing a function for manually canceling notifications when no abnormality is confirmed.

[1362] This invention is a system for ensuring safety in the lives of elderly people and single people living alone. This system uses data collection means, data preprocessing means, anomaly detection means, emotion analysis means, and notification means to collect and analyze various types of user data, and can quickly notify the user when an anomaly is detected.

[1363] System configuration

[1364] Data collection methods

[1365] The server collects data by:

[1366] 1. Obtain call logs, message history, and application usage data from mobile communications devices.

[1367] 2. Obtain call records from fixed communication devices.

[1368] 3. Receive real-time consumption data from electricity and gas smart meters.

[1369] 4. Collect vital data such as heart rate and step count, as well as location information, from wearable devices.

[1370] Data preprocessing measures

[1371] The server completes missing values ​​in the collected data and detects and corrects outliers. For example, missing parts of electricity consumption data are completed with the average value from past data. Abnormally high or low values ​​are normalized.

[1372] Anomaly detection means

[1373] The anomaly detection method uses a generative AI model to learn normal usage patterns. Here, the generative AI model is trained from past data and analyzes new data in real time to detect anomalies. For example, if electricity consumption is 0 kWh for one consecutive week, it will be detected as an anomaly.

[1374] Emotion analysis means

[1375] The emotion analysis means uses the wearable device's camera and microphone to analyze the user's facial expressions and voice. The emotional state is determined from the collected data, and if a negative emotional state or a prolonged period of unresponsiveness is detected, it is recognized as an abnormality.

[1376] Notification means

[1377] As soon as the server detects an abnormality, it sends an alert to the user, distant family members, medical institutions, nursing care services, and other relevant parties. Alerts are sent via multiple means, including phone, email, and SMS. If no abnormalities are confirmed, notifications can be manually canceled.

[1378] Specific examples

[1379] Suppose a user wears smart glasses all day long. The server collects heart rate, step count, and location information through the glasses' sensors, and also uses a camera and microphone to capture the user's facial expressions and voice in real time. For example, if one day the user's heart rate suddenly spikes, their face looks sad, and they begin to stay indoors, the server will detect this as an abnormality. As a result, the server will instantly send an alert to their family or a medical institution saying, "An abnormality has been detected. Urgent investigation is required."

[1380] Prompt Sentence Examples

[1381] Check whether the user's heart rate is within the range of 80 to 100, and detect abnormalities if the number of steps is extremely low, or if the user's facial expression or voice indicates that they are sad. If these conditions are met, the system will automatically begin the process of sending an alert.

[1382] This system allows users to detect anomalies early using existing devices and infrastructure, without the need to install special equipment. In addition, by combining it with emotion analysis, more advanced anomaly detection is possible.

[1383] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1384] Step 1:

[1385] The server collects data from mobile communication devices, fixed communication devices, electricity and gas smart meters, and wearable devices, providing users' call records, message history, application usage data, electricity and gas consumption data, heart rate, step count, location information, etc. The input is raw data from various devices, and the output is an integrated data set stored on the server.

[1386] Step 2:

[1387] The server preprocesses the collected data. During this process, missing values ​​are filled in and outliers are corrected. For example, if some power consumption data is missing, it is filled in with the historical average value, and if an abnormally high heart rate is detected, it is normalized. The input is the collected raw data, and the output is the clean data that has been filled and corrected.

[1388] Step 3:

[1389] The server uses a generative AI model to train anomaly detection. This AI model is used to learn normal usage patterns based on past data. For example, electricity consumption, heart rate, and water, gas, and utility usage patterns are used as training data. The input is the clean data that has been complemented and corrected, and the output is the trained generative AI model.

[1390] Step 4:

[1391] The server inputs new data into the generative AI model in real time to detect anomalies. For example, if electricity consumption drops to 0 kWh for one consecutive week or if the heart rate rises abnormally, this will be detected as an anomaly. The input is new data acquired in real time, and the output is a judgment result as to whether or not there is an anomaly.

[1392] Step 5:

[1393] As an emotion analysis method, the server analyzes image and audio data acquired from the camera and microphone of the wearable device. This allows it to determine the user's emotions from their facial expressions and voice, and detect, for example, negative emotional states or prolonged periods of unresponsiveness. The input is image and audio data, and the output is the analysis result regarding the user's emotional state.

[1394] Step 6:

[1395] As soon as the server detects an anomaly, it sends an alert using a notification method. This notification is sent by multiple means, including phone, email, and SMS, and includes details of the anomaly and the need for action. The input is the anomaly detection judgment result and emotion analysis result, and the output is an alert notification to the user, distant family members, medical institutions, etc.

[1396] Step 7:

[1397] When a user receives a notification, they can manually dismiss the notification if they confirm that there is no abnormality. This function makes it possible to disable false alerts in the event of a false positive. The input is feedback from the user, and the output is dismissal of the notification.

[1398] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1399] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1400] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1401] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1402] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1403] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1404] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1405] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1406] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1407] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1408] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1409] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1410] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1411] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1412] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1413] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1414] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1415] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1416] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1417] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1418] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1419] The following is further disclosed regarding the above embodiment.

[1420] (Claim 1)

[1421] As a means of collecting data,

[1422] means for retrieving call logs, message history, and application usage data from the mobile communications device;

[1423] means for retrieving call records from a fixed communication device;

[1424] means for receiving electricity and gas consumption data;

[1425] a means of storing and encrypting data;

[1426] As a data preprocessing method,

[1427] A means of imputing missing values ​​in the collected data;

[1428] means for detecting and correcting outliers in the collected data;

[1429] As an anomaly detection method,

[1430] a means for training a generative AI model that learns normal usage patterns using the captured data; and

[1431] A means to analyze new data in real time and detect anomalies;

[1432] As a means of notification,

[1433] A means of sending an alert to a notification destination when an abnormality is detected;

[1434] a means of providing notification by multiple means;

[1435] The system includes a means for providing a manual release function if no abnormalities are found.

[1436] (Claim 2)

[1437] 10. The system of claim 1, wherein the data collection means receives consumption data in real time from electricity and gas smart meters.

[1438] (Claim 3)

[1439] The system of claim 1, wherein the generative AI model learned by the anomaly detection means is a model that learns normal usage patterns based on past data.

[1440] "Example 1"

[1441] (Claim 1)

[1442] As a means of collecting data,

[1443] means for obtaining call logs, message history, and application usage data from the communication device;

[1444] means for retrieving call records from a fixed communication device;

[1445] means for receiving consumption data of the power supply and the fuel supply;

[1446] a means of storing and encrypting data;

[1447] As a data preprocessing method,

[1448] A means of imputing missing values ​​in the collected data;

[1449] means for detecting and correcting outliers in the collected data;

[1450] As an anomaly detection method,

[1451] a means for training a generative AI model that learns normal usage patterns using the captured data; and

[1452] A means to analyze new data in real time and detect anomalies;

[1453] As a means of notification,

[1454] A means for sending an alert to the communication partner when an abnormality is detected;

[1455] a means of providing notification by multiple means;

[1456] A means for providing a manual release function when no abnormality is confirmed;

[1457] The system includes a means for generating a prompt that sends a notification when collected data deviates from a normal pattern for a period of time.

[1458] (Claim 2)

[1459] 2. The system of claim 1, wherein the data collection means receives consumption data in real time from smart meters of the electricity supply and the fuel supply.

[1460] (Claim 3)

[1461] The system of claim 1, wherein the generative AI model learned by the anomaly detection means is a model that learns normal usage patterns based on past data.

[1462] "Application Example 1"

[1463] (Claim 1)

[1464] As a means of collecting data,

[1465] means for obtaining communication history, message history, and application usage data from the communication device;

[1466] A means for obtaining communication history from a fixed communication device;

[1467] means for receiving energy consumption data;

[1468] a means of storing and encrypting data;

[1469] As a data preprocessing method,

[1470] A means of imputing missing values ​​in the collected data;

[1471] means for detecting and correcting outliers in the collected data;

[1472] As an anomaly detection method,

[1473] a means for training a generative AI model that learns normal usage patterns using the captured data; and

[1474] A means to analyze new data in real time and detect anomalies;

[1475] As a means of notification,

[1476] A means of sending an alert to a notification destination when an abnormality is detected;

[1477] a means of providing notification by multiple means;

[1478] A means for providing a manual release function when no abnormality is confirmed;

[1479] a means for automatically sending messages to emergency contacts;

[1480] a means for continuously monitoring infrastructure equipment usage;

[1481] A system that includes a means to seamlessly coordinate data collection, data processing, anomaly detection, and notification.

[1482] (Claim 2)

[1483] 10. The system of claim 1, wherein the energy consumption data is received in real time from an energy meter.

[1484] (Claim 3)

[1485] The system of claim 1, wherein the generative AI model learned by the anomaly detection means is a model that learns normal usage patterns based on past data and automatically detects anomalies.

[1486] "Example 2: Combining Emotion Engines"

[1487] (Claim 1)

[1488] As a means of collecting data,

[1489] means for retrieving call logs, message history, and application usage data from the mobile communications device;

[1490] means for retrieving call records from a fixed communication device;

[1491] means for receiving electricity and gas consumption data;

[1492] a means of storing and encrypting data;

[1493] As a data preprocessing method,

[1494] A means of imputing missing values ​​in the collected data;

[1495] means for detecting and correcting outliers in the collected data;

[1496] As an anomaly detection method,

[1497] a means for training a generative AI model that learns normal usage patterns using the captured data; and

[1498] A means to analyze new data in real time and detect anomalies;

[1499] As a means of sentiment analysis,

[1500] means for analyzing call content and message text obtained from the mobile communication device to determine the user's emotional state;

[1501] As a means of notification,

[1502] A means of sending an alert to a notification destination when an abnormality is detected;

[1503] a means of providing notification by multiple means;

[1504] The system includes a means for providing a manual release function if no abnormalities are found.

[1505] (Claim 2)

[1506] 10. The system of claim 1, wherein the data collection means receives consumption data in real time from electricity and gas smart meters.

[1507] (Claim 3)

[1508] The system of claim 1, wherein the generative AI model learned by the anomaly detection means is a model that learns normal usage patterns based on past data.

[1509] "Application example 2 when combining emotion engines"

[1510] (Claim 1)

[1511] As a means of collecting data,

[1512] means for retrieving call logs, message history, and application usage data from the mobile communications device;

[1513] means for retrieving call records from a fixed communication device;

[1514] means for receiving electricity and gas consumption data;

[1515] a means for acquiring...

Claims

1. As a means of collecting data, means for retrieving call logs, message history, and application usage data from the mobile communications device; means for retrieving call records from a fixed communication device; means for receiving electricity and gas consumption data; a means of storing and encrypting data; As a data preprocessing method, A means of imputing missing values ​​in the collected data; means for detecting and correcting outliers in the collected data; As an anomaly detection method, a means for training a generative AI model that learns normal usage patterns using the captured data; and A means to analyze new data in real time and detect anomalies; As a means of notification, A means of sending an alert to a notification destination when an abnormality is detected; a means of providing notification by multiple means; The system includes a means for providing a manual release function if no abnormalities are found.

2. 10. The system of claim 1, wherein the data collection means receives consumption data in real time from electricity and gas smart meters.

3. 2. The system according to claim 1, wherein the generative AI model learned by the anomaly detection means is a model that learns normal usage patterns based on past data.

Images