System

Abstract

A system is provided.SOLUTION: A system including means for collecting environmental data from monitoring devices, means for inputting the collected data into a generative artificial intelligence model and analyzing the data for anomalies in real-time, means for detecting anomalous behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for providing context and remedial advice based on the owner's request, and means for recording all data and interaction content and generating feedback to improve the artificial intelligence model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional alarm and camera systems alone are often insufficient for home security measures, posing particular challenges when ensuring security when leaving the home. Conventional systems only provide limited notifications when an abnormality occurs, and are unable to provide specific countermeasures or continuous security improvements. The present invention aims to solve these issues by building a system that provides more advanced, real-time home security. [Means for solving the problem]

[0005] The system includes a means for collecting environmental data from monitoring devices, a means for inputting the collected data into a generative AI model, and a means for analyzing anomalies in real time. It also includes a means for detecting abnormal behavior or intrusion based on the analysis results, a means for notifying the owner of the detected anomaly, a means for checking the situation and providing advice on countermeasures based on the owner's request, and a means for recording all data and dialogue, generating feedback, and improving the AI ​​model. This configuration improves home security and effectively combats burglaries even when the owner is away.

[0006] A "surveillance device" is a device installed in a home to collect environmental data in and around the home, and is a general term for devices such as surveillance cameras and various sensors.

[0007] "Environmental data" refers to information collected from monitoring devices, such as video, audio, temperature, and the open / close status of doors and windows, and is a general term for a wide variety of data that reflects conditions inside and outside the home.

[0008] "Generative AI model" is a general term for high-performance machine learning algorithms designed to learn from large amounts of data and analyze environmental data and detect anomalies.

[0009] "Abnormal behavior" or "intrusion" refers to any behavior that is different from normal household activity patterns and indicates unauthorized or intruder activity, and is a general term for any behavior that poses a security risk.

[0010] "Owner" refers to the person who manages and uses the security system in a home, typically the resident or homeowner.

[0011] "Notifications" refer to alerts or messages sent to owners when abnormal behavior or intrusion is detected, and are provided via SMS, email, dedicated apps, etc.

[0012] "Interactive support" refers to the system's ability to check the situation and provide specific measures in response to requests from owners, and is a form of support that provides information and advice in real time.

[0013] "Feedback" refers to the process of updating an artificial intelligence model based on collected data and anomaly detection results to improve the system's accuracy and performance. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] MODE FOR CARRYING OUT THE INVENTION

[0036] overview

[0037] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the monitoring device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing interactive countermeasures.

[0038] System Configuration

[0039] The server is the core of the system, collecting data from various sensors and surveillance cameras, and performing real-time analysis using a generative artificial intelligence model. If an abnormality is detected, it immediately notifies the owner and interactively provides specific countermeasures upon request.

[0040] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[0041] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[0042] Program processing

[0043] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action.

[0044] The owner receives a notification and sends a specific request to the server. For example, if the request is "Please check if the door is open," the server will determine whether the living room door is open based on the latest sensor and camera data and respond to the owner. Alternatively, if the request is "Please lock the doors," the server will remotely lock all doors and report the result to the owner.

[0045] Specific examples

[0046] Example 1: Detecting and responding to anomalies during absence

[0047] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0048] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0049] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[0050] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[0051] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[0052] 5. Based on the latest data, the server sends the door open status and camera footage to the owner, reporting the specific situation.

[0053] 6. The user sends the command "Lock the front door."

[0054] 7. The server remotely locks the front door and reports "The front door is locked."

[0055] Example 2: Daily fail-safe checks

[0056] Consider a case where a user checks the security status of their home before going to bed.

[0057] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0058] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[0059] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[0060] In this way, the present invention functions as an advanced security system that can safely protect a home even when the owner is away or asleep.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[0064] Step 2:

[0065] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[0066] Step 3:

[0067] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[0068] Step 4:

[0069] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[0070] Step 5:

[0071] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[0072] Step 6:

[0073] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[0074] Step 7:

[0075] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it can check whether the living room door is open or closed and report the results to the owner.

[0076] Step 8:

[0077] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[0078] Step 9:

[0079] The server remotely executes specific measures based on the owner's instructions, for example, remotely locking all doors and notifying the owner of the results.

[0080] Step 10:

[0081] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[0082] Example 1

[0083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0084] The present invention aims to provide a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. Conventional security systems have the difficulty of responding quickly and effectively after detecting an anomaly, leaving security vulnerable when the owner is away or asleep. To solve this problem, a system is needed that can not only detect abnormal behavior and intrusions in real time and promptly notify the owner, but also provide and implement specific countermeasures.

[0085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0086] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for analyzing the latest sensor and camera data corresponding to the owner's request and responding to the situation, means for remotely implementing security measures and reporting the results to the owner, and means for recording all data and dialogue, generating feedback, and improving the AI ​​model. This enables a quick and effective response after an anomaly is detected, making it possible to maintain home security with peace of mind even when the owner is away or sleeping.

[0087] A "surveillance device" is a device such as a sensor or camera that collects environmental data.

[0088] The "server" is a central device that consolidates data collected from monitoring devices, analyzes it using generative AI models, detects abnormal behavior and intrusions, notifies users, manages requests, and implements security measures.

[0089] "Environmental data" refers to information that indicates the security situation inside and outside the home, such as whether doors and windows are open or closed, temperature, sound, and video.

[0090] A "generative AI model" is an artificial intelligence algorithm designed to identify anomalous behavior and intrusion patterns based on large amounts of training data.

[0091] "Real-time analysis" means processing data collected from monitoring devices immediately and without delay to detect abnormalities.

[0092] "Abnormal behavior or intrusion" refers to the act of entering a building by illegal means or any unnatural behavior that differs from normal.

[0093] An "owner" is a person using a home security system who receives notifications from the system and sends requests.

[0094] A "request" is a specific confirmation or instruction for action sent from the owner to the server.

[0095] A "sensor" is a device that detects environmental factors such as the opening and closing of doors and windows, temperature, and sound.

[0096] A "camera" is a photographing device for collecting video data.

[0097] "Notification" is a message sent from the server to the owner informing them of an abnormality or security situation.

[0098] "Implementing security measures remotely" means that the server will remotely implement security measures such as locking doors at the owner's request.

[0099] "Feedback" is evaluation information that the system uses to improve the performance of the generated AI model based on the content of the dialogue and data.

[0100] overview

[0101] This invention relates to a home security system that combines a generative AI model with advanced understanding capabilities with a surveillance device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the surveillance device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing specific countermeasures in an interactive format.

[0102] System Configuration

[0103] The system mainly consists of the following three elements:

[0104] 1. Server

[0105] The system's core component collects data from monitoring devices and performs real-time analysis using generative AI models. If an abnormality is detected, it immediately notifies the owner and, upon request, interactively provides specific countermeasures.

[0106] 2. Surveillance Devices

[0107] It consists of surveillance cameras and various sensors (door and window opening / closing sensors, temperature sensors, sound sensors, etc.), each of which has the role of transmitting environmental data to a server.

[0108] 3. User (Owner)

[0109] When an abnormality is detected, you can receive a notification and request specific measures from the server.

[0110] Program processing

[0111] The server inputs environmental data collected from the monitoring devices into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action to be taken.

[0112] The user receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the doors are open," the server determines whether the doors are open based on the latest sensor and camera data and responds to the owner. Similarly, in response to a request such as "Please lock the doors," the server remotely locks all doors and reports the result to the owner.

[0113] Specific examples

[0114] Example 1: Detecting and responding to anomalies during absence

[0115] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0116] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0117] 2. The server inputs the collected data into the generative AI model for real-time analysis. The generative AI model returns a high score for this behavior, identifying it as abnormal.

[0118] 3. If the server detects an abnormality, it immediately sends an SMS to the user notifying them that "The front door has been opened in an unnatural way."

[0119] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[0120] 5. Based on the latest data, the server notifies the user that the door is open and sends camera footage to report the specific situation.

[0121] 6. The user sends the command "Lock the front door."

[0122] 7. The server remotely locks the front door and reports "The front door is locked."

[0123] Example 2: Daily fail-safe checks

[0124] Consider a case where a user checks the security status of their home before going to bed.

[0125] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0126] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative AI model.

[0127] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the user that "all doors and windows are closed and no abnormal sounds have been detected."

[0128] In this way, the present invention functions as an advanced security system that can safely protect the home even when the user is away or asleep.

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Step 1:

[0131] The server collects environmental data from multiple devices (sensors and surveillance cameras). The input is data from various sensors (e.g., temperature, door opening / closing, sound, video), which is integrated into a database within the server. The output is the integrated environmental data. Since this data is collected in real time, the server always has the latest information.

[0132] Step 2:

[0133] The server inputs the collected environmental data into the generative AI model. The input is environmental data, and each data point is analyzed by the generative AI model. The AI ​​model is trained to identify abnormal and intrusive behavior. The output is an anomaly score assigned to each data point and the analysis results. Specifically, the AI ​​model detects sudden changes in temperature, unnatural opening and closing movements, and loud noises, and scores them as abnormal.

[0134] Step 3:

[0135] The server evaluates the analysis results returned by the generative AI model. The inputs are the anomaly score and the analysis results, and based on these, it determines whether an anomaly has been detected. The output is the specific details of the detected anomaly. For example, it can include specific information such as "the front door was opened unnaturally late at night."

[0136] Step 4:

[0137] If an abnormality is detected, the server will promptly send a notification to the user. The input is the detected abnormality, and the output is the notification to the user (e.g., SMS or dedicated app alert). The notification will include the specific details of the abnormality and a suggestion of immediate action to be taken (e.g., "The front door is open. Please check it.").

[0138] Step 5:

[0139] The user receives a notification and sends a specific request to the server. The input is the user's request (e.g., "Check the status of the front door"), and the output is that the request reaches the server. The specific action is that the user enters the request through the application.

[0140] Step 6:

[0141] The server recollects and analyzes the latest sensor and camera data based on the user's request. The input is the latest sensor and camera data, and the output is a response to the user. For example, specific information such as "The front door is open and a suspicious person was captured on camera" is sent to the user.

[0142] Step 7:

[0143] The server remotely implements security measures based on user instructions. The input is the user's instruction (e.g., "Lock the front door"), and the output is a report of the implemented security measures (e.g., "The front door is locked"). Specifically, the server sends a signal to the door lock control system to lock the door.

[0144] (Application example 1)

[0145] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0146] Modern home security systems have limited capabilities for detecting intrusions and other anomalies, and lack real-time monitoring and automated user response. Even if users detect an intrusion, it is difficult for them to immediately take appropriate countermeasures. This increases the burden on users and increases security risks. This can lead to delayed responses, especially when users are in remote locations or late at night, when immediate action is required.

[0147] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0148] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for checking the situation and providing advice on countermeasures based on the owner's request, means for remotely controlling the security status, means for executing instructions from the user in response to an abnormality that has occurred, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to highly automate the entire process from anomaly detection to countermeasure implementation, significantly reducing security risks while reducing the burden on the user.

[0149] "Surveillance devices" refers to various sensors and surveillance cameras used to collect environmental data.

[0150] "Environmental data" refers to data that indicates the physical and operational conditions within a space, including the open / close status of doors, temperature, and sound.

[0151] A "generative artificial intelligence model" is a system that uses machine learning and deep learning to analyze environmental data in real time and includes algorithms for detecting abnormal behavior and intrusions.

[0152] "Real-time analysis" is the process of instantly processing environmental data collected from monitoring devices and detecting abnormalities.

[0153] "Abnormal behavior" is anything that deviates from normal behavior and includes behavior that indicates intrusion or fraud.

[0154] "Notification" refers to the means of transmitting alerts and information to the owner when an abnormality is detected.

[0155] A "request" refers to a request from the owner to the server to check the status or to give instructions on how to deal with the problem.

[0156] "Status check" refers to the operation that the owner performs to check the current security status when an abnormality is detected.

[0157] "Countermeasure advice" refers to the specific countermeasure suggestions provided to the owner by the generative artificial intelligence model when an abnormality is detected.

[0158] "Remote operation means" refers to an interface that allows the owner to operate the various functions of the security system from a remote location.

[0159] "Feedback" is data generated from the dialogue and results of the system's operations that is used to improve the performance of generative artificial intelligence models.

[0160] "Means for improving artificial intelligence models" refers to the process of adjusting algorithms and parameters based on collected data and feedback information to improve the performance of the model.

[0161] MODE FOR CARRYING OUT THE INVENTION

[0162] Overall system configuration

[0163] The present invention is a system that consists of three main components: a monitoring device, a server, and a user terminal.

[0164] Surveillance Devices

[0165] The monitoring devices consist of various sensors (door sensors, window sensors, temperature sensors, sound sensors, etc.) and surveillance cameras for collecting environmental data. These devices are installed to detect abnormal behavior and intrusions within the home, and transmit data to a server in real time.

[0166] server

[0167] The server is the central control device of this system. It integrates environmental data collected from monitoring devices and performs real-time analysis using generative AI models. If abnormal behavior or intrusion is detected, the server immediately notifies the user. It also provides specific situation confirmation and advice on countermeasures based on the user's request.

[0168] The server includes the following features:

[0169] 1. Data collection function: Collects environmental data from monitoring devices.

[0170] 2. Real-time analytics: Analyze data using generative AI models to detect anomalies.

[0171] 3. Notification function: Sends a notification to the user when an abnormality is detected.

[0172] 4. Remote control function: The security status of the room can be controlled remotely according to the user's request.

[0173] 5. Feedback generation function: Record all data and conversations, generate feedback and improve the generative AI model.

[0174] User terminal

[0175] User terminals are devices such as smartphones, tablets, and smart glasses that allow users to interact with the system and check and operate the security status. Users can receive notifications from the server through their terminals and send status checks and countermeasure requests.

[0176] Program processing

[0177] Real-time data collection

[0178] The server consolidates the environmental data collected from each monitoring device, which is sent in formats such as JSON and includes information such as the status of doors and windows, temperature, and sound.

[0179] Real-time analytics

[0180] The data is fed into a generative AI model (e.g., RealTimeAnalyzer) and analyzed in real time. The generative AI model includes algorithms for detecting abnormal behavior and intrusions.

[0181] Anomaly detection and notification

[0182] If the server detects an anomaly based on the analysis results, it will immediately send a notification to the user using a notification service (e.g., Twilio API, Firebase Cloud Messaging). The notification will include details of the anomaly and the measures the user should take.

[0183] Request handling and remote operations

[0184] When a request from a user (e.g., "lock the door") is received, the server calls an API for remote operation and performs the corresponding operation.

[0185] Feedback Generation

[0186] The server records all data and conversations and generates feedback to improve the performance of the generative AI model, which is then used to adjust parameters and algorithms to improve anomaly detection capabilities next time.

[0187] Examples of specific examples and prompts

[0188] Example 1:

[0189] Consider a case where the rear window of a house suddenly opens in the middle of the night while the user is away on a trip. A suspicious person is clearly visible in the frame of a surveillance camera. The generative AI model detects this abnormal behavior with a high score, and the server immediately sends a notification to the user. The user then sends a request via their smartphone to "lock all doors and windows," and the server remotely locks all doors and windows.

[0190] Example 2:

[0191] If the server detects an abnormality such as the front door being opened and closed multiple times while the user is out, it will notify the user that "The front door has been opened and closed multiple times" and immediately lock the door based on the user's request to "lock the front door."

[0192] Example prompt:

[0193] "Analyzes abnormal behavior detected by sensors and cameras during specific times. Abnormal behavior detection: Determines whether a door has been opened or closed, whether a person is visible in the room, or whether an abnormal sound has been recorded."

[0194] This allows for the construction of an effective embodiment of the invention, and the system efficiently automates a series of processes from anomaly detection to countermeasure implementation, thereby reducing the burden on the user.

[0195] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0196] Step 1:

[0197] Data collection

[0198] The server collects environmental data from each monitoring device (door sensor, window sensor, temperature sensor, sound sensor, surveillance camera).

[0199] Input: Real-time data sent from each monitoring device (e.g., door opening, temperature change, sound generation, etc.)

[0200] Output: The integrated results of the collected environmental data (e.g., data in JSON format)

[0201] Specific Operation: The server pulls data from the monitoring devices at regular intervals and generates a consolidated data set.

[0202] Step 2:

[0203] Preparing for data analysis

[0204] The server performs preprocessing to analyze the collected environmental data, extracting only the data necessary for anomaly detection.

[0205] Input: Collected environmental data

[0206] Output: Preprocessed dataset

[0207] Specific operations: filtering unnecessary data, shaping data necessary for anomaly detection (e.g., data normalization)

[0208] Step 3:

[0209] Real-time analytics

[0210] The server passes the preprocessed dataset to a generative AI model to detect anomalous behavior and intrusions.

[0211] Input: Preprocessed dataset

[0212] Output: Anomaly detection result (e.g., anomaly score, if the score is high, an anomaly is detected)

[0213] How it works: The generative AI model analyzes the dataset and generates an anomaly score. High scores indicate anomalous behavior.

[0214] Step 4:

[0215] Sending abnormality notifications

[0216] If an abnormality is detected, the server sends a notification to the user device, which includes details of the abnormality and advice on how to deal with it.

[0217] Input: Anomaly detection result (high score)

[0218] Output: A message to inform the user

[0219] Specific operation: Notifications are sent to the user's smartphone or smart glasses using the Twilio API or Firebase Cloud Messaging.

[0220] Step 5:

[0221] Processing user requests

[0222] After receiving the notification, the user sends a request to the server to check the situation and take action.

[0223] Input: User request (e.g. "Lock the door" or "Check the camera footage")

[0224] Output: The result of sending the user request to the server

[0225] Specific operation: A request is sent from the user terminal to the server. The request content is analyzed and the next action is determined.

[0226] Step 6:

[0227] Performing remote operations

[0228] The server performs remote operations based on the user's requests.

[0229] Input: User request (e.g., "Lock the door")

[0230] Output: Result of remote operation (e.g. door locked)

[0231] Specific operation: The server calls the corresponding API to change the state of the device (e.g., locking the door using the smart lock's API).

[0232] Step 7:

[0233] Feedback Generation

[0234] The server records all data and interactions and generates feedback to improve the generative AI model.

[0235] Input: All collected data and conversations

[0236] Output: Feedback data for improving the generative AI model

[0237] Specific operation: The server analyzes the operation history and dialogue content to generate feedback as learning data for the generative AI model.

[0238] Each step ensures smooth operation of the entire system and automates the entire process from anomaly detection to countermeasure implementation, thereby streamlining users' security management.

[0239] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0240] MODE FOR CARRYING OUT THE INVENTION

[0241] overview

[0242] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with an emotion engine, and is particularly designed to combat burglaries and provide countermeasures tailored to the owner's emotions. The system analyzes environmental data collected from monitoring devices in real time, detects abnormal behavior or intrusions, and notifies the owner. The emotion engine also analyzes the owner's emotional state and provides personalized notifications and advice.

[0243] System Configuration

[0244] The server is the core of the system, collecting data from various sensors and surveillance cameras and performing real-time analysis using a generative AI model and emotion engine. If an abnormality is detected, it promptly notifies the owner and provides specific countermeasures in an interactive format upon request. It also adjusts the content of the notification and suggested countermeasures according to the owner's emotional state.

[0245] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[0246] The emotion engine analyzes emotions from the owner's voice, text, facial expressions, etc. to understand the owner's psychological state, enabling it to respond appropriately according to their emotional state.

[0247] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[0248] Program processing

[0249] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. This notification includes details of the abnormality and suggestions for immediate countermeasures. The server also uses an emotion engine to analyze the owner's emotional state and adjust appropriate responses.

[0250] The owner receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the door is open," the server determines whether the living room door is open based on the latest sensor and camera data and responds accordingly. In response to a command such as "Please lock the doors," the server remotely locks all doors and reports the results to the owner. The emotion engine then provides appropriate advice and additional information to reduce the owner's anxiety and tension.

[0251] Specific examples

[0252] Example 1: Absence anomaly detection and emotional response

[0253] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0254] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0255] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[0256] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[0257] 4. The emotion engine analyzes the owner's emotional state and determines that the owner is nervous. Based on this, a message encouraging calmness is added.

[0258] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[0259] 6. Based on the latest data, the server sends the owner the fact that the door is open and the camera footage, reporting the specific situation. The emotion engine explains the situation in an easy-to-understand manner to reassure the owner.

[0260] 7. The user sends the command "Lock the front door."

[0261] 8. The server remotely locks the front door and reports, "The front door is locked." At the same time, the emotion engine sends additional information and suggested actions to ease the owner's tension.

[0262] Example 2: Daily fail-safe checks and emotional responses

[0263] Consider a case where a user checks the security status of their home before going to bed.

[0264] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0265] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[0266] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[0267] 4. The emotion engine analyzes the owner's emotional state and adds messages to provide comfort.

[0268] In this way, by combining an emotion engine, the present invention provides notifications and countermeasures that are optimized according to the owner's psychological state, realizing more advanced and reassuring home security.

[0269] The processing flow will be explained below.

[0270] Step 1:

[0271] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[0272] Step 2:

[0273] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[0274] Step 3:

[0275] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[0276] Step 4:

[0277] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[0278] Step 5:

[0279] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[0280] Step 6:

[0281] The server uses the owner's emotion engine to analyze the owner's emotional state. It analyzes the owner's voice and text to determine whether the owner is feeling anxious or nervous. It also analyzes the owner's facial expressions using a surveillance camera to complement the owner's emotional state.

[0282] Step 7:

[0283] Based on the owner's emotional state as analyzed by the emotion engine, the server tailors the content of notifications and advice. For example, if the owner is nervous, it will provide a message encouraging them to stay calm or provide additional reassurance.

[0284] Step 8:

[0285] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[0286] Step 9:

[0287] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it may check whether the living room door is open or closed and report the results to the owner. The emotion engine also provides appropriate explanations and advice that take the owner's emotions into consideration.

[0288] Step 10:

[0289] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[0290] Step 11:

[0291] The server remotely executes specific measures based on the owner's instructions, such as remotely locking all doors and notifying the owner of the results. The emotion engine then sends additional information and suggested measures to ease the owner's tension.

[0292] Step 12:

[0293] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[0294] Example 2

[0295] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0296] Conventional home security systems focus on collecting environmental data and detecting abnormal behavior, but are indifferent to the owner's psychological state. As a result, when an abnormality is detected, the owner often becomes overly tense or anxious, which can delay appropriate response. There is also a need for a system that can provide appropriate countermeasures based on the owner's emotional state.

[0297] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0298] In this invention, the server includes means for collecting environmental data from the monitoring device, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for analyzing the owner's emotional state and providing countermeasures according to the emotional state, means for checking the situation and providing advice on countermeasures based on the owner's request, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to provide an optimal response according to the owner's psychological state, thereby reducing anxiety and tension.

[0299] "Monitoring devices" are devices used to collect environmental data, such as sensors and surveillance cameras.

[0300] "Environmental data" refers to data that indicates the physical conditions within the home, and includes temperature, sound, whether doors are open or closed, and video.

[0301] "Generative AI models" refer to models that use machine learning and neural networks to detect anomalous behavior and intrusions based on collected data.

[0302] "Real-time analysis" refers to the process of instantly analyzing collected data and quickly determining abnormalities and instructions for the next step.

[0303] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is deemed to pose a potential security risk.

[0304] "Breaking" refers to the act of attempting to gain unauthorized entry into a home.

[0305] "Owner" refers to the person who manages and uses the security system in a home.

[0306] "Notification" means a message sent by the System to the Owner, including a warning or information.

[0307] An "emotion engine" refers to an algorithm or component that analyzes the owner's emotional state and provides corresponding responses.

[0308] "Request" refers to a specific instruction or request for information made by the Owner to the System.

[0309] "Dialogue content" refers to a record of the information exchanged, instructions, and responses between the owner and the system.

[0310] "Feedback" refers to information used to improve system performance and review appropriate countermeasures based on past dialogue and data.

[0311] MODE FOR CARRYING OUT THE INVENTION

[0312] This invention is a system that enhances home security using environmental data collected from monitoring devices. Specifically, the operation of the system, which is centered around a server, terminals, and users, will be described below.

[0313] The server is the heart of the system and works by:

[0314] 1. Collecting data from monitoring devices

[0315] The server collects data from monitoring devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras. These devices transmit data to the server using communication protocols such as Bluetooth, Wi-Fi, and Zigbee. The environmental data is based on specific data patterns and includes sensor status information and video data.

[0316] 2. Real-time analysis of data

[0317] The server inputs the collected data into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior and intrusions. The AI ​​model may be implemented using machine learning libraries such as OpenCV or TensorFlow.

[0318] 3. Anomaly detection and notification

[0319] The server evaluates anomalies based on the analysis results from the AI ​​model. If an anomaly is detected, it sends a notification to the owner via a mobile app, SMS, email, etc. The notification includes a description of the anomaly and a suggestion for immediate action.

[0320] 4. Owner's emotional state analysis

[0321] The server uses an emotion engine to analyze the owner's emotional state. It extracts emotions from voice, text data, and facial expressions, and provides appropriate advice and countermeasures based on the results. The emotion engine incorporates NLP (natural language processing) and voice recognition technology.

[0322] 5. Processing Owner Requests

[0323] When an abnormality is detected, the user can send a request to the server. For example, a request might be, "Please check if the living room door is open." The server will respond with information based on the latest sensor and camera data. In response to a command such as, "Please lock the doors," the server will remotely lock all doors and report the results to their owners.

[0324] 6. Recording data and dialogue and generating feedback

[0325] The server records all data and interactions, generating feedback to improve the generative AI model, allowing the system to respond more intelligently to the owner's individual behavior, providing enhanced security and peace of mind.

[0326] A specific scenario would be:

[0327] Example 1: Absence anomaly detection and emotional response

[0328] The following example shows a case where the front door is opened unexpectedly late at night, immediately after the user has left the house.

[0329] 1. The server detects abnormal opening and closing behavior from the door sensor, and the surveillance camera captures footage of the entrance.

[0330] 2. The server inputs the collected data into a generative AI model for real-time analysis, which then scores the behavior as an anomaly.

[0331] 3. If the server detects an abnormality, it immediately sends a notification to the owner saying, "The front door has been opened in an unnatural way."

[0332] 4. The emotion engine analyzes the owner's emotional state, and if it determines that the owner is tense, it sends an additional message encouraging the owner to stay calm.

[0333] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[0334] 6. Based on the latest data, the server sends the owner information such as whether the door is open and camera footage, and reports the specific situation. The emotion engine sends additional information to encourage calm.

[0335] 7. The user sends the command "Lock the front door."

[0336] 8. The server remotely locks the front door and reports, "The front door is locked." The emotion engine also sends additional information and suggested actions to ease the owner's tension.

[0337] Example 2: Daily fail-safe checks and emotional responses

[0338] The following describes a case where a user checks the security status of his or her home before going to bed.

[0339] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0340] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[0341] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[0342] 4. The emotion engine analyzes the owner's emotional state and sends additional messages to provide reassurance.

[0343] In this way, the present invention is a system that combines an emotion engine to provide optimal notifications and responses according to the owner's psychological state, achieving both security and a sense of security.

[0344] Examples of prompt statements

[0345] 1. "What is the security situation at your home?"

[0346] 2. "Make sure the front door is open."

[0347] 3. "What should you do if signs of intrusion are detected?"

[0348] 4. "I'm feeling anxious, so please set an alarm."

[0349] 5. "Check the living room camera feed."

[0350] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0351] Program processing steps

[0352] Step 1:

[0353] A means by which the server collects environmental data from monitoring devices

[0354] Input: Data sent from surveillance devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras.

[0355] Output: Sensor data and video data stored in the server.

[0356] Specific operation: The server polls data from each monitoring device every 10 seconds and collects data using protocols such as Bluetooth, Wi-Fi, and Zigbee.

[0357] Data processing: Environmental data is classified by device and stored in chronological order. For example, if the front door sensor sends data indicating that the door is open, that information is stored in the database.

[0358] Step 2:

[0359] The server inputs the collected data into a generative artificial intelligence model and analyzes it in real time.

[0360] Input: Accumulated environmental data.

[0361] Output: Anomaly scores as the analysis result.

[0362] Specific operation: The collected data is input into a generative artificial intelligence model (e.g., a model using TensorFlow or OpenCV) and real-time analysis is performed.

[0363] Data processing: Preprocessing data and converting it into a format suitable for anomaly detection. For example, analyzing door opening and closing behavior at night and scoring whether the behavior deviates from normal behavioral patterns.

[0364] Step 3:

[0365] A method for the server to detect abnormal behavior or intrusions based on analysis results

[0366] Input: Anomaly scores obtained from a generative AI model.

[0367] Output: Whether or not there is an abnormality and its content.

[0368] Specific operation: If the anomaly score exceeds a set threshold, it is determined to be an anomaly. For example, an anomaly score of 80 or more is considered an anomaly, and it is determined that there is a high possibility of intrusion.

[0369] Data processing: Evaluate the analysis results and determine the type and urgency of the anomaly. If no anomaly is detected, return to the next cycle.

[0370] Step 4:

[0371] A means for the server to notify the owner of detected anomalies

[0372] Input: Whether or not an anomaly was detected and its details.

[0373] Output: A notification message to the owner.

[0374] Specific operation: If an abnormality is detected, the owner will be notified via SMS, email, or a dedicated app.

[0375] Data processing: Generate a notification message. For example, generate and send a message such as "The front door was opened unexpectedly. All doors have been locked."

[0376] Step 5:

[0377] A means for the server to analyze the owner's emotional state and provide countermeasures according to that emotional state

[0378] Input: Owner's voice, text messages, and facial expression data.

[0379] Output: Emotional state and coping strategies.

[0380] Specific operation: The emotion engine analyzes emotions from the owner's voice and text, extracts emotional data, and generates countermeasures.

[0381] Data processing: Using NLP and speech recognition technology, we analyze emotions and generate advice and messages based on the emotional state. For example, we provide messages encouraging calm, such as "Please stay calm. Would you like to contact the police immediately?"

[0382] Step 6:

[0383] The means by which a user sends a request to a server and the server responds

[0384] Input: A request from the owner, for example, "Make sure the living room door is open."

[0385] Output: The response to the request.

[0386] Specific operation: Upon receiving the owner's request, the server analyzes the information based on the latest sensor and camera data and responds to the owner.

[0387] Data processing: Parse the request, collect relevant data, and generate a specific response message for the owner, for example, "The living room door is open."

[0388] Step 7:

[0389] A means for the server to record all data and interactions and generate feedback to improve the AI ​​model

[0390] Input: Sensor data, dialogue content, system response results.

[0391] Output: Feedback data, new learning model.

[0392] Specific operation: All sensor data and conversations are logged, and feedback data is generated to improve the performance of AI models.

[0393] Data processing: Analyze all collected data and create a feedback loop to retrain the model. For example, analyze reaction times when an anomaly is detected or changes in the owner's emotions to identify areas for improvement in the model or system.

[0394] (Application example 2)

[0395] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0396] In recent years, there has been a demand for security systems to improve safety within homes. However, existing systems are limited to detecting abnormal behavior and intrusions, and are unable to respond flexibly to the owner's psychological state. Furthermore, they lack the ability to fully implement real-time environmental monitoring and remote control, making it impossible to completely eliminate the owner's sense of anxiety. Given this background, there is a need for a security system that not only detects anomalies but also provides optimal countermeasures based on emotional analysis, thereby increasing the sense of security.

[0397] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0398] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected abnormality, means for checking the situation and providing advice on countermeasures based on the owner's request, means for analyzing the owner's emotional state using an emotion engine and optimizing the notification content and countermeasures, means for monitoring the home environmental conditions in real time and performing remote control as necessary, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This not only enables anomaly detection but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[0399] A "surveillance device" is a piece of equipment used to collect environmental data, including sensors, cameras, and other devices.

[0400] "Environmental data" refers to data collected from monitoring devices that indicates conditions inside and outside the home, and includes, for example, information on temperature, sound, and movement.

[0401] A "generative artificial intelligence model" is a model that uses machine learning and deep learning techniques to analyze collected environmental data and detect anomalous behavior and intrusions.

[0402] "Means for analyzing in real time" refers to a processing method for instantly processing collected data and outputting the results.

[0403] "Abnormal behavior" refers to actions or movements that deviate from the normal range of operation, such as intrusions or acts of vandalism in a surveillance system.

[0404] "Means of notification" refers to a method for notifying the owner of an abnormal behavior or intrusion when such behavior or intrusion is detected.

[0405] An "owner request" is a request for confirmation or measures sent by the owner to the system.

[0406] "Confirming the situation" and "advice on countermeasures" refer to operations to check the status of the current monitoring environment and to provide advice on countermeasures for abnormal situations.

[0407] The "emotion engine" is a system that analyzes the owner's psychological state and adjusts notification content and countermeasures based on that information.

[0408] "Remote control" is a function that allows the owner to send commands to the monitoring system and operate it from a remote location.

[0409] "Data and interaction recording" means that the system stores all data collected and interactions with the owner.

[0410] The "means for generating feedback" is a method for generating information to improve the performance of the generative artificial intelligence model based on the recorded data.

[0411] The program of the system that realizes this application example is configured in the following way.

[0412] First, the server collects environmental data from monitoring devices, such as temperature sensors, sound sensors, motion sensors, and surveillance cameras. The environmental data collected by these devices is sent to the server.

[0413] The server then inputs the collected data into a generative AI model that analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning techniques, and is capable of detecting anomalous behavior and intrusions with high accuracy.

[0414] If data analysis detects any abnormal behavior or intrusion, the server will send a notification to the owner's smartphone, including the specific location and circumstances of the anomaly.

[0415] Additionally, the emotion engine analyzes the owner's voice and text input to understand their emotional state. For example, if voice analysis determines that the owner is feeling anxious, an additional message to help them stay calm will be displayed.

[0416] According to the owner's request, the server will check the situation and provide advice on countermeasures. For example, if the owner requests, "Check the status of the front door," the server will check whether the door is open or closed based on the latest sensor data and camera footage and report the status to the owner. Also, if the owner instructs, "Lock the door," the server will remotely lock the door and notify the owner of the result.

[0417] All this data and conversation content is recorded on the server and used as feedback to improve the generative AI model.

[0418] Hardware and Software

[0419] Hardware: Smartphone, home sensors (door open / close sensor, temperature sensor, sound sensor, surveillance camera)

[0420] Software: EmotionEngine (emotion engine), AIModel (generative artificial intelligence model), communication library between server and client (e.g., requests)

[0421] Specific examples

[0422] Nighttime anomaly detection and countermeasures

[0423] 1. Late at night, the sensor on the front door detects abnormal opening and closing behavior and sends the data to the server.

[0424] 2. The server analyzes using a generative artificial intelligence model and detects anomalies with a high probability.

[0425] 3. The owner will receive a notification that their front door has been opened in an unnatural manner.

[0426] 4. The emotion engine analyzes the owner's psychological state and displays a message encouraging them to stay calm (e.g., "Please stay calm").

[0427] 5. When the owner requests, "Check the status of the front door," the server reports the situation based on camera footage and the latest sensor data.

[0428] 6. When the owner commands, "Lock the front door," the server remotely locks the door and notifies the owner, "The front door has been locked."

[0429] Prompt Sentence Examples

[0430] The user types "Check the status of the front door" into their smartphone.

[0431] The server analyzes the situation based on the latest camera footage and sensor data.

[0432] The app notifies you, "The front door is safely closed. No particular abnormalities are observed."

[0433] In this way, this invention realizes a system that not only detects abnormalities but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[0434] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0435] Step 1:

[0436] The server collects environmental data from the monitoring devices. The collected data includes temperature, sound, motion information, and camera footage. This data is sent to the server in real time by the monitoring devices. The server's input is the environmental data sent from the monitoring devices, and its output is the storage of the collected data and preparation for analysis.

[0437] Step 2:

[0438] The server inputs the collected environmental data into a generative AI model and analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning technologies and detects abnormal behavior and intrusions with high accuracy. The input at this stage is environmental data, and the output is the analysis results. Specifically, an anomaly score is calculated and the presence or absence of anomalies is determined based on that score.

[0439] Step 3:

[0440] The server detects abnormal behavior or intrusions based on the analysis results. If the anomaly score exceeds a certain threshold, the system determines this to be an anomaly. The input to this step is the analysis result of the generative AI model, and the output is the anomaly detection result. Specifically, it detects anomalies such as "unnatural door opening and closing" or "suspicious noises."

[0441] Step 4:

[0442] The server notifies the owner of the detected abnormality. The notification includes the specific location and circumstances of the abnormality. In this step, the input is the abnormality detection result, and the output is a notification message to the owner. Specifically, a notification such as "The front door was opened in an unnatural way" is sent to the smartphone.

[0443] Step 5:

[0444] The emotion engine analyzes the owner's voice and text input to understand the owner's emotional state. The server adjusts the content of notifications and countermeasure messages based on the results of the emotion analysis. The input for this step is the owner's voice and text data, and the output is the adjusted message. Specifically, if the owner is nervous, "advice to stay calm" is added.

[0445] Step 6:

[0446] Based on the owner's request, the server checks the situation and provides advice on countermeasures. The input for this step is the owner's request, and the output is a situation report and advice on countermeasures. Specifically, in response to a request such as "Check the status of the front door," the server reports the situation based on the latest sensor data and camera footage.

[0447] Step 7:

[0448] The server executes the owner's remote control instructions. For example, if the owner commands "lock the doors," the server remotely locks the doors and notifies the owner of the result. The input of this step is the remote control instruction, and the output is the execution result. Specifically, a confirmation message such as "All doors have been locked" is sent to the owner.

[0449] Step 8:

[0450] The server records all data and dialogue, generates feedback, and improves the AI ​​model. The input for this step is sensor data and dialogue, and the output is an improved AI model. Specifically, the model is retrained based on the analysis results and owner feedback to improve the accuracy of the entire system.

[0451] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0452] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0453] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0454] [Second embodiment]

[0455] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0456] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0457] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0458] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0459] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0460] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0461] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0462] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0463] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0464] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0465] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0466] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0467] MODE FOR CARRYING OUT THE INVENTION

[0468] overview

[0469] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the monitoring device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing interactive countermeasures.

[0470] System Configuration

[0471] The server is the core of the system, collecting data from various sensors and surveillance cameras, and performing real-time analysis using a generative artificial intelligence model. If an abnormality is detected, it immediately notifies the owner and interactively provides specific countermeasures upon request.

[0472] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[0473] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[0474] Program processing

[0475] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action.

[0476] The owner receives a notification and sends a specific request to the server. For example, if the request is "Please check if the door is open," the server will determine whether the living room door is open based on the latest sensor and camera data and respond to the owner. Alternatively, if the request is "Please lock the doors," the server will remotely lock all doors and report the result to the owner.

[0477] Specific examples

[0478] Example 1: Detecting and responding to anomalies during absence

[0479] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0480] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0481] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[0482] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[0483] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[0484] 5. Based on the latest data, the server sends the door open status and camera footage to the owner, reporting the specific situation.

[0485] 6. The user sends the command "Lock the front door."

[0486] 7. The server remotely locks the front door and reports "The front door is locked."

[0487] Example 2: Daily fail-safe checks

[0488] Consider a case where a user checks the security status of their home before going to bed.

[0489] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0490] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[0491] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[0492] In this way, the present invention functions as an advanced security system that can safely protect a home even when the owner is away or asleep.

[0493] The processing flow will be explained below.

[0494] Step 1:

[0495] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[0496] Step 2:

[0497] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[0498] Step 3:

[0499] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[0500] Step 4:

[0501] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[0502] Step 5:

[0503] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[0504] Step 6:

[0505] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[0506] Step 7:

[0507] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it can check whether the living room door is open or closed and report the results to the owner.

[0508] Step 8:

[0509] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[0510] Step 9:

[0511] The server remotely executes specific measures based on the owner's instructions, for example, remotely locking all doors and notifying the owner of the results.

[0512] Step 10:

[0513] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[0514] Example 1

[0515] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0516] The present invention aims to provide a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. Conventional security systems have the difficulty of responding quickly and effectively after detecting an anomaly, leaving security vulnerable when the owner is away or asleep. To solve this problem, a system is needed that can not only detect abnormal behavior and intrusions in real time and promptly notify the owner, but also provide and implement specific countermeasures.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0518] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for analyzing the latest sensor and camera data corresponding to the owner's request and responding to the situation, means for remotely implementing security measures and reporting the results to the owner, and means for recording all data and dialogue, generating feedback, and improving the AI ​​model. This enables a quick and effective response after an anomaly is detected, making it possible to maintain home security with peace of mind even when the owner is away or sleeping.

[0519] A "surveillance device" is a device such as a sensor or camera that collects environmental data.

[0520] The "server" is a central device that consolidates data collected from monitoring devices, analyzes it using generative AI models, detects abnormal behavior and intrusions, notifies users, manages requests, and implements security measures.

[0521] "Environmental data" refers to information that indicates the security situation inside and outside the home, such as whether doors and windows are open or closed, temperature, sound, and video.

[0522] A "generative AI model" is an artificial intelligence algorithm designed to identify anomalous behavior and intrusion patterns based on large amounts of training data.

[0523] "Real-time analysis" means processing data collected from monitoring devices immediately and without delay to detect abnormalities.

[0524] "Abnormal behavior or intrusion" refers to the act of entering a building by illegal means or any unnatural behavior that differs from normal.

[0525] An "owner" is a person using a home security system who receives notifications from the system and sends requests.

[0526] A "request" is a specific confirmation or instruction for action sent from the owner to the server.

[0527] A "sensor" is a device that detects environmental factors such as the opening and closing of doors and windows, temperature, and sound.

[0528] A "camera" is a photographing device for collecting video data.

[0529] "Notification" is a message sent from the server to the owner informing them of an abnormality or security situation.

[0530] "Implementing security measures remotely" means that the server will remotely implement security measures such as locking doors at the owner's request.

[0531] "Feedback" is evaluation information that the system uses to improve the performance of the generated AI model based on the content of the dialogue and data.

[0532] overview

[0533] This invention relates to a home security system that combines a generative AI model with advanced understanding capabilities with a surveillance device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the surveillance device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing specific countermeasures in an interactive format.

[0534] System Configuration

[0535] The system mainly consists of the following three elements:

[0536] 1. Server

[0537] The system's core component collects data from monitoring devices and performs real-time analysis using generative AI models. If an abnormality is detected, it immediately notifies the owner and, upon request, interactively provides specific countermeasures.

[0538] 2. Surveillance Devices

[0539] It consists of surveillance cameras and various sensors (door and window opening / closing sensors, temperature sensors, sound sensors, etc.), each of which has the role of transmitting environmental data to a server.

[0540] 3. User (Owner)

[0541] When an abnormality is detected, you can receive a notification and request specific measures from the server.

[0542] Program processing

[0543] The server inputs environmental data collected from the monitoring devices into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action to be taken.

[0544] The user receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the doors are open," the server determines whether the doors are open based on the latest sensor and camera data and responds to the owner. Similarly, in response to a request such as "Please lock the doors," the server remotely locks all doors and reports the result to the owner.

[0545] Specific examples

[0546] Example 1: Detecting and responding to anomalies during absence

[0547] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0548] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0549] 2. The server inputs the collected data into the generative AI model for real-time analysis. The generative AI model returns a high score for this behavior, identifying it as abnormal.

[0550] 3. If the server detects an abnormality, it immediately sends an SMS to the user notifying them that "The front door has been opened in an unnatural way."

[0551] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[0552] 5. Based on the latest data, the server notifies the user that the door is open and sends camera footage to report the specific situation.

[0553] 6. The user sends the command "Lock the front door."

[0554] 7. The server remotely locks the front door and reports "The front door is locked."

[0555] Example 2: Daily fail-safe checks

[0556] Consider a case where a user checks the security status of their home before going to bed.

[0557] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0558] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative AI model.

[0559] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the user that "all doors and windows are closed and no abnormal sounds have been detected."

[0560] In this way, the present invention functions as an advanced security system that can safely protect the home even when the user is away or asleep.

[0561] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0562] Step 1:

[0563] The server collects environmental data from multiple devices (sensors and surveillance cameras). The input is data from various sensors (e.g., temperature, door opening / closing, sound, video), which is integrated into a database within the server. The output is the integrated environmental data. Since this data is collected in real time, the server always has the latest information.

[0564] Step 2:

[0565] The server inputs the collected environmental data into the generative AI model. The input is environmental data, and each data point is analyzed by the generative AI model. The AI ​​model is trained to identify abnormal and intrusive behavior. The output is an anomaly score assigned to each data point and the analysis results. Specifically, the AI ​​model detects sudden changes in temperature, unnatural opening and closing movements, and loud noises, and scores them as abnormal.

[0566] Step 3:

[0567] The server evaluates the analysis results returned by the generative AI model. The inputs are the anomaly score and the analysis results, and based on these, it determines whether an anomaly has been detected. The output is the specific details of the detected anomaly. For example, it can include specific information such as "the front door was opened unnaturally late at night."

[0568] Step 4:

[0569] If an abnormality is detected, the server will promptly send a notification to the user. The input is the detected abnormality, and the output is the notification to the user (e.g., SMS or dedicated app alert). The notification will include the specific details of the abnormality and a suggestion of immediate action to be taken (e.g., "The front door is open. Please check it.").

[0570] Step 5:

[0571] The user receives a notification and sends a specific request to the server. The input is the user's request (e.g., "Check the status of the front door"), and the output is that the request reaches the server. The specific action is that the user enters the request through the application.

[0572] Step 6:

[0573] The server recollects and analyzes the latest sensor and camera data based on the user's request. The input is the latest sensor and camera data, and the output is a response to the user. For example, specific information such as "The front door is open and a suspicious person was captured on camera" is sent to the user.

[0574] Step 7:

[0575] The server remotely implements security measures based on user instructions. The input is the user's instruction (e.g., "Lock the front door"), and the output is a report of the implemented security measures (e.g., "The front door is locked"). Specifically, the server sends a signal to the door lock control system to lock the door.

[0576] (Application example 1)

[0577] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] Modern home security systems have limited capabilities for detecting intrusions and other anomalies, and lack real-time monitoring and automated user response. Even if users detect an intrusion, it is difficult for them to immediately take appropriate countermeasures. This increases the burden on users and increases security risks. This can lead to delayed responses, especially when users are in remote locations or late at night, when immediate action is required.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0580] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for checking the situation and providing advice on countermeasures based on the owner's request, means for remotely controlling the security status, means for executing instructions from the user in response to an abnormality that has occurred, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to highly automate the entire process from anomaly detection to countermeasure implementation, significantly reducing security risks while reducing the burden on the user.

[0581] "Surveillance devices" refers to various sensors and surveillance cameras used to collect environmental data.

[0582] "Environmental data" refers to data that indicates the physical and operational conditions within a space, including the open / close status of doors, temperature, and sound.

[0583] A "generative artificial intelligence model" is a system that uses machine learning and deep learning to analyze environmental data in real time and includes algorithms for detecting abnormal behavior and intrusions.

[0584] "Real-time analysis" is the process of instantly processing environmental data collected from monitoring devices and detecting abnormalities.

[0585] "Abnormal behavior" is anything that deviates from normal behavior and includes behavior that indicates intrusion or fraud.

[0586] "Notification" refers to the means of transmitting alerts and information to the owner when an abnormality is detected.

[0587] A "request" refers to a request from the owner to the server to check the status or to give instructions on how to deal with the problem.

[0588] "Status check" refers to the operation that the owner performs to check the current security status when an abnormality is detected.

[0589] "Countermeasure advice" refers to the specific countermeasure suggestions provided to the owner by the generative artificial intelligence model when an abnormality is detected.

[0590] "Remote operation means" refers to an interface that allows the owner to operate the various functions of the security system from a remote location.

[0591] "Feedback" is data generated from the dialogue and results of the system's operations that is used to improve the performance of generative artificial intelligence models.

[0592] "Means for improving artificial intelligence models" refers to the process of adjusting algorithms and parameters based on collected data and feedback information to improve the performance of the model.

[0593] MODE FOR CARRYING OUT THE INVENTION

[0594] Overall system configuration

[0595] The present invention is a system that consists of three main components: a monitoring device, a server, and a user terminal.

[0596] Surveillance Devices

[0597] The monitoring devices consist of various sensors (door sensors, window sensors, temperature sensors, sound sensors, etc.) and surveillance cameras for collecting environmental data. These devices are installed to detect abnormal behavior and intrusions within the home, and transmit data to a server in real time.

[0598] server

[0599] The server is the central control device of this system. It integrates environmental data collected from monitoring devices and performs real-time analysis using generative AI models. If abnormal behavior or intrusion is detected, the server immediately notifies the user. It also provides specific situation confirmation and advice on countermeasures based on the user's request.

[0600] The server includes the following features:

[0601] 1. Data collection function: Collects environmental data from monitoring devices.

[0602] 2. Real-time analytics: Analyze data using generative AI models to detect anomalies.

[0603] 3. Notification function: Sends a notification to the user when an abnormality is detected.

[0604] 4. Remote control function: The security status of the room can be controlled remotely according to the user's request.

[0605] 5. Feedback generation function: Record all data and conversations, generate feedback and improve the generative AI model.

[0606] User terminal

[0607] User terminals are devices such as smartphones, tablets, and smart glasses that allow users to interact with the system and check and operate the security status. Users can receive notifications from the server through their terminals and send status checks and countermeasure requests.

[0608] Program processing

[0609] Real-time data collection

[0610] The server consolidates the environmental data collected from each monitoring device, which is sent in formats such as JSON and includes information such as the status of doors and windows, temperature, and sound.

[0611] Real-time analytics

[0612] The data is fed into a generative AI model (e.g., RealTimeAnalyzer) and analyzed in real time. The generative AI model includes algorithms for detecting abnormal behavior and intrusions.

[0613] Anomaly detection and notification

[0614] If the server detects an anomaly based on the analysis results, it will immediately send a notification to the user using a notification service (e.g., Twilio API, Firebase Cloud Messaging). The notification will include details of the anomaly and the measures the user should take.

[0615] Request handling and remote operations

[0616] When a request from a user (e.g., "lock the door") is received, the server calls an API for remote operation and performs the corresponding operation.

[0617] Feedback Generation

[0618] The server records all data and conversations and generates feedback to improve the performance of the generative AI model, which is then used to adjust parameters and algorithms to improve anomaly detection capabilities next time.

[0619] Examples of specific examples and prompts

[0620] Example 1:

[0621] Consider a case where the rear window of a house suddenly opens in the middle of the night while the user is away on a trip. A suspicious person is clearly visible in the frame of a surveillance camera. The generative AI model detects this abnormal behavior with a high score, and the server immediately sends a notification to the user. The user then sends a request via their smartphone to "lock all doors and windows," and the server remotely locks all doors and windows.

[0622] Example 2:

[0623] If the server detects an abnormality such as the front door being opened and closed multiple times while the user is out, it will notify the user that "The front door has been opened and closed multiple times" and immediately lock the door based on the user's request to "lock the front door."

[0624] Example prompt:

[0625] "Analyzes abnormal behavior detected by sensors and cameras during specific times. Abnormal behavior detection: Determines whether a door has been opened or closed, whether a person is visible in the room, or whether an abnormal sound has been recorded."

[0626] This allows for the construction of an effective embodiment of the invention, and the system efficiently automates a series of processes from anomaly detection to countermeasure implementation, thereby reducing the burden on the user.

[0627] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0628] Step 1:

[0629] Data collection

[0630] The server collects environmental data from each monitoring device (door sensor, window sensor, temperature sensor, sound sensor, surveillance camera).

[0631] Input: Real-time data sent from each monitoring device (e.g., door opening, temperature change, sound generation, etc.)

[0632] Output: The integrated results of the collected environmental data (e.g., data in JSON format)

[0633] Specific Operation: The server pulls data from the monitoring devices at regular intervals and generates a consolidated data set.

[0634] Step 2:

[0635] Preparing for data analysis

[0636] The server performs preprocessing to analyze the collected environmental data, extracting only the data necessary for anomaly detection.

[0637] Input: Collected environmental data

[0638] Output: Preprocessed dataset

[0639] Specific operations: filtering unnecessary data, shaping data necessary for anomaly detection (e.g., data normalization)

[0640] Step 3:

[0641] Real-time analytics

[0642] The server passes the preprocessed dataset to a generative AI model to detect anomalous behavior and intrusions.

[0643] Input: Preprocessed dataset

[0644] Output: Anomaly detection result (e.g., anomaly score, if the score is high, an anomaly is detected)

[0645] How it works: The generative AI model analyzes the dataset and generates an anomaly score. High scores indicate anomalous behavior.

[0646] Step 4:

[0647] Sending abnormality notifications

[0648] If an abnormality is detected, the server sends a notification to the user device, which includes details of the abnormality and advice on how to deal with it.

[0649] Input: Anomaly detection result (high score)

[0650] Output: A message to inform the user

[0651] Specific operation: Notifications are sent to the user's smartphone or smart glasses using the Twilio API or Firebase Cloud Messaging.

[0652] Step 5:

[0653] Processing user requests

[0654] After receiving the notification, the user sends a request to the server to check the situation and take action.

[0655] Input: User request (e.g. "Lock the door" or "Check the camera footage")

[0656] Output: The result of sending the user request to the server

[0657] Specific operation: A request is sent from the user terminal to the server. The request content is analyzed and the next action is determined.

[0658] Step 6:

[0659] Performing remote operations

[0660] The server performs remote operations based on the user's requests.

[0661] Input: User request (e.g., "Lock the door")

[0662] Output: Result of remote operation (e.g. door locked)

[0663] Specific operation: The server calls the corresponding API to change the state of the device (e.g., locking the door using the smart lock's API).

[0664] Step 7:

[0665] Feedback Generation

[0666] The server records all data and interactions and generates feedback to improve the generative AI model.

[0667] Input: All collected data and conversations

[0668] Output: Feedback data for improving the generative AI model

[0669] Specific operation: The server analyzes the operation history and dialogue content to generate feedback as learning data for the generative AI model.

[0670] Each step ensures smooth operation of the entire system and automates the entire process from anomaly detection to countermeasure implementation, thereby streamlining users' security management.

[0671] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0672] MODE FOR CARRYING OUT THE INVENTION

[0673] overview

[0674] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with an emotion engine, and is particularly designed to combat burglaries and provide countermeasures tailored to the owner's emotions. The system analyzes environmental data collected from monitoring devices in real time, detects abnormal behavior or intrusions, and notifies the owner. The emotion engine also analyzes the owner's emotional state and provides personalized notifications and advice.

[0675] System Configuration

[0676] The server is the core of the system, collecting data from various sensors and surveillance cameras and performing real-time analysis using a generative AI model and emotion engine. If an abnormality is detected, it promptly notifies the owner and provides specific countermeasures in an interactive format upon request. It also adjusts the content of the notification and suggested countermeasures according to the owner's emotional state.

[0677] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[0678] The emotion engine analyzes emotions from the owner's voice, text, facial expressions, etc. to understand the owner's psychological state, enabling it to respond appropriately according to their emotional state.

[0679] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[0680] Program processing

[0681] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. This notification includes details of the abnormality and suggestions for immediate countermeasures. The server also uses an emotion engine to analyze the owner's emotional state and adjust appropriate responses.

[0682] The owner receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the door is open," the server determines whether the living room door is open based on the latest sensor and camera data and responds accordingly. In response to a command such as "Please lock the doors," the server remotely locks all doors and reports the results to the owner. The emotion engine then provides appropriate advice and additional information to reduce the owner's anxiety and tension.

[0683] Specific examples

[0684] Example 1: Absence anomaly detection and emotional response

[0685] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0686] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0687] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[0688] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[0689] 4. The emotion engine analyzes the owner's emotional state and determines that the owner is nervous. Based on this, a message encouraging calmness is added.

[0690] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[0691] 6. Based on the latest data, the server sends the owner the fact that the door is open and the camera footage, reporting the specific situation. The emotion engine explains the situation in an easy-to-understand manner to reassure the owner.

[0692] 7. The user sends the command "Lock the front door."

[0693] 8. The server remotely locks the front door and reports, "The front door is locked." At the same time, the emotion engine sends additional information and suggested actions to ease the owner's tension.

[0694] Example 2: Daily fail-safe checks and emotional responses

[0695] Consider a case where a user checks the security status of their home before going to bed.

[0696] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0697] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[0698] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[0699] 4. The emotion engine analyzes the owner's emotional state and adds messages to provide comfort.

[0700] In this way, by combining an emotion engine, the present invention provides notifications and countermeasures that are optimized according to the owner's psychological state, realizing more advanced and reassuring home security.

[0701] The processing flow will be explained below.

[0702] Step 1:

[0703] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[0704] Step 2:

[0705] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[0706] Step 3:

[0707] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[0708] Step 4:

[0709] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[0710] Step 5:

[0711] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[0712] Step 6:

[0713] The server uses the owner's emotion engine to analyze the owner's emotional state. It analyzes the owner's voice and text to determine whether the owner is feeling anxious or nervous. It also analyzes the owner's facial expressions using a surveillance camera to complement the owner's emotional state.

[0714] Step 7:

[0715] Based on the owner's emotional state as analyzed by the emotion engine, the server tailors the content of notifications and advice. For example, if the owner is nervous, it will provide a message encouraging them to stay calm or provide additional reassurance.

[0716] Step 8:

[0717] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[0718] Step 9:

[0719] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it may check whether the living room door is open or closed and report the results to the owner. The emotion engine also provides appropriate explanations and advice that take the owner's emotions into consideration.

[0720] Step 10:

[0721] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[0722] Step 11:

[0723] The server remotely executes specific measures based on the owner's instructions, such as remotely locking all doors and notifying the owner of the results. The emotion engine then sends additional information and suggested measures to ease the owner's tension.

[0724] Step 12:

[0725] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[0726] Example 2

[0727] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] Conventional home security systems focus on collecting environmental data and detecting abnormal behavior, but are indifferent to the owner's psychological state. As a result, when an abnormality is detected, the owner often becomes overly tense or anxious, which can delay appropriate response. There is also a need for a system that can provide appropriate countermeasures based on the owner's emotional state.

[0729] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0730] In this invention, the server includes means for collecting environmental data from the monitoring device, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for analyzing the owner's emotional state and providing countermeasures according to the emotional state, means for checking the situation and providing advice on countermeasures based on the owner's request, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to provide an optimal response according to the owner's psychological state, thereby reducing anxiety and tension.

[0731] "Monitoring devices" are devices used to collect environmental data, such as sensors and surveillance cameras.

[0732] "Environmental data" refers to data that indicates the physical conditions within the home, and includes temperature, sound, whether doors are open or closed, and video.

[0733] "Generative AI models" refer to models that use machine learning and neural networks to detect anomalous behavior and intrusions based on collected data.

[0734] "Real-time analysis" refers to the process of instantly analyzing collected data and quickly determining abnormalities and instructions for the next step.

[0735] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is deemed to pose a potential security risk.

[0736] "Breaking" refers to the act of attempting to gain unauthorized entry into a home.

[0737] "Owner" refers to the person who manages and uses the security system in a home.

[0738] "Notification" means a message sent by the System to the Owner, including a warning or information.

[0739] An "emotion engine" refers to an algorithm or component that analyzes the owner's emotional state and provides corresponding responses.

[0740] "Request" refers to a specific instruction or request for information made by the Owner to the System.

[0741] "Dialogue content" refers to a record of the information exchanged, instructions, and responses between the owner and the system.

[0742] "Feedback" refers to information used to improve system performance and review appropriate countermeasures based on past dialogue and data.

[0743] MODE FOR CARRYING OUT THE INVENTION

[0744] This invention is a system that enhances home security using environmental data collected from monitoring devices. Specifically, the operation of the system, which is centered around a server, terminals, and users, will be described below.

[0745] The server is the heart of the system and works by:

[0746] 1. Collecting data from monitoring devices

[0747] The server collects data from monitoring devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras. These devices transmit data to the server using communication protocols such as Bluetooth, Wi-Fi, and Zigbee. The environmental data is based on specific data patterns and includes sensor status information and video data.

[0748] 2. Real-time analysis of data

[0749] The server inputs the collected data into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior and intrusions. The AI ​​model may be implemented using machine learning libraries such as OpenCV or TensorFlow.

[0750] 3. Anomaly detection and notification

[0751] The server evaluates anomalies based on the analysis results from the AI ​​model. If an anomaly is detected, it sends a notification to the owner via a mobile app, SMS, email, etc. The notification includes a description of the anomaly and a suggestion for immediate action.

[0752] 4. Owner's emotional state analysis

[0753] The server uses an emotion engine to analyze the owner's emotional state. It extracts emotions from voice, text data, and facial expressions, and provides appropriate advice and countermeasures based on the results. The emotion engine incorporates NLP (natural language processing) and voice recognition technology.

[0754] 5. Processing Owner Requests

[0755] When an abnormality is detected, the user can send a request to the server. For example, a request might be, "Please check if the living room door is open." The server will respond with information based on the latest sensor and camera data. In response to a command such as, "Please lock the doors," the server will remotely lock all doors and report the results to their owners.

[0756] 6. Recording data and dialogue and generating feedback

[0757] The server records all data and interactions, generating feedback to improve the generative AI model, allowing the system to respond more intelligently to the owner's individual behavior, providing enhanced security and peace of mind.

[0758] A specific scenario would be:

[0759] Example 1: Absence anomaly detection and emotional response

[0760] The following example shows a case where the front door is opened unexpectedly late at night, immediately after the user has left the house.

[0761] 1. The server detects abnormal opening and closing behavior from the door sensor, and the surveillance camera captures footage of the entrance.

[0762] 2. The server inputs the collected data into a generative AI model for real-time analysis, which then scores the behavior as an anomaly.

[0763] 3. If the server detects an abnormality, it immediately sends a notification to the owner saying, "The front door has been opened in an unnatural way."

[0764] 4. The emotion engine analyzes the owner's emotional state, and if it determines that the owner is tense, it sends an additional message encouraging the owner to stay calm.

[0765] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[0766] 6. Based on the latest data, the server sends the owner information such as whether the door is open and camera footage, and reports the specific situation. The emotion engine sends additional information to encourage calm.

[0767] 7. The user sends the command "Lock the front door."

[0768] 8. The server remotely locks the front door and reports, "The front door is locked." The emotion engine also sends additional information and suggested actions to ease the owner's tension.

[0769] Example 2: Daily fail-safe checks and emotional responses

[0770] The following describes a case where a user checks the security status of his or her home before going to bed.

[0771] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0772] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[0773] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[0774] 4. The emotion engine analyzes the owner's emotional state and sends additional messages to provide reassurance.

[0775] In this way, the present invention is a system that combines an emotion engine to provide optimal notifications and responses according to the owner's psychological state, achieving both security and a sense of security.

[0776] Examples of prompt statements

[0777] 1. "What is the security situation at your home?"

[0778] 2. "Make sure the front door is open."

[0779] 3. "What should you do if signs of intrusion are detected?"

[0780] 4. "I'm feeling anxious, so please set an alarm."

[0781] 5. "Check the living room camera feed."

[0782] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0783] Program processing steps

[0784] Step 1:

[0785] A means by which the server collects environmental data from monitoring devices

[0786] Input: Data sent from surveillance devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras.

[0787] Output: Sensor data and video data stored in the server.

[0788] Specific operation: The server polls data from each monitoring device every 10 seconds and collects data using protocols such as Bluetooth, Wi-Fi, and Zigbee.

[0789] Data processing: Environmental data is classified by device and stored in chronological order. For example, if the front door sensor sends data indicating that the door is open, that information is stored in the database.

[0790] Step 2:

[0791] The server inputs the collected data into a generative artificial intelligence model and analyzes it in real time.

[0792] Input: Accumulated environmental data.

[0793] Output: Anomaly scores as the analysis result.

[0794] Specific operation: The collected data is input into a generative artificial intelligence model (e.g., a model using TensorFlow or OpenCV) and real-time analysis is performed.

[0795] Data processing: Preprocessing data and converting it into a format suitable for anomaly detection. For example, analyzing door opening and closing behavior at night and scoring whether the behavior deviates from normal behavioral patterns.

[0796] Step 3:

[0797] A method for the server to detect abnormal behavior or intrusions based on analysis results

[0798] Input: Anomaly scores obtained from a generative AI model.

[0799] Output: Whether or not there is an abnormality and its content.

[0800] Specific operation: If the anomaly score exceeds a set threshold, it is determined to be an anomaly. For example, an anomaly score of 80 or more is considered an anomaly, and it is determined that there is a high possibility of intrusion.

[0801] Data processing: Evaluate the analysis results and determine the type and urgency of the anomaly. If no anomaly is detected, return to the next cycle.

[0802] Step 4:

[0803] A means for the server to notify the owner of detected anomalies

[0804] Input: Whether or not an anomaly was detected and its details.

[0805] Output: A notification message to the owner.

[0806] Specific operation: If an abnormality is detected, the owner will be notified via SMS, email, or a dedicated app.

[0807] Data processing: Generate a notification message. For example, generate and send a message such as "The front door was opened unexpectedly. All doors have been locked."

[0808] Step 5:

[0809] A means for the server to analyze the owner's emotional state and provide countermeasures according to that emotional state

[0810] Input: Owner's voice, text messages, and facial expression data.

[0811] Output: Emotional state and coping strategies.

[0812] Specific operation: The emotion engine analyzes emotions from the owner's voice and text, extracts emotional data, and generates countermeasures.

[0813] Data processing: Using NLP and speech recognition technology, we analyze emotions and generate advice and messages based on the emotional state. For example, we provide messages encouraging calm, such as "Please stay calm. Would you like to contact the police immediately?"

[0814] Step 6:

[0815] The means by which a user sends a request to a server and the server responds

[0816] Input: A request from the owner, for example, "Make sure the living room door is open."

[0817] Output: The response to the request.

[0818] Specific operation: Upon receiving the owner's request, the server analyzes the information based on the latest sensor and camera data and responds to the owner.

[0819] Data processing: Parse the request, collect relevant data, and generate a specific response message for the owner, for example, "The living room door is open."

[0820] Step 7:

[0821] A means for the server to record all data and interactions and generate feedback to improve the AI ​​model

[0822] Input: Sensor data, dialogue content, system response results.

[0823] Output: Feedback data, new learning model.

[0824] Specific operation: All sensor data and conversations are logged, and feedback data is generated to improve the performance of AI models.

[0825] Data processing: Analyze all collected data and create a feedback loop to retrain the model. For example, analyze reaction times when an anomaly is detected or changes in the owner's emotions to identify areas for improvement in the model or system.

[0826] (Application example 2)

[0827] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0828] In recent years, there has been a demand for security systems to improve safety within homes. However, existing systems are limited to detecting abnormal behavior and intrusions, and are unable to respond flexibly to the owner's psychological state. Furthermore, they lack the ability to fully implement real-time environmental monitoring and remote control, making it impossible to completely eliminate the owner's sense of anxiety. Given this background, there is a need for a security system that not only detects anomalies but also provides optimal countermeasures based on emotional analysis, thereby increasing the sense of security.

[0829] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0830] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected abnormality, means for checking the situation and providing advice on countermeasures based on the owner's request, means for analyzing the owner's emotional state using an emotion engine and optimizing the notification content and countermeasures, means for monitoring the home environmental conditions in real time and performing remote control as necessary, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This not only enables anomaly detection but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[0831] A "surveillance device" is a piece of equipment used to collect environmental data, including sensors, cameras, and other devices.

[0832] "Environmental data" refers to data collected from monitoring devices that indicates conditions inside and outside the home, and includes, for example, information on temperature, sound, and movement.

[0833] A "generative artificial intelligence model" is a model that uses machine learning and deep learning techniques to analyze collected environmental data and detect anomalous behavior and intrusions.

[0834] "Means for analyzing in real time" refers to a processing method for instantly processing collected data and outputting the results.

[0835] "Abnormal behavior" refers to actions or movements that deviate from the normal range of operation, such as intrusions or acts of vandalism in a surveillance system.

[0836] "Means of notification" refers to a method for notifying the owner of an abnormal behavior or intrusion when such behavior or intrusion is detected.

[0837] An "owner request" is a request for confirmation or measures sent by the owner to the system.

[0838] "Confirming the situation" and "advice on countermeasures" refer to operations to check the status of the current monitoring environment and to provide advice on countermeasures for abnormal situations.

[0839] The "emotion engine" is a system that analyzes the owner's psychological state and adjusts notification content and countermeasures based on that information.

[0840] "Remote control" is a function that allows the owner to send commands to the monitoring system and operate it from a remote location.

[0841] "Data and interaction recording" means that the system stores all data collected and interactions with the owner.

[0842] The "means for generating feedback" is a method for generating information to improve the performance of the generative artificial intelligence model based on the recorded data.

[0843] The program of the system that realizes this application example is configured in the following way.

[0844] First, the server collects environmental data from monitoring devices, such as temperature sensors, sound sensors, motion sensors, and surveillance cameras. The environmental data collected by these devices is sent to the server.

[0845] The server then inputs the collected data into a generative AI model that analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning techniques, and is capable of detecting anomalous behavior and intrusions with high accuracy.

[0846] If data analysis detects any abnormal behavior or intrusion, the server will send a notification to the owner's smartphone, including the specific location and circumstances of the anomaly.

[0847] Additionally, the emotion engine analyzes the owner's voice and text input to understand their emotional state. For example, if voice analysis determines that the owner is feeling anxious, an additional message to help them stay calm will be displayed.

[0848] According to the owner's request, the server will check the situation and provide advice on countermeasures. For example, if the owner requests, "Check the status of the front door," the server will check whether the door is open or closed based on the latest sensor data and camera footage and report the status to the owner. Also, if the owner instructs, "Lock the door," the server will remotely lock the door and notify the owner of the result.

[0849] All this data and conversation content is recorded on the server and used as feedback to improve the generative AI model.

[0850] Hardware and Software

[0851] Hardware: Smartphone, home sensors (door open / close sensor, temperature sensor, sound sensor, surveillance camera)

[0852] Software: EmotionEngine (emotion engine), AIModel (generative artificial intelligence model), communication library between server and client (e.g., requests)

[0853] Specific examples

[0854] Nighttime anomaly detection and countermeasures

[0855] 1. Late at night, the sensor on the front door detects abnormal opening and closing behavior and sends the data to the server.

[0856] 2. The server analyzes using a generative artificial intelligence model and detects anomalies with a high probability.

[0857] 3. The owner will receive a notification that their front door has been opened in an unnatural manner.

[0858] 4. The emotion engine analyzes the owner's psychological state and displays a message encouraging them to stay calm (e.g., "Please stay calm").

[0859] 5. When the owner requests, "Check the status of the front door," the server reports the situation based on camera footage and the latest sensor data.

[0860] 6. When the owner commands, "Lock the front door," the server remotely locks the door and notifies the owner, "The front door has been locked."

[0861] Prompt Sentence Examples

[0862] The user types "Check the status of the front door" into their smartphone.

[0863] The server analyzes the situation based on the latest camera footage and sensor data.

[0864] The app notifies you, "The front door is safely closed. No particular abnormalities are observed."

[0865] In this way, this invention realizes a system that not only detects abnormalities but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[0866] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0867] Step 1:

[0868] The server collects environmental data from the monitoring devices. The collected data includes temperature, sound, motion information, and camera footage. This data is sent to the server in real time by the monitoring devices. The server's input is the environmental data sent from the monitoring devices, and its output is the storage of the collected data and preparation for analysis.

[0869] Step 2:

[0870] The server inputs the collected environmental data into a generative AI model and analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning technologies and detects abnormal behavior and intrusions with high accuracy. The input at this stage is environmental data, and the output is the analysis results. Specifically, an anomaly score is calculated and the presence or absence of anomalies is determined based on that score.

[0871] Step 3:

[0872] The server detects abnormal behavior or intrusions based on the analysis results. If the anomaly score exceeds a certain threshold, the system determines this to be an anomaly. The input to this step is the analysis result of the generative AI model, and the output is the anomaly detection result. Specifically, it detects anomalies such as "unnatural door opening and closing" or "suspicious noises."

[0873] Step 4:

[0874] The server notifies the owner of the detected abnormality. The notification includes the specific location and circumstances of the abnormality. In this step, the input is the abnormality detection result, and the output is a notification message to the owner. Specifically, a notification such as "The front door was opened in an unnatural way" is sent to the smartphone.

[0875] Step 5:

[0876] The emotion engine analyzes the owner's voice and text input to understand the owner's emotional state. The server adjusts the content of notifications and countermeasure messages based on the results of the emotion analysis. The input for this step is the owner's voice and text data, and the output is the adjusted message. Specifically, if the owner is nervous, "advice to stay calm" is added.

[0877] Step 6:

[0878] Based on the owner's request, the server checks the situation and provides advice on countermeasures. The input for this step is the owner's request, and the output is a situation report and advice on countermeasures. Specifically, in response to a request such as "Check the status of the front door," the server reports the situation based on the latest sensor data and camera footage.

[0879] Step 7:

[0880] The server executes the owner's remote control instructions. For example, if the owner commands "lock the doors," the server remotely locks the doors and notifies the owner of the result. The input of this step is the remote control instruction, and the output is the execution result. Specifically, a confirmation message such as "All doors have been locked" is sent to the owner.

[0881] Step 8:

[0882] The server records all data and dialogue, generates feedback, and improves the AI ​​model. The input for this step is sensor data and dialogue, and the output is an improved AI model. Specifically, the model is retrained based on the analysis results and owner feedback to improve the accuracy of the entire system.

[0883] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0884] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0885] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0886] [Third embodiment]

[0887] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0888] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0889] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0890] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0891] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0892] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0893] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0894] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0895] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0896] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0897] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0898] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0899] MODE FOR CARRYING OUT THE INVENTION

[0900] overview

[0901] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the monitoring device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing interactive countermeasures.

[0902] System Configuration

[0903] The server is the core of the system, collecting data from various sensors and surveillance cameras, and performing real-time analysis using a generative artificial intelligence model. If an abnormality is detected, it immediately notifies the owner and interactively provides specific countermeasures upon request.

[0904] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[0905] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[0906] Program processing

[0907] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action.

[0908] The owner receives a notification and sends a specific request to the server. For example, if the request is "Please check if the door is open," the server will determine whether the living room door is open based on the latest sensor and camera data and respond to the owner. Alternatively, if the request is "Please lock the doors," the server will remotely lock all doors and report the result to the owner.

[0909] Specific examples

[0910] Example 1: Detecting and responding to anomalies during absence

[0911] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0912] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0913] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[0914] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[0915] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[0916] 5. Based on the latest data, the server sends the door open status and camera footage to the owner, reporting the specific situation.

[0917] 6. The user sends the command "Lock the front door."

[0918] 7. The server remotely locks the front door and reports "The front door is locked."

[0919] Example 2: Daily fail-safe checks

[0920] Consider a case where a user checks the security status of their home before going to bed.

[0921] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0922] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[0923] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[0924] In this way, the present invention functions as an advanced security system that can safely protect a home even when the owner is away or asleep.

[0925] The processing flow will be explained below.

[0926] Step 1:

[0927] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[0928] Step 2:

[0929] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[0930] Step 3:

[0931] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[0932] Step 4:

[0933] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[0934] Step 5:

[0935] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[0936] Step 6:

[0937] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[0938] Step 7:

[0939] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it can check whether the living room door is open or closed and report the results to the owner.

[0940] Step 8:

[0941] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[0942] Step 9:

[0943] The server remotely executes specific measures based on the owner's instructions, for example, remotely locking all doors and notifying the owner of the results.

[0944] Step 10:

[0945] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[0946] Example 1

[0947] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0948] The present invention aims to provide a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. Conventional security systems have the difficulty of responding quickly and effectively after detecting an anomaly, leaving security vulnerable when the owner is away or asleep. To solve this problem, a system is needed that can not only detect abnormal behavior and intrusions in real time and promptly notify the owner, but also provide and implement specific countermeasures.

[0949] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0950] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for analyzing the latest sensor and camera data corresponding to the owner's request and responding to the situation, means for remotely implementing security measures and reporting the results to the owner, and means for recording all data and dialogue, generating feedback, and improving the AI ​​model. This enables a quick and effective response after an anomaly is detected, making it possible to maintain home security with peace of mind even when the owner is away or sleeping.

[0951] A "surveillance device" is a device such as a sensor or camera that collects environmental data.

[0952] The "server" is a central device that consolidates data collected from monitoring devices, analyzes it using generative AI models, detects abnormal behavior and intrusions, notifies users, manages requests, and implements security measures.

[0953] "Environmental data" refers to information that indicates the security situation inside and outside the home, such as whether doors and windows are open or closed, temperature, sound, and video.

[0954] A "generative AI model" is an artificial intelligence algorithm designed to identify anomalous behavior and intrusion patterns based on large amounts of training data.

[0955] "Real-time analysis" means processing data collected from monitoring devices immediately and without delay to detect abnormalities.

[0956] "Abnormal behavior or intrusion" refers to the act of entering a building by illegal means or any unnatural behavior that differs from normal.

[0957] An "owner" is a person using a home security system who receives notifications from the system and sends requests.

[0958] A "request" is a specific confirmation or instruction for action sent from the owner to the server.

[0959] A "sensor" is a device that detects environmental factors such as the opening and closing of doors and windows, temperature, and sound.

[0960] A "camera" is a photographing device for collecting video data.

[0961] "Notification" is a message sent from the server to the owner informing them of an abnormality or security situation.

[0962] "Implementing security measures remotely" means that the server will remotely implement security measures such as locking doors at the owner's request.

[0963] "Feedback" is evaluation information that the system uses to improve the performance of the generated AI model based on the content of the dialogue and data.

[0964] overview

[0965] This invention relates to a home security system that combines a generative AI model with advanced understanding capabilities with a surveillance device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the surveillance device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing specific countermeasures in an interactive format.

[0966] System Configuration

[0967] The system mainly consists of the following three elements:

[0968] 1. Server

[0969] The system's core component collects data from monitoring devices and performs real-time analysis using generative AI models. If an abnormality is detected, it immediately notifies the owner and, upon request, interactively provides specific countermeasures.

[0970] 2. Surveillance Devices

[0971] It consists of surveillance cameras and various sensors (door and window opening / closing sensors, temperature sensors, sound sensors, etc.), each of which has the role of transmitting environmental data to a server.

[0972] 3. User (Owner)

[0973] When an abnormality is detected, you can receive a notification and request specific measures from the server.

[0974] Program processing

[0975] The server inputs environmental data collected from the monitoring devices into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action to be taken.

[0976] The user receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the doors are open," the server determines whether the doors are open based on the latest sensor and camera data and responds to the owner. Similarly, in response to a request such as "Please lock the doors," the server remotely locks all doors and reports the result to the owner.

[0977] Specific examples

[0978] Example 1: Detecting and responding to anomalies during absence

[0979] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[0980] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[0981] 2. The server inputs the collected data into the generative AI model for real-time analysis. The generative AI model returns a high score for this behavior, identifying it as abnormal.

[0982] 3. If the server detects an abnormality, it immediately sends an SMS to the user notifying them that "The front door has been opened in an unnatural way."

[0983] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[0984] 5. Based on the latest data, the server notifies the user that the door is open and sends camera footage to report the specific situation.

[0985] 6. The user sends the command "Lock the front door."

[0986] 7. The server remotely locks the front door and reports "The front door is locked."

[0987] Example 2: Daily fail-safe checks

[0988] Consider a case where a user checks the security status of their home before going to bed.

[0989] 1. The user launches the app and sends a request to "check the security status of the entire house."

[0990] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative AI model.

[0991] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the user that "all doors and windows are closed and no abnormal sounds have been detected."

[0992] In this way, the present invention functions as an advanced security system that can safely protect the home even when the user is away or asleep.

[0993] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0994] Step 1:

[0995] The server collects environmental data from multiple devices (sensors and surveillance cameras). The input is data from various sensors (e.g., temperature, door opening / closing, sound, video), which is integrated into a database within the server. The output is the integrated environmental data. Since this data is collected in real time, the server always has the latest information.

[0996] Step 2:

[0997] The server inputs the collected environmental data into the generative AI model. The input is environmental data, and each data point is analyzed by the generative AI model. The AI ​​model is trained to identify abnormal and intrusive behavior. The output is an anomaly score assigned to each data point and the analysis results. Specifically, the AI ​​model detects sudden changes in temperature, unnatural opening and closing movements, and loud noises, and scores them as abnormal.

[0998] Step 3:

[0999] The server evaluates the analysis results returned by the generative AI model. The inputs are the anomaly score and the analysis results, and based on these, it determines whether an anomaly has been detected. The output is the specific details of the detected anomaly. For example, it can include specific information such as "the front door was opened unnaturally late at night."

[1000] Step 4:

[1001] If an abnormality is detected, the server will promptly send a notification to the user. The input is the detected abnormality, and the output is the notification to the user (e.g., SMS or dedicated app alert). The notification will include the specific details of the abnormality and a suggestion of immediate action to be taken (e.g., "The front door is open. Please check it.").

[1002] Step 5:

[1003] The user receives a notification and sends a specific request to the server. The input is the user's request (e.g., "Check the status of the front door"), and the output is that the request reaches the server. The specific action is that the user enters the request through the application.

[1004] Step 6:

[1005] The server recollects and analyzes the latest sensor and camera data based on the user's request. The input is the latest sensor and camera data, and the output is a response to the user. For example, specific information such as "The front door is open and a suspicious person was captured on camera" is sent to the user.

[1006] Step 7:

[1007] The server remotely implements security measures based on user instructions. The input is the user's instruction (e.g., "Lock the front door"), and the output is a report of the implemented security measures (e.g., "The front door is locked"). Specifically, the server sends a signal to the door lock control system to lock the door.

[1008] (Application example 1)

[1009] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] Modern home security systems have limited capabilities for detecting intrusions and other anomalies, and lack real-time monitoring and automated user response. Even if users detect an intrusion, it is difficult for them to immediately take appropriate countermeasures. This increases the burden on users and increases security risks. This can lead to delayed responses, especially when users are in remote locations or late at night, when immediate action is required.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1012] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for checking the situation and providing advice on countermeasures based on the owner's request, means for remotely controlling the security status, means for executing instructions from the user in response to an abnormality that has occurred, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to highly automate the entire process from anomaly detection to countermeasure implementation, significantly reducing security risks while reducing the burden on the user.

[1013] "Surveillance devices" refers to various sensors and surveillance cameras used to collect environmental data.

[1014] "Environmental data" refers to data that indicates the physical and operational conditions within a space, including the open / close status of doors, temperature, and sound.

[1015] A "generative artificial intelligence model" is a system that uses machine learning and deep learning to analyze environmental data in real time and includes algorithms for detecting abnormal behavior and intrusions.

[1016] "Real-time analysis" is the process of instantly processing environmental data collected from monitoring devices and detecting abnormalities.

[1017] "Abnormal behavior" is anything that deviates from normal behavior and includes behavior that indicates intrusion or fraud.

[1018] "Notification" refers to the means of transmitting alerts and information to the owner when an abnormality is detected.

[1019] A "request" refers to a request from the owner to the server to check the status or to give instructions on how to deal with the problem.

[1020] "Status check" refers to the operation that the owner performs to check the current security status when an abnormality is detected.

[1021] "Countermeasure advice" refers to the specific countermeasure suggestions provided to the owner by the generative artificial intelligence model when an abnormality is detected.

[1022] "Remote operation means" refers to an interface that allows the owner to operate the various functions of the security system from a remote location.

[1023] "Feedback" is data generated from the dialogue and results of the system's operations that is used to improve the performance of generative artificial intelligence models.

[1024] "Means for improving artificial intelligence models" refers to the process of adjusting algorithms and parameters based on collected data and feedback information to improve the performance of the model.

[1025] MODE FOR CARRYING OUT THE INVENTION

[1026] Overall system configuration

[1027] The present invention is a system that consists of three main components: a monitoring device, a server, and a user terminal.

[1028] Surveillance Devices

[1029] The monitoring devices consist of various sensors (door sensors, window sensors, temperature sensors, sound sensors, etc.) and surveillance cameras for collecting environmental data. These devices are installed to detect abnormal behavior and intrusions within the home, and transmit data to a server in real time.

[1030] server

[1031] The server is the central control device of this system. It integrates environmental data collected from monitoring devices and performs real-time analysis using generative AI models. If abnormal behavior or intrusion is detected, the server immediately notifies the user. It also provides specific situation confirmation and advice on countermeasures based on the user's request.

[1032] The server includes the following features:

[1033] 1. Data collection function: Collects environmental data from monitoring devices.

[1034] 2. Real-time analytics: Analyze data using generative AI models to detect anomalies.

[1035] 3. Notification function: Sends a notification to the user when an abnormality is detected.

[1036] 4. Remote control function: The security status of the room can be controlled remotely according to the user's request.

[1037] 5. Feedback generation function: Record all data and conversations, generate feedback and improve the generative AI model.

[1038] User terminal

[1039] User terminals are devices such as smartphones, tablets, and smart glasses that allow users to interact with the system and check and operate the security status. Users can receive notifications from the server through their terminals and send status checks and countermeasure requests.

[1040] Program processing

[1041] Real-time data collection

[1042] The server consolidates the environmental data collected from each monitoring device, which is sent in formats such as JSON and includes information such as the status of doors and windows, temperature, and sound.

[1043] Real-time analytics

[1044] The data is fed into a generative AI model (e.g., RealTimeAnalyzer) and analyzed in real time. The generative AI model includes algorithms for detecting abnormal behavior and intrusions.

[1045] Anomaly detection and notification

[1046] If the server detects an anomaly based on the analysis results, it will immediately send a notification to the user using a notification service (e.g., Twilio API, Firebase Cloud Messaging). The notification will include details of the anomaly and the measures the user should take.

[1047] Request handling and remote operations

[1048] When a request from a user (e.g., "lock the door") is received, the server calls an API for remote operation and performs the corresponding operation.

[1049] Feedback Generation

[1050] The server records all data and conversations and generates feedback to improve the performance of the generative AI model, which is then used to adjust parameters and algorithms to improve anomaly detection capabilities next time.

[1051] Examples of specific examples and prompts

[1052] Example 1:

[1053] Consider a case where the rear window of a house suddenly opens in the middle of the night while the user is away on a trip. A suspicious person is clearly visible in the frame of a surveillance camera. The generative AI model detects this abnormal behavior with a high score, and the server immediately sends a notification to the user. The user then sends a request via their smartphone to "lock all doors and windows," and the server remotely locks all doors and windows.

[1054] Example 2:

[1055] If the server detects an abnormality such as the front door being opened and closed multiple times while the user is out, it will notify the user that "The front door has been opened and closed multiple times" and immediately lock the door based on the user's request to "lock the front door."

[1056] Example prompt:

[1057] "Analyzes abnormal behavior detected by sensors and cameras during specific times. Abnormal behavior detection: Determines whether a door has been opened or closed, whether a person is visible in the room, or whether an abnormal sound has been recorded."

[1058] This allows for the construction of an effective embodiment of the invention, and the system efficiently automates a series of processes from anomaly detection to countermeasure implementation, thereby reducing the burden on the user.

[1059] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1060] Step 1:

[1061] Data collection

[1062] The server collects environmental data from each monitoring device (door sensor, window sensor, temperature sensor, sound sensor, surveillance camera).

[1063] Input: Real-time data sent from each monitoring device (e.g., door opening, temperature change, sound generation, etc.)

[1064] Output: The integrated results of the collected environmental data (e.g., data in JSON format)

[1065] Specific Operation: The server pulls data from the monitoring devices at regular intervals and generates a consolidated data set.

[1066] Step 2:

[1067] Preparing for data analysis

[1068] The server performs preprocessing to analyze the collected environmental data, extracting only the data necessary for anomaly detection.

[1069] Input: Collected environmental data

[1070] Output: Preprocessed dataset

[1071] Specific operations: filtering unnecessary data, shaping data necessary for anomaly detection (e.g., data normalization)

[1072] Step 3:

[1073] Real-time analytics

[1074] The server passes the preprocessed dataset to a generative AI model to detect anomalous behavior and intrusions.

[1075] Input: Preprocessed dataset

[1076] Output: Anomaly detection result (e.g., anomaly score, if the score is high, an anomaly is detected)

[1077] How it works: The generative AI model analyzes the dataset and generates an anomaly score. High scores indicate anomalous behavior.

[1078] Step 4:

[1079] Sending abnormality notifications

[1080] If an abnormality is detected, the server sends a notification to the user device, which includes details of the abnormality and advice on how to deal with it.

[1081] Input: Anomaly detection result (high score)

[1082] Output: A message to inform the user

[1083] Specific operation: Notifications are sent to the user's smartphone or smart glasses using the Twilio API or Firebase Cloud Messaging.

[1084] Step 5:

[1085] Processing user requests

[1086] After receiving the notification, the user sends a request to the server to check the situation and take action.

[1087] Input: User request (e.g. "Lock the door" or "Check the camera footage")

[1088] Output: The result of sending the user request to the server

[1089] Specific operation: A request is sent from the user terminal to the server. The request content is analyzed and the next action is determined.

[1090] Step 6:

[1091] Performing remote operations

[1092] The server performs remote operations based on the user's requests.

[1093] Input: User request (e.g., "Lock the door")

[1094] Output: Result of remote operation (e.g. door locked)

[1095] Specific operation: The server calls the corresponding API to change the state of the device (e.g., locking the door using the smart lock's API).

[1096] Step 7:

[1097] Feedback Generation

[1098] The server records all data and interactions and generates feedback to improve the generative AI model.

[1099] Input: All collected data and conversations

[1100] Output: Feedback data for improving the generative AI model

[1101] Specific operation: The server analyzes the operation history and dialogue content to generate feedback as learning data for the generative AI model.

[1102] Each step ensures smooth operation of the entire system and automates the entire process from anomaly detection to countermeasure implementation, thereby streamlining users' security management.

[1103] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1104] MODE FOR CARRYING OUT THE INVENTION

[1105] overview

[1106] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with an emotion engine, and is particularly designed to combat burglaries and provide countermeasures tailored to the owner's emotions. The system analyzes environmental data collected from monitoring devices in real time, detects abnormal behavior or intrusions, and notifies the owner. The emotion engine also analyzes the owner's emotional state and provides personalized notifications and advice.

[1107] System Configuration

[1108] The server is the core of the system, collecting data from various sensors and surveillance cameras and performing real-time analysis using a generative AI model and emotion engine. If an abnormality is detected, it promptly notifies the owner and provides specific countermeasures in an interactive format upon request. It also adjusts the content of the notification and suggested countermeasures according to the owner's emotional state.

[1109] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[1110] The emotion engine analyzes emotions from the owner's voice, text, facial expressions, etc. to understand the owner's psychological state, enabling it to respond appropriately according to their emotional state.

[1111] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[1112] Program processing

[1113] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. This notification includes details of the abnormality and suggestions for immediate countermeasures. The server also uses an emotion engine to analyze the owner's emotional state and adjust appropriate responses.

[1114] The owner receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the door is open," the server determines whether the living room door is open based on the latest sensor and camera data and responds accordingly. In response to a command such as "Please lock the doors," the server remotely locks all doors and reports the results to the owner. The emotion engine then provides appropriate advice and additional information to reduce the owner's anxiety and tension.

[1115] Specific examples

[1116] Example 1: Absence anomaly detection and emotional response

[1117] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[1118] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[1119] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[1120] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[1121] 4. The emotion engine analyzes the owner's emotional state and determines that the owner is nervous. Based on this, a message encouraging calmness is added.

[1122] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[1123] 6. Based on the latest data, the server sends the owner the fact that the door is open and the camera footage, reporting the specific situation. The emotion engine explains the situation in an easy-to-understand manner to reassure the owner.

[1124] 7. The user sends the command "Lock the front door."

[1125] 8. The server remotely locks the front door and reports, "The front door is locked." At the same time, the emotion engine sends additional information and suggested actions to ease the owner's tension.

[1126] Example 2: Daily fail-safe checks and emotional responses

[1127] Consider a case where a user checks the security status of their home before going to bed.

[1128] 1. The user launches the app and sends a request to "check the security status of the entire house."

[1129] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[1130] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[1131] 4. The emotion engine analyzes the owner's emotional state and adds messages to provide comfort.

[1132] In this way, by combining an emotion engine, the present invention provides notifications and countermeasures that are optimized according to the owner's psychological state, realizing more advanced and reassuring home security.

[1133] The processing flow will be explained below.

[1134] Step 1:

[1135] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[1136] Step 2:

[1137] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[1138] Step 3:

[1139] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[1140] Step 4:

[1141] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[1142] Step 5:

[1143] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[1144] Step 6:

[1145] The server uses the owner's emotion engine to analyze the owner's emotional state. It analyzes the owner's voice and text to determine whether the owner is feeling anxious or nervous. It also analyzes the owner's facial expressions using a surveillance camera to complement the owner's emotional state.

[1146] Step 7:

[1147] Based on the owner's emotional state as analyzed by the emotion engine, the server tailors the content of notifications and advice. For example, if the owner is nervous, it will provide a message encouraging them to stay calm or provide additional reassurance.

[1148] Step 8:

[1149] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[1150] Step 9:

[1151] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it may check whether the living room door is open or closed and report the results to the owner. The emotion engine also provides appropriate explanations and advice that take the owner's emotions into consideration.

[1152] Step 10:

[1153] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[1154] Step 11:

[1155] The server remotely executes specific measures based on the owner's instructions, such as remotely locking all doors and notifying the owner of the results. The emotion engine then sends additional information and suggested measures to ease the owner's tension.

[1156] Step 12:

[1157] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[1158] Example 2

[1159] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1160] Conventional home security systems focus on collecting environmental data and detecting abnormal behavior, but are indifferent to the owner's psychological state. As a result, when an abnormality is detected, the owner often becomes overly tense or anxious, which can delay appropriate response. There is also a need for a system that can provide appropriate countermeasures based on the owner's emotional state.

[1161] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1162] In this invention, the server includes means for collecting environmental data from the monitoring device, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for analyzing the owner's emotional state and providing countermeasures according to the emotional state, means for checking the situation and providing advice on countermeasures based on the owner's request, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to provide an optimal response according to the owner's psychological state, thereby reducing anxiety and tension.

[1163] "Monitoring devices" are devices used to collect environmental data, such as sensors and surveillance cameras.

[1164] "Environmental data" refers to data that indicates the physical conditions within the home, and includes temperature, sound, whether doors are open or closed, and video.

[1165] "Generative AI models" refer to models that use machine learning and neural networks to detect anomalous behavior and intrusions based on collected data.

[1166] "Real-time analysis" refers to the process of instantly analyzing collected data and quickly determining abnormalities and instructions for the next step.

[1167] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is deemed to pose a potential security risk.

[1168] "Breaking" refers to the act of attempting to gain unauthorized entry into a home.

[1169] "Owner" refers to the person who manages and uses the security system in a home.

[1170] "Notification" means a message sent by the System to the Owner, including a warning or information.

[1171] An "emotion engine" refers to an algorithm or component that analyzes the owner's emotional state and provides corresponding responses.

[1172] "Request" refers to a specific instruction or request for information made by the Owner to the System.

[1173] "Dialogue content" refers to a record of the information exchanged, instructions, and responses between the owner and the system.

[1174] "Feedback" refers to information used to improve system performance and review appropriate countermeasures based on past dialogue and data.

[1175] MODE FOR CARRYING OUT THE INVENTION

[1176] This invention is a system that enhances home security using environmental data collected from monitoring devices. Specifically, the operation of the system, which is centered around a server, terminals, and users, will be described below.

[1177] The server is the heart of the system and works by:

[1178] 1. Collecting data from monitoring devices

[1179] The server collects data from monitoring devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras. These devices transmit data to the server using communication protocols such as Bluetooth, Wi-Fi, and Zigbee. The environmental data is based on specific data patterns and includes sensor status information and video data.

[1180] 2. Real-time analysis of data

[1181] The server inputs the collected data into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior and intrusions. The AI ​​model may be implemented using machine learning libraries such as OpenCV or TensorFlow.

[1182] 3. Anomaly detection and notification

[1183] The server evaluates anomalies based on the analysis results from the AI ​​model. If an anomaly is detected, it sends a notification to the owner via a mobile app, SMS, email, etc. The notification includes a description of the anomaly and a suggestion for immediate action.

[1184] 4. Owner's emotional state analysis

[1185] The server uses an emotion engine to analyze the owner's emotional state. It extracts emotions from voice, text data, and facial expressions, and provides appropriate advice and countermeasures based on the results. The emotion engine incorporates NLP (natural language processing) and voice recognition technology.

[1186] 5. Processing Owner Requests

[1187] When an abnormality is detected, the user can send a request to the server. For example, a request might be, "Please check if the living room door is open." The server will respond with information based on the latest sensor and camera data. In response to a command such as, "Please lock the doors," the server will remotely lock all doors and report the results to their owners.

[1188] 6. Recording data and dialogue and generating feedback

[1189] The server records all data and interactions, generating feedback to improve the generative AI model, allowing the system to respond more intelligently to the owner's individual behavior, providing enhanced security and peace of mind.

[1190] A specific scenario would be:

[1191] Example 1: Absence anomaly detection and emotional response

[1192] The following example shows a case where the front door is opened unexpectedly late at night, immediately after the user has left the house.

[1193] 1. The server detects abnormal opening and closing behavior from the door sensor, and the surveillance camera captures footage of the entrance.

[1194] 2. The server inputs the collected data into a generative AI model for real-time analysis, which then scores the behavior as an anomaly.

[1195] 3. If the server detects an abnormality, it immediately sends a notification to the owner saying, "The front door has been opened in an unnatural way."

[1196] 4. The emotion engine analyzes the owner's emotional state, and if it determines that the owner is tense, it sends an additional message encouraging the owner to stay calm.

[1197] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[1198] 6. Based on the latest data, the server sends the owner information such as whether the door is open and camera footage, and reports the specific situation. The emotion engine sends additional information to encourage calm.

[1199] 7. The user sends the command "Lock the front door."

[1200] 8. The server remotely locks the front door and reports, "The front door is locked." The emotion engine also sends additional information and suggested actions to ease the owner's tension.

[1201] Example 2: Daily fail-safe checks and emotional responses

[1202] The following describes a case where a user checks the security status of his or her home before going to bed.

[1203] 1. The user launches the app and sends a request to "check the security status of the entire house."

[1204] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[1205] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[1206] 4. The emotion engine analyzes the owner's emotional state and sends additional messages to provide reassurance.

[1207] In this way, the present invention is a system that combines an emotion engine to provide optimal notifications and responses according to the owner's psychological state, achieving both security and a sense of security.

[1208] Examples of prompt statements

[1209] 1. "What is the security situation at your home?"

[1210] 2. "Make sure the front door is open."

[1211] 3. "What should you do if signs of intrusion are detected?"

[1212] 4. "I'm feeling anxious, so please set an alarm."

[1213] 5. "Check the living room camera feed."

[1214] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1215] Program processing steps

[1216] Step 1:

[1217] A means by which the server collects environmental data from monitoring devices

[1218] Input: Data sent from surveillance devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras.

[1219] Output: Sensor data and video data stored in the server.

[1220] Specific operation: The server polls data from each monitoring device every 10 seconds and collects data using protocols such as Bluetooth, Wi-Fi, and Zigbee.

[1221] Data processing: Environmental data is classified by device and stored in chronological order. For example, if the front door sensor sends data indicating that the door is open, that information is stored in the database.

[1222] Step 2:

[1223] The server inputs the collected data into a generative artificial intelligence model and analyzes it in real time.

[1224] Input: Accumulated environmental data.

[1225] Output: Anomaly scores as the analysis result.

[1226] Specific operation: The collected data is input into a generative artificial intelligence model (e.g., a model using TensorFlow or OpenCV) and real-time analysis is performed.

[1227] Data processing: Preprocessing data and converting it into a format suitable for anomaly detection. For example, analyzing door opening and closing behavior at night and scoring whether the behavior deviates from normal behavioral patterns.

[1228] Step 3:

[1229] A method for the server to detect abnormal behavior or intrusions based on analysis results

[1230] Input: Anomaly scores obtained from a generative AI model.

[1231] Output: Whether or not there is an abnormality and its content.

[1232] Specific operation: If the anomaly score exceeds a set threshold, it is determined to be an anomaly. For example, an anomaly score of 80 or more is considered an anomaly, and it is determined that there is a high possibility of intrusion.

[1233] Data processing: Evaluate the analysis results and determine the type and urgency of the anomaly. If no anomaly is detected, return to the next cycle.

[1234] Step 4:

[1235] A means for the server to notify the owner of detected anomalies

[1236] Input: Whether or not an anomaly was detected and its details.

[1237] Output: A notification message to the owner.

[1238] Specific operation: If an abnormality is detected, the owner will be notified via SMS, email, or a dedicated app.

[1239] Data processing: Generate a notification message. For example, generate and send a message such as "The front door was opened unexpectedly. All doors have been locked."

[1240] Step 5:

[1241] A means for the server to analyze the owner's emotional state and provide countermeasures according to that emotional state

[1242] Input: Owner's voice, text messages, and facial expression data.

[1243] Output: Emotional state and coping strategies.

[1244] Specific operation: The emotion engine analyzes emotions from the owner's voice and text, extracts emotional data, and generates countermeasures.

[1245] Data processing: Using NLP and speech recognition technology, we analyze emotions and generate advice and messages based on the emotional state. For example, we provide messages encouraging calm, such as "Please stay calm. Would you like to contact the police immediately?"

[1246] Step 6:

[1247] The means by which a user sends a request to a server and the server responds

[1248] Input: A request from the owner, for example, "Make sure the living room door is open."

[1249] Output: The response to the request.

[1250] Specific operation: Upon receiving the owner's request, the server analyzes the information based on the latest sensor and camera data and responds to the owner.

[1251] Data processing: Parse the request, collect relevant data, and generate a specific response message for the owner, for example, "The living room door is open."

[1252] Step 7:

[1253] A means for the server to record all data and interactions and generate feedback to improve the AI ​​model

[1254] Input: Sensor data, dialogue content, system response results.

[1255] Output: Feedback data, new learning model.

[1256] Specific operation: All sensor data and conversations are logged, and feedback data is generated to improve the performance of AI models.

[1257] Data processing: Analyze all collected data and create a feedback loop to retrain the model. For example, analyze reaction times when an anomaly is detected or changes in the owner's emotions to identify areas for improvement in the model or system.

[1258] (Application example 2)

[1259] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1260] In recent years, there has been a demand for security systems to improve safety within homes. However, existing systems are limited to detecting abnormal behavior and intrusions, and are unable to respond flexibly to the owner's psychological state. Furthermore, they lack the ability to fully implement real-time environmental monitoring and remote control, making it impossible to completely eliminate the owner's sense of anxiety. Given this background, there is a need for a security system that not only detects anomalies but also provides optimal countermeasures based on emotional analysis, thereby increasing the sense of security.

[1261] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1262] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected abnormality, means for checking the situation and providing advice on countermeasures based on the owner's request, means for analyzing the owner's emotional state using an emotion engine and optimizing the notification content and countermeasures, means for monitoring the home environmental conditions in real time and performing remote control as necessary, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This not only enables anomaly detection but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[1263] A "surveillance device" is a piece of equipment used to collect environmental data, including sensors, cameras, and other devices.

[1264] "Environmental data" refers to data collected from monitoring devices that indicates conditions inside and outside the home, and includes, for example, information on temperature, sound, and movement.

[1265] A "generative artificial intelligence model" is a model that uses machine learning and deep learning techniques to analyze collected environmental data and detect anomalous behavior and intrusions.

[1266] "Means for analyzing in real time" refers to a processing method for instantly processing collected data and outputting the results.

[1267] "Abnormal behavior" refers to actions or movements that deviate from the normal range of operation, such as intrusions or acts of vandalism in a surveillance system.

[1268] "Means of notification" refers to a method for notifying the owner of an abnormal behavior or intrusion when such behavior or intrusion is detected.

[1269] An "owner request" is a request for confirmation or measures sent by the owner to the system.

[1270] "Confirming the situation" and "advice on countermeasures" refer to operations to check the status of the current monitoring environment and to provide advice on countermeasures for abnormal situations.

[1271] The "emotion engine" is a system that analyzes the owner's psychological state and adjusts notification content and countermeasures based on that information.

[1272] "Remote control" is a function that allows the owner to send commands to the monitoring system and operate it from a remote location.

[1273] "Data and interaction recording" means that the system stores all data collected and interactions with the owner.

[1274] The "means for generating feedback" is a method for generating information to improve the performance of the generative artificial intelligence model based on the recorded data.

[1275] The program of the system that realizes this application example is configured in the following way.

[1276] First, the server collects environmental data from monitoring devices, such as temperature sensors, sound sensors, motion sensors, and surveillance cameras. The environmental data collected by these devices is sent to the server.

[1277] The server then inputs the collected data into a generative AI model that analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning techniques, and is capable of detecting anomalous behavior and intrusions with high accuracy.

[1278] If data analysis detects any abnormal behavior or intrusion, the server will send a notification to the owner's smartphone, including the specific location and circumstances of the anomaly.

[1279] Additionally, the emotion engine analyzes the owner's voice and text input to understand their emotional state. For example, if voice analysis determines that the owner is feeling anxious, an additional message to help them stay calm will be displayed.

[1280] According to the owner's request, the server will check the situation and provide advice on countermeasures. For example, if the owner requests, "Check the status of the front door," the server will check whether the door is open or closed based on the latest sensor data and camera footage and report the status to the owner. Also, if the owner instructs, "Lock the door," the server will remotely lock the door and notify the owner of the result.

[1281] All this data and conversation content is recorded on the server and used as feedback to improve the generative AI model.

[1282] Hardware and Software

[1283] Hardware: Smartphone, home sensors (door open / close sensor, temperature sensor, sound sensor, surveillance camera)

[1284] Software: EmotionEngine (emotion engine), AIModel (generative artificial intelligence model), communication library between server and client (e.g., requests)

[1285] Specific examples

[1286] Nighttime anomaly detection and countermeasures

[1287] 1. Late at night, the sensor on the front door detects abnormal opening and closing behavior and sends the data to the server.

[1288] 2. The server analyzes using a generative artificial intelligence model and detects anomalies with a high probability.

[1289] 3. The owner will receive a notification that their front door has been opened in an unnatural manner.

[1290] 4. The emotion engine analyzes the owner's psychological state and displays a message encouraging them to stay calm (e.g., "Please stay calm").

[1291] 5. When the owner requests, "Check the status of the front door," the server reports the situation based on camera footage and the latest sensor data.

[1292] 6. When the owner commands, "Lock the front door," the server remotely locks the door and notifies the owner, "The front door has been locked."

[1293] Prompt Sentence Examples

[1294] The user types "Check the status of the front door" into their smartphone.

[1295] The server analyzes the situation based on the latest camera footage and sensor data.

[1296] The app notifies you, "The front door is safely closed. No particular abnormalities are observed."

[1297] In this way, this invention realizes a system that not only detects abnormalities but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[1298] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1299] Step 1:

[1300] The server collects environmental data from the monitoring devices. The collected data includes temperature, sound, motion information, and camera footage. This data is sent to the server in real time by the monitoring devices. The server's input is the environmental data sent from the monitoring devices, and its output is the storage of the collected data and preparation for analysis.

[1301] Step 2:

[1302] The server inputs the collected environmental data into a generative AI model and analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning technologies and detects abnormal behavior and intrusions with high accuracy. The input at this stage is environmental data, and the output is the analysis results. Specifically, an anomaly score is calculated and the presence or absence of anomalies is determined based on that score.

[1303] Step 3:

[1304] The server detects abnormal behavior or intrusions based on the analysis results. If the anomaly score exceeds a certain threshold, the system determines this to be an anomaly. The input to this step is the analysis result of the generative AI model, and the output is the anomaly detection result. Specifically, it detects anomalies such as "unnatural door opening and closing" or "suspicious noises."

[1305] Step 4:

[1306] The server notifies the owner of the detected abnormality. The notification includes the specific location and circumstances of the abnormality. In this step, the input is the abnormality detection result, and the output is a notification message to the owner. Specifically, a notification such as "The front door was opened in an unnatural way" is sent to the smartphone.

[1307] Step 5:

[1308] The emotion engine analyzes the owner's voice and text input to understand the owner's emotional state. The server adjusts the content of notifications and countermeasure messages based on the results of the emotion analysis. The input for this step is the owner's voice and text data, and the output is the adjusted message. Specifically, if the owner is nervous, "advice to stay calm" is added.

[1309] Step 6:

[1310] Based on the owner's request, the server checks the situation and provides advice on countermeasures. The input for this step is the owner's request, and the output is a situation report and advice on countermeasures. Specifically, in response to a request such as "Check the status of the front door," the server reports the situation based on the latest sensor data and camera footage.

[1311] Step 7:

[1312] The server executes the owner's remote control instructions. For example, if the owner commands "lock the doors," the server remotely locks the doors and notifies the owner of the result. The input of this step is the remote control instruction, and the output is the execution result. Specifically, a confirmation message such as "All doors have been locked" is sent to the owner.

[1313] Step 8:

[1314] The server records all data and dialogue, generates feedback, and improves the AI ​​model. The input for this step is sensor data and dialogue, and the output is an improved AI model. Specifically, the model is retrained based on the analysis results and owner feedback to improve the accuracy of the entire system.

[1315] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1316] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1317] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1318] [Fourth embodiment]

[1319] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1320] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1321] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1322] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1323] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1324] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1325] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1326] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1327] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1328] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1329] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1330] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1331] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1332] MODE FOR CARRYING OUT THE INVENTION

[1333] overview

[1334] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the monitoring device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing interactive countermeasures.

[1335] System Configuration

[1336] The server is the core of the system, collecting data from various sensors and surveillance cameras, and performing real-time analysis using a generative artificial intelligence model. If an abnormality is detected, it immediately notifies the owner and interactively provides specific countermeasures upon request.

[1337] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[1338] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[1339] Program processing

[1340] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action.

[1341] The owner receives a notification and sends a specific request to the server. For example, if the request is "Please check if the door is open," the server will determine whether the living room door is open based on the latest sensor and camera data and respond to the owner. Alternatively, if the request is "Please lock the doors," the server will remotely lock all doors and report the result to the owner.

[1342] Specific examples

[1343] Example 1: Detecting and responding to anomalies during absence

[1344] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[1345] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[1346] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[1347] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[1348] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[1349] 5. Based on the latest data, the server sends the door open status and camera footage to the owner, reporting the specific situation.

[1350] 6. The user sends the command "Lock the front door."

[1351] 7. The server remotely locks the front door and reports "The front door is locked."

[1352] Example 2: Daily fail-safe checks

[1353] Consider a case where a user checks the security status of their home before going to bed.

[1354] 1. The user launches the app and sends a request to "check the security status of the entire house."

[1355] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[1356] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[1357] In this way, the present invention functions as an advanced security system that can safely protect a home even when the owner is away or asleep.

[1358] The processing flow will be explained below.

[1359] Step 1:

[1360] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[1361] Step 2:

[1362] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[1363] Step 3:

[1364] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[1365] Step 4:

[1366] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[1367] Step 5:

[1368] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[1369] Step 6:

[1370] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[1371] Step 7:

[1372] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it can check whether the living room door is open or closed and report the results to the owner.

[1373] Step 8:

[1374] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[1375] Step 9:

[1376] The server remotely executes specific measures based on the owner's instructions, for example, remotely locking all doors and notifying the owner of the results.

[1377] Step 10:

[1378] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[1379] Example 1

[1380] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1381] The present invention aims to provide a home security system that combines a generative AI model with advanced understanding capabilities with a monitoring device, and is particularly effective in preventing burglaries. Conventional security systems have the difficulty of responding quickly and effectively after detecting an anomaly, leaving security vulnerable when the owner is away or asleep. To solve this problem, a system is needed that can not only detect abnormal behavior and intrusions in real time and promptly notify the owner, but also provide and implement specific countermeasures.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1383] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for analyzing the latest sensor and camera data corresponding to the owner's request and responding to the situation, means for remotely implementing security measures and reporting the results to the owner, and means for recording all data and dialogue, generating feedback, and improving the AI ​​model. This enables a quick and effective response after an anomaly is detected, making it possible to maintain home security with peace of mind even when the owner is away or sleeping.

[1384] A "surveillance device" is a device such as a sensor or camera that collects environmental data.

[1385] The "server" is a central device that consolidates data collected from monitoring devices, analyzes it using generative AI models, detects abnormal behavior and intrusions, notifies users, manages requests, and implements security measures.

[1386] "Environmental data" refers to information that indicates the security situation inside and outside the home, such as whether doors and windows are open or closed, temperature, sound, and video.

[1387] A "generative AI model" is an artificial intelligence algorithm designed to identify anomalous behavior and intrusion patterns based on large amounts of training data.

[1388] "Real-time analysis" means processing data collected from monitoring devices immediately and without delay to detect abnormalities.

[1389] "Abnormal behavior or intrusion" refers to the act of entering a building by illegal means or any unnatural behavior that differs from normal.

[1390] An "owner" is a person using a home security system who receives notifications from the system and sends requests.

[1391] A "request" is a specific confirmation or instruction for action sent from the owner to the server.

[1392] A "sensor" is a device that detects environmental factors such as the opening and closing of doors and windows, temperature, and sound.

[1393] A "camera" is a photographing device for collecting video data.

[1394] "Notification" is a message sent from the server to the owner informing them of an abnormality or security situation.

[1395] "Implementing security measures remotely" means that the server will remotely implement security measures such as locking doors at the owner's request.

[1396] "Feedback" is evaluation information that the system uses to improve the performance of the generated AI model based on the content of the dialogue and data.

[1397] overview

[1398] This invention relates to a home security system that combines a generative AI model with advanced understanding capabilities with a surveillance device, and is particularly effective in preventing burglaries. This system strengthens home security by analyzing environmental data collected from the surveillance device in real time, detecting abnormal behavior and intrusions, notifying the owner, and providing specific countermeasures in an interactive format.

[1399] System Configuration

[1400] The system mainly consists of the following three elements:

[1401] 1. Server

[1402] The system's core component collects data from monitoring devices and performs real-time analysis using generative AI models. If an abnormality is detected, it immediately notifies the owner and, upon request, interactively provides specific countermeasures.

[1403] 2. Surveillance Devices

[1404] It consists of surveillance cameras and various sensors (door and window opening / closing sensors, temperature sensors, sound sensors, etc.), each of which has the role of transmitting environmental data to a server.

[1405] 3. User (Owner)

[1406] When an abnormality is detected, you can receive a notification and request specific measures from the server.

[1407] Program processing

[1408] The server inputs environmental data collected from the monitoring devices into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. The notification includes a description of the abnormality and a suggestion for immediate action to be taken.

[1409] The user receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the doors are open," the server determines whether the doors are open based on the latest sensor and camera data and responds to the owner. Similarly, in response to a request such as "Please lock the doors," the server remotely locks all doors and reports the result to the owner.

[1410] Specific examples

[1411] Example 1: Detecting and responding to anomalies during absence

[1412] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[1413] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[1414] 2. The server inputs the collected data into the generative AI model for real-time analysis. The generative AI model returns a high score for this behavior, identifying it as abnormal.

[1415] 3. If the server detects an abnormality, it immediately sends an SMS to the user notifying them that "The front door has been opened in an unnatural way."

[1416] 4. The user receives the notification and sends a request to the server to "check the status of the front door."

[1417] 5. Based on the latest data, the server notifies the user that the door is open and sends camera footage to report the specific situation.

[1418] 6. The user sends the command "Lock the front door."

[1419] 7. The server remotely locks the front door and reports "The front door is locked."

[1420] Example 2: Daily fail-safe checks

[1421] Consider a case where a user checks the security status of their home before going to bed.

[1422] 1. The user launches the app and sends a request to "check the security status of the entire house."

[1423] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative AI model.

[1424] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the user that "all doors and windows are closed and no abnormal sounds have been detected."

[1425] In this way, the present invention functions as an advanced security system that can safely protect the home even when the user is away or asleep.

[1426] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1427] Step 1:

[1428] The server collects environmental data from multiple devices (sensors and surveillance cameras). The input is data from various sensors (e.g., temperature, door opening / closing, sound, video), which is integrated into a database within the server. The output is the integrated environmental data. Since this data is collected in real time, the server always has the latest information.

[1429] Step 2:

[1430] The server inputs the collected environmental data into the generative AI model. The input is environmental data, and each data point is analyzed by the generative AI model. The AI ​​model is trained to identify abnormal and intrusive behavior. The output is an anomaly score assigned to each data point and the analysis results. Specifically, the AI ​​model detects sudden changes in temperature, unnatural opening and closing movements, and loud noises, and scores them as abnormal.

[1431] Step 3:

[1432] The server evaluates the analysis results returned by the generative AI model. The inputs are the anomaly score and the analysis results, and based on these, it determines whether an anomaly has been detected. The output is the specific details of the detected anomaly. For example, it can include specific information such as "the front door was opened unnaturally late at night."

[1433] Step 4:

[1434] If an abnormality is detected, the server will promptly send a notification to the user. The input is the detected abnormality, and the output is the notification to the user (e.g., SMS or dedicated app alert). The notification will include the specific details of the abnormality and a suggestion of immediate action to be taken (e.g., "The front door is open. Please check it.").

[1435] Step 5:

[1436] The user receives a notification and sends a specific request to the server. The input is the user's request (e.g., "Check the status of the front door"), and the output is that the request reaches the server. The specific action is that the user enters the request through the application.

[1437] Step 6:

[1438] The server recollects and analyzes the latest sensor and camera data based on the user's request. The input is the latest sensor and camera data, and the output is a response to the user. For example, specific information such as "The front door is open and a suspicious person was captured on camera" is sent to the user.

[1439] Step 7:

[1440] The server remotely implements security measures based on user instructions. The input is the user's instruction (e.g., "Lock the front door"), and the output is a report of the implemented security measures (e.g., "The front door is locked"). Specifically, the server sends a signal to the door lock control system to lock the door.

[1441] (Application example 1)

[1442] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1443] Modern home security systems have limited capabilities for detecting intrusions and other anomalies, and lack real-time monitoring and automated user response. Even if users detect an intrusion, it is difficult for them to immediately take appropriate countermeasures. This increases the burden on users and increases security risks. This can lead to delayed responses, especially when users are in remote locations or late at night, when immediate action is required.

[1444] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1445] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for checking the situation and providing advice on countermeasures based on the owner's request, means for remotely controlling the security status, means for executing instructions from the user in response to an abnormality that has occurred, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to highly automate the entire process from anomaly detection to countermeasure implementation, significantly reducing security risks while reducing the burden on the user.

[1446] "Surveillance devices" refers to various sensors and surveillance cameras used to collect environmental data.

[1447] "Environmental data" refers to data that indicates the physical and operational conditions within a space, including the open / close status of doors, temperature, and sound.

[1448] A "generative artificial intelligence model" is a system that uses machine learning and deep learning to analyze environmental data in real time and includes algorithms for detecting abnormal behavior and intrusions.

[1449] "Real-time analysis" is the process of instantly processing environmental data collected from monitoring devices and detecting abnormalities.

[1450] "Abnormal behavior" is anything that deviates from normal behavior and includes behavior that indicates intrusion or fraud.

[1451] "Notification" refers to the means of transmitting alerts and information to the owner when an abnormality is detected.

[1452] A "request" refers to a request from the owner to the server to check the status or to give instructions on how to deal with the problem.

[1453] "Status check" refers to the operation that the owner performs to check the current security status when an abnormality is detected.

[1454] "Countermeasure advice" refers to the specific countermeasure suggestions provided to the owner by the generative artificial intelligence model when an abnormality is detected.

[1455] "Remote operation means" refers to an interface that allows the owner to operate the various functions of the security system from a remote location.

[1456] "Feedback" is data generated from the dialogue and results of the system's operations that is used to improve the performance of generative artificial intelligence models.

[1457] "Means for improving artificial intelligence models" refers to the process of adjusting algorithms and parameters based on collected data and feedback information to improve the performance of the model.

[1458] MODE FOR CARRYING OUT THE INVENTION

[1459] Overall system configuration

[1460] The present invention is a system that consists of three main components: a monitoring device, a server, and a user terminal.

[1461] Surveillance Devices

[1462] The monitoring devices consist of various sensors (door sensors, window sensors, temperature sensors, sound sensors, etc.) and surveillance cameras for collecting environmental data. These devices are installed to detect abnormal behavior and intrusions within the home, and transmit data to a server in real time.

[1463] server

[1464] The server is the central control device of this system. It integrates environmental data collected from monitoring devices and performs real-time analysis using generative AI models. If abnormal behavior or intrusion is detected, the server immediately notifies the user. It also provides specific situation confirmation and advice on countermeasures based on the user's request.

[1465] The server includes the following features:

[1466] 1. Data collection function: Collects environmental data from monitoring devices.

[1467] 2. Real-time analytics: Analyze data using generative AI models to detect anomalies.

[1468] 3. Notification function: Sends a notification to the user when an abnormality is detected.

[1469] 4. Remote control function: The security status of the room can be controlled remotely according to the user's request.

[1470] 5. Feedback generation function: Record all data and conversations, generate feedback and improve the generative AI model.

[1471] User terminal

[1472] User terminals are devices such as smartphones, tablets, and smart glasses that allow users to interact with the system and check and operate the security status. Users can receive notifications from the server through their terminals and send status checks and countermeasure requests.

[1473] Program processing

[1474] Real-time data collection

[1475] The server consolidates the environmental data collected from each monitoring device, which is sent in formats such as JSON and includes information such as the status of doors and windows, temperature, and sound.

[1476] Real-time analytics

[1477] The data is fed into a generative AI model (e.g., RealTimeAnalyzer) and analyzed in real time. The generative AI model includes algorithms for detecting abnormal behavior and intrusions.

[1478] Anomaly detection and notification

[1479] If the server detects an anomaly based on the analysis results, it will immediately send a notification to the user using a notification service (e.g., Twilio API, Firebase Cloud Messaging). The notification will include details of the anomaly and the measures the user should take.

[1480] Request handling and remote operations

[1481] When a request from a user (e.g., "lock the door") is received, the server calls an API for remote operation and performs the corresponding operation.

[1482] Feedback Generation

[1483] The server records all data and conversations and generates feedback to improve the performance of the generative AI model, which is then used to adjust parameters and algorithms to improve anomaly detection capabilities next time.

[1484] Examples of specific examples and prompts

[1485] Example 1:

[1486] Consider a case where the rear window of a house suddenly opens in the middle of the night while the user is away on a trip. A suspicious person is clearly visible in the frame of a surveillance camera. The generative AI model detects this abnormal behavior with a high score, and the server immediately sends a notification to the user. The user then sends a request via their smartphone to "lock all doors and windows," and the server remotely locks all doors and windows.

[1487] Example 2:

[1488] If the server detects an abnormality such as the front door being opened and closed multiple times while the user is out, it will notify the user that "The front door has been opened and closed multiple times" and immediately lock the door based on the user's request to "lock the front door."

[1489] Example prompt:

[1490] "Analyzes abnormal behavior detected by sensors and cameras during specific times. Abnormal behavior detection: Determines whether a door has been opened or closed, whether a person is visible in the room, or whether an abnormal sound has been recorded."

[1491] This allows for the construction of an effective embodiment of the invention, and the system efficiently automates a series of processes from anomaly detection to countermeasure implementation, thereby reducing the burden on the user.

[1492] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1493] Step 1:

[1494] Data collection

[1495] The server collects environmental data from each monitoring device (door sensor, window sensor, temperature sensor, sound sensor, surveillance camera).

[1496] Input: Real-time data sent from each monitoring device (e.g., door opening, temperature change, sound generation, etc.)

[1497] Output: The integrated results of the collected environmental data (e.g., data in JSON format)

[1498] Specific Operation: The server pulls data from the monitoring devices at regular intervals and generates a consolidated data set.

[1499] Step 2:

[1500] Preparing for data analysis

[1501] The server performs preprocessing to analyze the collected environmental data, extracting only the data necessary for anomaly detection.

[1502] Input: Collected environmental data

[1503] Output: Preprocessed dataset

[1504] Specific operations: filtering unnecessary data, shaping data necessary for anomaly detection (e.g., data normalization)

[1505] Step 3:

[1506] Real-time analytics

[1507] The server passes the preprocessed dataset to a generative AI model to detect anomalous behavior and intrusions.

[1508] Input: Preprocessed dataset

[1509] Output: Anomaly detection result (e.g., anomaly score, if the score is high, an anomaly is detected)

[1510] How it works: The generative AI model analyzes the dataset and generates an anomaly score. High scores indicate anomalous behavior.

[1511] Step 4:

[1512] Sending abnormality notifications

[1513] If an abnormality is detected, the server sends a notification to the user device, which includes details of the abnormality and advice on how to deal with it.

[1514] Input: Anomaly detection result (high score)

[1515] Output: A message to inform the user

[1516] Specific operation: Notifications are sent to the user's smartphone or smart glasses using the Twilio API or Firebase Cloud Messaging.

[1517] Step 5:

[1518] Processing user requests

[1519] After receiving the notification, the user sends a request to the server to check the situation and take action.

[1520] Input: User request (e.g. "Lock the door" or "Check the camera footage")

[1521] Output: The result of sending the user request to the server

[1522] Specific operation: A request is sent from the user terminal to the server. The request content is analyzed and the next action is determined.

[1523] Step 6:

[1524] Performing remote operations

[1525] The server performs remote operations based on the user's requests.

[1526] Input: User request (e.g., "Lock the door")

[1527] Output: Result of remote operation (e.g. door locked)

[1528] Specific operation: The server calls the corresponding API to change the state of the device (e.g., locking the door using the smart lock's API).

[1529] Step 7:

[1530] Feedback Generation

[1531] The server records all data and interactions and generates feedback to improve the generative AI model.

[1532] Input: All collected data and conversations

[1533] Output: Feedback data for improving the generative AI model

[1534] Specific operation: The server analyzes the operation history and dialogue content to generate feedback as learning data for the generative AI model.

[1535] Each step ensures smooth operation of the entire system and automates the entire process from anomaly detection to countermeasure implementation, thereby streamlining users' security management.

[1536] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1537] MODE FOR CARRYING OUT THE INVENTION

[1538] overview

[1539] This invention is a home security system that combines a generative AI model with advanced understanding capabilities with an emotion engine, and is particularly designed to combat burglaries and provide countermeasures tailored to the owner's emotions. The system analyzes environmental data collected from monitoring devices in real time, detects abnormal behavior or intrusions, and notifies the owner. The emotion engine also analyzes the owner's emotional state and provides personalized notifications and advice.

[1540] System Configuration

[1541] The server is the core of the system, collecting data from various sensors and surveillance cameras and performing real-time analysis using a generative AI model and emotion engine. If an abnormality is detected, it promptly notifies the owner and provides specific countermeasures in an interactive format upon request. It also adjusts the content of the notification and suggested countermeasures according to the owner's emotional state.

[1542] The monitoring devices consist of sensors (door and window open / close sensors, temperature sensors, sound sensors, etc.) and surveillance cameras, each of which has the role of transmitting environmental data to a server.

[1543] The emotion engine analyzes emotions from the owner's voice, text, facial expressions, etc. to understand the owner's psychological state, enabling it to respond appropriately according to their emotional state.

[1544] The user, or owner, receives a notification when an abnormality is detected and can request specific measures from the server.

[1545] Program processing

[1546] The server collects environmental data from the monitoring devices and analyzes it in real time. The generative AI model uses the data to detect abnormal behavior or intrusions and returns the results to the server. The server evaluates the analysis results and promptly sends a notification to the owner if an abnormality is detected. This notification includes details of the abnormality and suggestions for immediate countermeasures. The server also uses an emotion engine to analyze the owner's emotional state and adjust appropriate responses.

[1547] The owner receives a notification and sends a specific request to the server. For example, in response to a request such as "Please check if the door is open," the server determines whether the living room door is open based on the latest sensor and camera data and responds accordingly. In response to a command such as "Please lock the doors," the server remotely locks all doors and reports the results to the owner. The emotion engine then provides appropriate advice and additional information to reduce the owner's anxiety and tension.

[1548] Specific examples

[1549] Example 1: Absence anomaly detection and emotional response

[1550] Consider a case where the front door is opened strangely late at night, just after the user has left the house.

[1551] 1. The server detects abnormal opening and closing behavior from the door sensor, and at the same time, the surveillance camera captures video of the entrance.

[1552] 2. The server inputs the collected data into a generative AI model for real-time analysis. The AI ​​model returns a high score for the abnormal behavior.

[1553] 3. If the server detects an abnormality, it will immediately send an SMS to the owner stating that "the front door has been opened in an unnatural way."

[1554] 4. The emotion engine analyzes the owner's emotional state and determines that the owner is nervous. Based on this, a message encouraging calmness is added.

[1555] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[1556] 6. Based on the latest data, the server sends the owner the fact that the door is open and the camera footage, reporting the specific situation. The emotion engine explains the situation in an easy-to-understand manner to reassure the owner.

[1557] 7. The user sends the command "Lock the front door."

[1558] 8. The server remotely locks the front door and reports, "The front door is locked." At the same time, the emotion engine sends additional information and suggested actions to ease the owner's tension.

[1559] Example 2: Daily fail-safe checks and emotional responses

[1560] Consider a case where a user checks the security status of their home before going to bed.

[1561] 1. The user launches the app and sends a request to "check the security status of the entire house."

[1562] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[1563] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[1564] 4. The emotion engine analyzes the owner's emotional state and adds messages to provide comfort.

[1565] In this way, by combining an emotion engine, the present invention provides notifications and countermeasures that are optimized according to the owner's psychological state, realizing more advanced and reassuring home security.

[1566] The processing flow will be explained below.

[1567] Step 1:

[1568] The server collects environmental data from monitoring devices (sensors and cameras). The sensors measure data such as door and window opening and closing status, temperature, and sound levels, while the cameras capture real-time video. The server periodically sends data collection requests to these devices and collects the resulting data.

[1569] Step 2:

[1570] The server inputs the collected data into a generative artificial intelligence model, which preprocesses the collected data and prepares it as an input dataset for anomaly detection. Data preprocessing includes noise removal, data normalization, and video frame analysis.

[1571] Step 3:

[1572] The server inputs the preprocessed data into the generative AI model for real-time analysis. The generative AI model calculates an anomaly score and returns the result to the server. This anomaly score detects patterns that differ from normal household activity, and if the score is high, it is determined to be abnormal behavior or an intrusion.

[1573] Step 4:

[1574] The server evaluates the analysis results from the generative AI model and determines that an anomaly has occurred if the anomaly score exceeds a certain threshold. Based on this information, the server detects abnormal behavior or possible intrusion and proceeds to the next step.

[1575] Step 5:

[1576] If an abnormality is detected, the server immediately sends a notification to the owner. The notification includes details of the location and situation where the abnormality was detected, as well as suggestions for immediate measures to be taken. Notification methods include SMS, email, and a dedicated app.

[1577] Step 6:

[1578] The server uses the owner's emotion engine to analyze the owner's emotional state. It analyzes the owner's voice and text to determine whether the owner is feeling anxious or nervous. It also analyzes the owner's facial expressions using a surveillance camera to complement the owner's emotional state.

[1579] Step 7:

[1580] Based on the owner's emotional state as analyzed by the emotion engine, the server tailors the content of notifications and advice. For example, if the owner is nervous, it will provide a message encouraging them to stay calm or provide additional reassurance.

[1581] Step 8:

[1582] After receiving the notification from the server, the user can send a request to the server as needed, for example, a specific request such as "check if the door is open."

[1583] Step 9:

[1584] The server retrieves the latest data from each sensor and camera in response to the owner's request and responds to the owner based on that information. For example, it may check whether the living room door is open or closed and report the results to the owner. The emotion engine also provides appropriate explanations and advice that take the owner's emotions into consideration.

[1585] Step 10:

[1586] Based on the report from the server, the user can request additional measures, for example, by sending an instruction such as "Please lock the door."

[1587] Step 11:

[1588] The server remotely executes specific measures based on the owner's instructions, such as remotely locking all doors and notifying the owner of the results. The emotion engine then sends additional information and suggested measures to ease the owner's tension.

[1589] Step 12:

[1590] The server records all data and interactions in detail and feeds them back into the generative AI model to improve the system's accuracy. This feedback process allows the system to continuously improve itself and improve future anomaly detection accuracy.

[1591] Example 2

[1592] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1593] Conventional home security systems focus on collecting environmental data and detecting abnormal behavior, but are indifferent to the owner's psychological state. As a result, when an abnormality is detected, the owner often becomes overly tense or anxious, which can delay appropriate response. There is also a need for a system that can provide appropriate countermeasures based on the owner's emotional state.

[1594] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1595] In this invention, the server includes means for collecting environmental data from the monitoring device, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected anomaly, means for analyzing the owner's emotional state and providing countermeasures according to the emotional state, means for checking the situation and providing advice on countermeasures based on the owner's request, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This makes it possible to provide an optimal response according to the owner's psychological state, thereby reducing anxiety and tension.

[1596] "Monitoring devices" are devices used to collect environmental data, such as sensors and surveillance cameras.

[1597] "Environmental data" refers to data that indicates the physical conditions within the home, and includes temperature, sound, whether doors are open or closed, and video.

[1598] "Generative AI models" refer to models that use machine learning and neural networks to detect anomalous behavior and intrusions based on collected data.

[1599] "Real-time analysis" refers to the process of instantly analyzing collected data and quickly determining abnormalities and instructions for the next step.

[1600] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is deemed to pose a potential security risk.

[1601] "Breaking" refers to the act of attempting to gain unauthorized entry into a home.

[1602] "Owner" refers to the person who manages and uses the security system in a home.

[1603] "Notification" means a message sent by the System to the Owner, including a warning or information.

[1604] An "emotion engine" refers to an algorithm or component that analyzes the owner's emotional state and provides corresponding responses.

[1605] "Request" refers to a specific instruction or request for information made by the Owner to the System.

[1606] "Dialogue content" refers to a record of the information exchanged, instructions, and responses between the owner and the system.

[1607] "Feedback" refers to information used to improve system performance and review appropriate countermeasures based on past dialogue and data.

[1608] MODE FOR CARRYING OUT THE INVENTION

[1609] This invention is a system that enhances home security using environmental data collected from monitoring devices. Specifically, the operation of the system, which is centered around a server, terminals, and users, will be described below.

[1610] The server is the heart of the system and works by:

[1611] 1. Collecting data from monitoring devices

[1612] The server collects data from monitoring devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras. These devices transmit data to the server using communication protocols such as Bluetooth, Wi-Fi, and Zigbee. The environmental data is based on specific data patterns and includes sensor status information and video data.

[1613] 2. Real-time analysis of data

[1614] The server inputs the collected data into a generative AI model for real-time analysis. The generative AI model uses the data to detect abnormal behavior and intrusions. The AI ​​model may be implemented using machine learning libraries such as OpenCV or TensorFlow.

[1615] 3. Anomaly detection and notification

[1616] The server evaluates anomalies based on the analysis results from the AI ​​model. If an anomaly is detected, it sends a notification to the owner via a mobile app, SMS, email, etc. The notification includes a description of the anomaly and a suggestion for immediate action.

[1617] 4. Owner's emotional state analysis

[1618] The server uses an emotion engine to analyze the owner's emotional state. It extracts emotions from voice, text data, and facial expressions, and provides appropriate advice and countermeasures based on the results. The emotion engine incorporates NLP (natural language processing) and voice recognition technology.

[1619] 5. Processing Owner Requests

[1620] When an abnormality is detected, the user can send a request to the server. For example, a request might be, "Please check if the living room door is open." The server will respond with information based on the latest sensor and camera data. In response to a command such as, "Please lock the doors," the server will remotely lock all doors and report the results to their owners.

[1621] 6. Recording data and dialogue and generating feedback

[1622] The server records all data and interactions, generating feedback to improve the generative AI model, allowing the system to respond more intelligently to the owner's individual behavior, providing enhanced security and peace of mind.

[1623] A specific scenario would be:

[1624] Example 1: Absence anomaly detection and emotional response

[1625] The following example shows a case where the front door is opened unexpectedly late at night, immediately after the user has left the house.

[1626] 1. The server detects abnormal opening and closing behavior from the door sensor, and the surveillance camera captures footage of the entrance.

[1627] 2. The server inputs the collected data into a generative AI model for real-time analysis, which then scores the behavior as an anomaly.

[1628] 3. If the server detects an abnormality, it immediately sends a notification to the owner saying, "The front door has been opened in an unnatural way."

[1629] 4. The emotion engine analyzes the owner's emotional state, and if it determines that the owner is tense, it sends an additional message encouraging the owner to stay calm.

[1630] 5. The user receives the notification and sends a request to the server to "check the status of the front door."

[1631] 6. Based on the latest data, the server sends the owner information such as whether the door is open and camera footage, and reports the specific situation. The emotion engine sends additional information to encourage calm.

[1632] 7. The user sends the command "Lock the front door."

[1633] 8. The server remotely locks the front door and reports, "The front door is locked." The emotion engine also sends additional information and suggested actions to ease the owner's tension.

[1634] Example 2: Daily fail-safe checks and emotional responses

[1635] The following describes a case where a user checks the security status of his or her home before going to bed.

[1636] 1. The user launches the app and sends a request to "check the security status of the entire house."

[1637] 2. The server collects the latest data from all sensors and cameras and analyzes it using a generative artificial intelligence model.

[1638] 3. The server creates a status report based on the status of each door and window, whether or not there are any abnormal sounds, and the camera footage, and notifies the owner that "all doors and windows are closed and no abnormal sounds have been detected."

[1639] 4. The emotion engine analyzes the owner's emotional state and sends additional messages to provide reassurance.

[1640] In this way, the present invention is a system that combines an emotion engine to provide optimal notifications and responses according to the owner's psychological state, achieving both security and a sense of security.

[1641] Examples of prompt statements

[1642] 1. "What is the security situation at your home?"

[1643] 2. "Make sure the front door is open."

[1644] 3. "What should you do if signs of intrusion are detected?"

[1645] 4. "I'm feeling anxious, so please set an alarm."

[1646] 5. "Check the living room camera feed."

[1647] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1648] Program processing steps

[1649] Step 1:

[1650] A means by which the server collects environmental data from monitoring devices

[1651] Input: Data sent from surveillance devices such as door and window sensors, temperature sensors, sound sensors, and surveillance cameras.

[1652] Output: Sensor data and video data stored in the server.

[1653] Specific operation: The server polls data from each monitoring device every 10 seconds and collects data using protocols such as Bluetooth, Wi-Fi, and Zigbee.

[1654] Data processing: Environmental data is classified by device and stored in chronological order. For example, if the front door sensor sends data indicating that the door is open, that information is stored in the database.

[1655] Step 2:

[1656] The server inputs the collected data into a generative artificial intelligence model and analyzes it in real time.

[1657] Input: Accumulated environmental data.

[1658] Output: Anomaly scores as the analysis result.

[1659] Specific operation: The collected data is input into a generative artificial intelligence model (e.g., a model using TensorFlow or OpenCV) and real-time analysis is performed.

[1660] Data processing: Preprocessing data and converting it into a format suitable for anomaly detection. For example, analyzing door opening and closing behavior at night and scoring whether the behavior deviates from normal behavioral patterns.

[1661] Step 3:

[1662] A method for the server to detect abnormal behavior or intrusions based on analysis results

[1663] Input: Anomaly scores obtained from a generative AI model.

[1664] Output: Whether or not there is an abnormality and its content.

[1665] Specific operation: If the anomaly score exceeds a set threshold, it is determined to be an anomaly. For example, an anomaly score of 80 or more is considered an anomaly, and it is determined that there is a high possibility of intrusion.

[1666] Data processing: Evaluate the analysis results and determine the type and urgency of the anomaly. If no anomaly is detected, return to the next cycle.

[1667] Step 4:

[1668] A means for the server to notify the owner of detected anomalies

[1669] Input: Whether or not an anomaly was detected and its details.

[1670] Output: A notification message to the owner.

[1671] Specific operation: If an abnormality is detected, the owner will be notified via SMS, email, or a dedicated app.

[1672] Data processing: Generate a notification message. For example, generate and send a message such as "The front door was opened unexpectedly. All doors have been locked."

[1673] Step 5:

[1674] A means for the server to analyze the owner's emotional state and provide countermeasures according to that emotional state

[1675] Input: Owner's voice, text messages, and facial expression data.

[1676] Output: Emotional state and coping strategies.

[1677] Specific operation: The emotion engine analyzes emotions from the owner's voice and text, extracts emotional data, and generates countermeasures.

[1678] Data processing: Using NLP and speech recognition technology, we analyze emotions and generate advice and messages based on the emotional state. For example, we provide messages encouraging calm, such as "Please stay calm. Would you like to contact the police immediately?"

[1679] Step 6:

[1680] The means by which a user sends a request to a server and the server responds

[1681] Input: A request from the owner, for example, "Make sure the living room door is open."

[1682] Output: The response to the request.

[1683] Specific operation: Upon receiving the owner's request, the server analyzes the information based on the latest sensor and camera data and responds to the owner.

[1684] Data processing: Parse the request, collect relevant data, and generate a specific response message for the owner, for example, "The living room door is open."

[1685] Step 7:

[1686] A means for the server to record all data and interactions and generate feedback to improve the AI ​​model

[1687] Input: Sensor data, dialogue content, system response results.

[1688] Output: Feedback data, new learning model.

[1689] Specific operation: All sensor data and conversations are logged, and feedback data is generated to improve the performance of AI models.

[1690] Data processing: Analyze all collected data and create a feedback loop to retrain the model. For example, analyze reaction times when an anomaly is detected or changes in the owner's emotions to identify areas for improvement in the model or system.

[1691] (Application example 2)

[1692] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1693] In recent years, there has been a demand for security systems to improve safety within homes. However, existing systems are limited to detecting abnormal behavior and intrusions, and are unable to respond flexibly to the owner's psychological state. Furthermore, they lack the ability to fully implement real-time environmental monitoring and remote control, making it impossible to completely eliminate the owner's sense of anxiety. Given this background, there is a need for a security system that not only detects anomalies but also provides optimal countermeasures based on emotional analysis, thereby increasing the sense of security.

[1694] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1695] In this invention, the server includes means for collecting environmental data from the monitoring devices, means for inputting the collected data into a generative AI model and analyzing anomalies in real time, means for detecting abnormal behavior or intrusion based on the analysis results, means for notifying the owner of the detected abnormality, means for checking the situation and providing advice on countermeasures based on the owner's request, means for analyzing the owner's emotional state using an emotion engine and optimizing the notification content and countermeasures, means for monitoring the home environmental conditions in real time and performing remote control as necessary, and means for recording all data and dialogue content, generating feedback, and improving the AI ​​model. This not only enables anomaly detection but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[1696] A "surveillance device" is a piece of equipment used to collect environmental data, including sensors, cameras, and other devices.

[1697] "Environmental data" refers to data collected from monitoring devices that indicates conditions inside and outside the home, and includes, for example, information on temperature, sound, and movement.

[1698] A "generative artificial intelligence model" is a model that uses machine learning and deep learning techniques to analyze collected environmental data and detect anomalous behavior and intrusions.

[1699] "Means for analyzing in real time" refers to a processing method for instantly processing collected data and outputting the results.

[1700] "Abnormal behavior" refers to actions or movements that deviate from the normal range of operation, such as intrusions or acts of vandalism in a surveillance system.

[1701] "Means of notification" refers to a method for notifying the owner of an abnormal behavior or intrusion when such behavior or intrusion is detected.

[1702] An "owner request" is a request for confirmation or measures sent by the owner to the system.

[1703] "Confirming the situation" and "advice on countermeasures" refer to operations to check the status of the current monitoring environment and to provide advice on countermeasures for abnormal situations.

[1704] The "emotion engine" is a system that analyzes the owner's psychological state and adjusts notification content and countermeasures based on that information.

[1705] "Remote control" is a function that allows the owner to send commands to the monitoring system and operate it from a remote location.

[1706] "Data and interaction recording" means that the system stores all data collected and interactions with the owner.

[1707] The "means for generating feedback" is a method for generating information to improve the performance of the generative artificial intelligence model based on the recorded data.

[1708] The program of the system that realizes this application example is configured in the following way.

[1709] First, the server collects environmental data from monitoring devices, such as temperature sensors, sound sensors, motion sensors, and surveillance cameras. The environmental data collected by these devices is sent to the server.

[1710] The server then inputs the collected data into a generative AI model that analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning techniques, and is capable of detecting anomalous behavior and intrusions with high accuracy.

[1711] If data analysis detects any abnormal behavior or intrusion, the server will send a notification to the owner's smartphone, including the specific location and circumstances of the anomaly.

[1712] Additionally, the emotion engine analyzes the owner's voice and text input to understand their emotional state. For example, if voice analysis determines that the owner is feeling anxious, an additional message to help them stay calm will be displayed.

[1713] According to the owner's request, the server will check the situation and provide advice on countermeasures. For example, if the owner requests, "Check the status of the front door," the server will check whether the door is open or closed based on the latest sensor data and camera footage and report the status to the owner. Also, if the owner instructs, "Lock the door," the server will remotely lock the door and notify the owner of the result.

[1714] All this data and conversation content is recorded on the server and used as feedback to improve the generative AI model.

[1715] Hardware and Software

[1716] Hardware: Smartphone, home sensors (door open / close sensor, temperature sensor, sound sensor, surveillance camera)

[1717] Software: EmotionEngine (emotion engine), AIModel (generative artificial intelligence model), communication library between server and client (e.g., requests)

[1718] Specific examples

[1719] Nighttime anomaly detection and countermeasures

[1720] 1. Late at night, the sensor on the front door detects abnormal opening and closing behavior and sends the data to the server.

[1721] 2. The server analyzes using a generative artificial intelligence model and detects anomalies with a high probability.

[1722] 3. The owner will receive a notification that their front door has been opened in an unnatural manner.

[1723] 4. The emotion engine analyzes the owner's psychological state and displays a message encouraging them to stay calm (e.g., "Please stay calm").

[1724] 5. When the owner requests, "Check the status of the front door," the server reports the situation based on camera footage and the latest sensor data.

[1725] 6. When the owner commands, "Lock the front door," the server remotely locks the door and notifies the owner, "The front door has been locked."

[1726] Prompt Sentence Examples

[1727] The user types "Check the status of the front door" into their smartphone.

[1728] The server analyzes the situation based on the latest camera footage and sensor data.

[1729] The app notifies you, "The front door is safely closed. No particular abnormalities are observed."

[1730] In this way, this invention realizes a system that not only detects abnormalities but also provides appropriate countermeasures according to the owner's emotions, and enables real-time environmental monitoring and remote control, thereby significantly improving the owner's safety and sense of security.

[1731] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1732] Step 1:

[1733] The server collects environmental data from the monitoring devices. The collected data includes temperature, sound, motion information, and camera footage. This data is sent to the server in real time by the monitoring devices. The server's input is the environmental data sent from the monitoring devices, and its output is the storage of the collected data and preparation for analysis.

[1734] Step 2:

[1735] The server inputs the collected environmental data into a generative AI model and analyzes anomalies in real time. The generative AI model is built using machine learning and deep learning technologies and detects abnormal behavior and intrusions with high accuracy. The input at this stage is environmental data, and the output is the analysis results. Specifically, an anomaly score is calculated and the presence or absence of anomalies is determined based on that score.

[1736] Step 3:

[1737] The server detects abnormal behavior or intrusions based on the analysis results. If the anomaly score exceeds a certain threshold, the system determines this to be an anomaly. The input to this step is the analysis result of the generative AI model, and the output is the anomaly detection result. Specifically, it detects anomalies such as "unnatural door opening and closing" or "suspicious noises."

[1738] Step 4:

[1739] The server notifies the owner of the detected abnormality. The notification includes the specific location and circumstances of the abnormality. In this step, the input is the abnormality detection result, and the output is a notification message to the owner. Specifically, a notification such as "The front door was opened in an unnatural way" is sent to the smartphone.

[1740] Step 5:

[1741] The emotion engine analyzes the owner's voice and text input to understand the owner's emotional state. The server adjusts the content of notifications and countermeasure messages based on the results of the emotion analysis. The input for this step is the owner's voice and text data, and the output is the adjusted message. Specifically, if the owner is nervous, "advice to stay calm" is added.

[1742] Step 6:

[1743] Based on the owner's request, the server checks the situation and provides advice on countermeasures. The input for this step is the owner's request, and the output is a situation report and advice on countermeasures. Specifically, in response to a request such as "Check the status of the front door," the server reports the situation based on the latest sensor data and camera footage.

[1744] Step 7:

[1745] The server executes the owner's remote control instructions. For example, if the owner commands "lock the doors," the server remotely locks the doors and notifies the owner of the result. The input of this step is the remote control instruction, and the output is the execution result. Specifically, a confirmation message such as "All doors have been locked" is sent to the owner.

[1746] Step 8:

[1747] The server records all data and dialogue, generates feedback, and improves the AI ​​model. The input for this step is sensor data and dialogue, and the output is an improved AI model. Specifically, the model is retrained based on the analysis results and owner feedback to improve the accuracy of the entire system.

[1748] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1749] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1750] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1751] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1752] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1753] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1754] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1755] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1756] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1757] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values,...

Claims

1. means for collecting environmental data from a monitoring device; A means for inputting the collected data into a generative artificial intelligence model and analyzing anomalies in real time; A means for detecting abnormal behavior or intrusion based on the analysis results; means for notifying the owner of the detected anomaly; A means of checking the situation and providing advice on countermeasures based on the owner's request, and A means to record all data and interactions and generate feedback to improve the AI ​​model; and A system including:

2. The system of claim 1 , wherein the monitoring devices include a surveillance camera and various sensors.

3. The system of claim 1 , wherein the generative artificial intelligence model detects anomalous behavior based on an anomaly score.

Images