System

Abstract

A system is provided.SOLUTION: The system includes video analysis means for detecting abnormal behavior using a generative machine learning model, alert generation and transmission means for generating and transmitting an alert based on the detected abnormal behavior, and notification means for notifying the police or a user of information related to the occurrence of an abnormality.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the number of crimes and accidents has been increasing, placing a greater burden on police officers. Conventional security camera systems only have a recording function, making it difficult to detect abnormalities in real time and report them immediately. As a result, there are many cases where response to abnormalities is delayed, limiting the effectiveness of such systems in terms of local safety. Furthermore, the shortage of police officers in regional cities is becoming a serious problem. [Means for solving the problem]

[0005] The present invention provides a system that uses a generative machine learning model to detect abnormal behavior and generates and transmits an alert based on the detected abnormal behavior. This system includes a reporting means that immediately reports relevant information to the police and users when an abnormality occurs. Furthermore, the system includes a data analysis and learning means that updates the generative machine learning model, and a learning model transmission means that transmits the updated learning model to each terminal, thereby achieving continuous performance improvement. Furthermore, the system includes a video review means that sends a notification of abnormal behavior to the user's smartphone in real time and allows the user to review the video in real time, enabling immediate response. In this way, a system is realized that can reduce the workload of police officers and improve local safety.

[0006] A "generative machine learning model" is a machine learning algorithm designed to detect new patterns and anomalies based on input data.

[0007] The "video analysis means" is a key component of the system that analyzes video data acquired from cameras in real time and detects abnormal behavior.

[0008] The "alert generation and transmission means" is a mechanism for generating an alert in response to abnormal behavior detected by the video analysis means and transmitting the alert to an appropriate recipient (police, user, etc.).

[0009] The "reporting means" is a means for receiving the alert generated by the alert generating and transmitting means and sending a report including detailed information about the abnormality to the police or other relevant parties.

[0010] "Data analysis and learning means" refers to the processes and tools used by the system to analyze collected data and update machine learning models based on the results.

[0011] The "learning model transmission means" is a mechanism for transmitting an updated machine learning model to each terminal in the system to improve anomaly detection performance on the terminal side.

[0012] The "notification means" is a function for sending notifications to relevant users' smartphones or other devices in real time when abnormal behavior is detected.

[0013] "Video review means" refers to an interface and technology that allows a user to review video of abnormal behavior in real time. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention relates to a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to enable rapid and appropriate responses when an abnormality occurs.

[0036] 1. Initial Setup

[0037] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0038] 2. Data collection and learning

[0039] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0040] 3. Anomaly Detection

[0041] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0042] 4. Alerts and Notifications

[0043] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0044] 5. Real-time notifications and actions

[0045] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[0046] Specific examples

[0047] Example 1: Detecting a thief

[0048] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[0049] Example 2: Traffic accident detection

[0050] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0051] In this way, this system uses generative machine learning models to detect crimes and accidents in real time and enable rapid response, reducing the workload of police officers while improving the safety of local communities.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] The server sends an initial configuration file to each security camera (device), which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[0055] Step 2:

[0056] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[0057] Step 3:

[0058] The device collects video data through its camera 24 hours a day and stores it in its internal memory.

[0059] Step 4:

[0060] The terminal transmits the collected video data to the server at regular intervals.

[0061] Step 5:

[0062] The server receives the transmitted video data and performs an initial analysis to identify patterns of crimes and accidents.

[0063] Step 6:

[0064] The server updates the generative machine learning model based on the analyzed data.

[0065] Step 7:

[0066] The server sends the updated generative machine learning model to each terminal.

[0067] Step 8:

[0068] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[0069] Step 9:

[0070] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[0071] Step 10:

[0072] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[0073] Step 11:

[0074] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0075] Step 12:

[0076] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[0077] Step 13:

[0078] If the server determines that a report is necessary, it will make the report based on the alert information. Reports are made via SMS, app notification, email, etc.

[0079] Step 14:

[0080] Users receive alert notifications via an app on their smartphones.

[0081] Step 15:

[0082] The user opens the app to see the notification and view the footage in real time.

[0083] Step 16:

[0084] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[0085] In this way, by showing the specific operations and processing flow performed at each step, specific embodiments of the invention can be clearly understood.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] Conventional monitoring systems have difficulty detecting anomalies in real time and responding quickly, which means they are unable to adequately prevent crimes and accidents or respond quickly. Furthermore, the accuracy of anomaly detection is low, resulting in frequent false positives and oversights. Therefore, there is a need to improve the efficiency and accuracy of monitoring systems.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes means for transmitting an initial setting file, means for collecting video data, means for transmitting the collected video data to the server, means for analyzing the video data using a generative machine learning model to detect abnormal behavior, means for generating and transmitting an alert when abnormal behavior is detected, means for reporting related information to the police and users when an abnormality occurs, and means for transmitting notifications of abnormal behavior in real time, thereby enabling real-time detection of crimes and accidents and rapid response.

[0091] An "initial setting file" is a file that contains initial information such as the security camera's unique ID, location information, monitoring area, and monitoring rules.

[0092] "Means for collecting video data" refers to a function for acquiring video data in real time using security cameras.

[0093] The "means for transmitting to the server" is a communication means for periodically transmitting collected video data to the server.

[0094] A "generative machine learning model" is an algorithmic model that learns from collected data and detects abnormal behavior.

[0095] The "means for detecting abnormal behavior" is a function that uses a generative machine learning model to identify abnormal behavior from video data.

[0096] "Means for generating and sending alerts" refers to the function of creating and sending a warning message when abnormal behavior is detected.

[0097] "Means for reporting" is a function that notifies the police and users of relevant information when an abnormality occurs.

[0098] "Means for sending notifications of abnormal behavior in real time" is a function that immediately sends a notification to the user's smartphone, etc. when abnormal behavior is detected.

[0099] "Data analysis and learning means" is a function that analyzes collected video data and updates the generative machine learning model.

[0100] The "learning model transmission means" is a communication means for transmitting the updated generated machine learning model to each terminal.

[0101] The "video confirmation means" is a function that allows the user to check the video in real time.

[0102] This invention is a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to respond quickly and appropriately to abnormal events.

[0103] First, the basic elements that make up the system include a server, terminals (security cameras and their control devices), and users' smartphones. The hardware used includes security cameras, servers, and smartphones, while the software includes generative machine learning models, data transmission protocols, alert generation tools, and reporting systems.

[0104] The server collects and analyzes video data, and trains and updates the generative machine learning model. Specifically, the server analyzes the video data using machine learning frameworks such as TENSORFLOW (registered trademark) and PyTorch to learn patterns of abnormal behavior. The trained generative machine learning model is periodically updated and sent to each device. This operation continuously improves the anomaly detection capabilities of the entire system.

[0105] The device collects video data in real time using security cameras. The collected video data is filtered and compressed, and then periodically sent to a server. The device's most important role is to analyze the video data in real time using a generative machine learning model to detect anomalous behavior. If anomalous behavior is detected, an alert is immediately generated and detailed information about the alert (such as the type of anomaly, its location, the time of occurrence, and video capture) is sent to the server.

[0106] The server analyzes the received alerts and notifies the police or relevant users as necessary. Notification methods include SMS, app notifications, and email. Users can also monitor and operate the system through a smartphone app. When an abnormality is detected, users can check the video in real time and take appropriate action, such as recording and saving the video or making an additional report to the police.

[0107] Specific examples

[0108] Example 1: Detecting a thief

[0109] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[0110] Example 2: Traffic accident detection

[0111] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0112] This system will detect crimes and accidents in real time and enable rapid response, which is expected to reduce the workload of police officers and improve local safety.

[0113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0114] Step 1:

[0115] The server transmits an initial setting file to each security camera.

[0116] Input: Setting information (camera unique ID, location information, monitoring area, monitoring rules)

[0117] Output: Initialization file

[0118] Specific operation: The server obtains the necessary setting information for each camera from the management database, generates an initial setting file, and sends that file to each terminal via the network.

[0119] Step 2:

[0120] The device receives this initial setting file and configures the security camera.

[0121] Input: Initialization file

[0122] Output: Security camera with settings reflected

[0123] Specific operation: The device loads the initial setting file, sets the security camera's unique ID, and applies location information, monitoring area, and monitoring rules to the camera.

[0124] Step 3:

[0125] The device uses a camera to collect video data in real time.

[0126] Input: Real-time video

[0127] Output: Collected video data

[0128] Specific operation: The device's security camera starts operating and continues to capture video. The video data is temporarily stored in memory.

[0129] Step 4:

[0130] The terminal filters and compresses the collected video data and sends it to the server.

[0131] Input: Collected video data

[0132] Output: Compressed video data

[0133] Specific operation: The device filters the collected video data according to the specified format, applies compression algorithms as needed, and then packetizes the data and sends it to the server.

[0134] Step 5:

[0135] The server analyzes the received video data and uses a generative machine learning model to learn patterns of crimes and accidents.

[0136] Input: Video data

[0137] Output: An updated generative machine learning model

[0138] Specific operation: The server inputs the received video data into an analysis tool to extract patterns of crimes and accidents. It analyzes the data using TensorFlow and PyTorch and generates a new trained model.

[0139] Step 6:

[0140] The server sends the updated generative machine learning model to each terminal.

[0141] Input: Updated generative machine learning model

[0142] Output: A new model is distributed to each device.

[0143] How it works: The server packets the latest generative machine learning model for each device and sends it over the network. Each device receives the new model.

[0144] Step 7:

[0145] The device uses a generative machine learning model to analyze video data in real time and detect abnormal behavior.

[0146] Input: Real-time video data, generative machine learning model

[0147] Output: Abnormal behavior detection results

[0148] Specific operation: The device analyzes the captured video data in real time, applies the trained model, and generates an alert if abnormal behavior is detected.

[0149] Step 8:

[0150] When an abnormality is detected, the device generates an alert and sends detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0151] Input: Abnormal behavior detection results

[0152] Output: Alert details

[0153] Specific operation: When the terminal detects abnormal behavior, it generates alert information and packets the data to the server.

[0154] Step 9:

[0155] The server analyzes the received alerts and notifies the police or relevant users as necessary.

[0156] Input: Alert details

[0157] Output: Notification message

[0158] Specific operation: The server analyzes the alert and calls an API to send SMS, email, or app notifications to the police and relevant users.

[0159] Step 10:

[0160] Users can monitor and operate the system through a smartphone app, and when they receive a notification of an abnormality, they can check the video in real time.

[0161] Input: Anomaly detection notification

[0162] Output: Real-time video confirmation by the user

[0163] Specific operation: The user's smartphone app receives a notification and displays real-time video. The user can then use the app to check the video and take necessary action (e.g., save the recording, report additional incidents).

[0164] The above steps enable real-time detection of crimes and accidents and rapid response.

[0165] (Application example 1)

[0166] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] Conventional security and surveillance systems have insufficient accuracy in detecting abnormal behavior, resulting in delayed appropriate responses when an abnormality occurs. Furthermore, even after abnormal behavior is detected, the functions that allow users to respond quickly are limited, making it difficult to check and operate in real time. The present invention aims to solve these problems by providing a system that quickly and accurately detects abnormal behavior, generates and sends appropriate alerts, and allows users to check abnormalities in real time and take appropriate action.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0169] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating an alert based on the detected abnormal behavior and transmitting it to a third party, a notification means for notifying the third party of information related to the occurrence of an abnormality, a notification means for notifying the mobile terminal of the abnormal behavior in real time, and a video confirmation means for enabling the mobile terminal to check the real-time video. This allows abnormal behavior to be detected quickly, and enables the user to check the abnormality in real time and take appropriate action.

[0170] A "generative machine learning model" is an algorithm that learns patterns based on large amounts of data and makes predictions and judgments about new data.

[0171] "Video analysis means" refers to a means for processing and analyzing video data acquired from a camera to detect abnormal behavior.

[0172] "Alert generation and transmission means" refers to means for generating an alert based on detected abnormal behavior and transmitting it to a third party.

[0173] The "reporting means" is a means for reporting related information to a third party when an abnormality occurs.

[0174] A "mobile terminal" is a communication device that a user can carry with them (e.g., a smartphone or tablet).

[0175] The "notification means" is a means for notifying the mobile terminal of abnormal behavior in real time.

[0176] The "video confirmation means" is a means by which a user can check the video in real time using a mobile terminal.

[0177] "Data analysis and learning means" refers to means for analyzing collected data and updating generative machine learning models.

[0178] The "transmission means" is a means for transmitting the updated generative machine learning model to each end device.

[0179] The "confirmation means" is a means for the user of the mobile terminal who has been notified of abnormal behavior by the notification means to issue an alarm.

[0180] The "remote control means" is a means for accessing the mobile terminal at any time to check the details of the alarm and perform additional actions.

[0181] The present invention relates to a crime prevention and surveillance system that uses a generative machine learning model, and is a system that enables real-time detection of crimes and accidents and rapid response. Specific embodiments are described below.

[0182] Initial Setup

[0183] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0184] Data collection and learning

[0185] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0186] Anomaly Detection

[0187] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0188] Alerts and Notifications

[0189] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0190] Real-time notifications and actions

[0191] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[0192] Program processing description

[0193] The system's program consists of an engine that uses Python, OpenCV, and TensorFlow / Keras to detect abnormal behavior. Specifically, it performs the following processes:

[0194] Hardware and software used

[0195] Hardware: Security cameras

[0196] Software: Python, OpenCV, and TensorFlow

[0197] Communication tool: Twilio

[0198] Data processing and calculation

[0199] The server collects real-time video data sent from the camera and analyzes it using a generative machine learning model.

[0200] The terminal uses the generative machine learning model to perform real-time detection of anomalous behavior. When an anomaly is detected, the terminal generates an alert using the alert generation and transmission means and transmits the alert to the server.

[0201] The server analyzes detailed information about abnormal behavior and reports it to relevant organizations and users via SMS, app notifications, and email.

[0202] Specific examples

[0203] Example 1: Thief detection

[0204] While a security camera is monitoring the area around a house, it detects a suspicious person forcing open a window. It recognizes this behavior as a burglary and generates an alert that is sent to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user checks the notification and views the real-time video footage to confirm the burglar's actions. They then work with the police to take further action.

[0205] Example 2: Traffic accident detection

[0206] While monitoring an intersection, a collision between two vehicles is detected. This is determined to be a traffic accident, and an alert is immediately generated and sent to the server. The server receives the alert and notifies the emergency call center and relevant users. The users confirm the notification and view the video in real time. Based on this information, emergency responses can be implemented quickly.

[0207] Prompt Sentence Examples

[0208] Generate AI model prompt:

[0209] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[0210] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[0211] 3. Detect abnormal behavior in public areas and notify in real time.

[0212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0213] Step 1:

[0214] Phase Name: Initial Setup

[0215] How it works: The server sends an initial configuration file to each security camera.

[0216] Input: Camera unique ID, location information, monitoring area, monitoring rules

[0217] Data processing: Assigning unique settings to each camera

[0218] Output: The configuration file is installed on the security camera.

[0219] Details: The server sends a configuration file specific to the security camera, and the device receives this configuration file and prepares for monitoring.

[0220] Step 2:

[0221] Phase Name: Data Collection and Learning

[0222] Operation: The device collects video data in real time through the camera and periodically transmits it to the server.

[0223] Input: Video data from the camera

[0224] Data processing: Collection of video data, pre-processing for analysis, and transmission

[0225] Output: Pre-processed video data is sent to the server

[0226] Details: The device converts the collected video data into a specific format and sends it to a server, where it is stored and analyzed.

[0227] Step 3:

[0228] Phase Name: Data Analysis and Model Update

[0229] How it works: The server analyzes the transmitted video data and updates the generative machine learning model.

[0230] Input: Video data sent from the device

[0231] Data processing: feature extraction from video data, learning of abnormal patterns, model updating

[0232] Output: An updated generative machine learning model

[0233] Details: The server analyzes the collected video data and learns new abnormal behavior patterns. Based on the learning results, it updates the generative machine learning model and sends the new model to each device.

[0234] Step 4:

[0235] Phase Name: Anomalous Behavior Detection

[0236] How it works: The device analyzes video data in real time using updated generative machine learning models.

[0237] Input: Real-time video data from the camera, updated model

[0238] Data processing: feature extraction, matching with abnormal patterns, and determining abnormal behavior

[0239] Output: Abnormal behavior detection result (abnormal / normal)

[0240] Details: The device analyzes the video data from the camera and detects abnormal behavior. If an abnormality is detected, it proceeds to the next step.

[0241] Step 5:

[0242] Phase Name: Alert Generation and Sending

[0243] Operation: If an abnormality is detected, the device immediately generates an alert and sends detailed information to the server.

[0244] Input: Abnormal behavior detection results, detailed information on the abnormality (type, location, time of occurrence, video capture)

[0245] Data processing: Packaging of abnormal behavior data, alert generation

[0246] Output: Generated alerts, sent alerts

[0247] Details: When abnormal behavior is detected, the device generates an alert with detailed information and sends it to the server.

[0248] Step 6:

[0249] Phase Name: Report and Notification

[0250] Operation: The server analyzes the received alert and notifies a third party if necessary.

[0251] Input: Alert received from the device

[0252] Data processing: Alert analysis, notification destination determination, notification content generation

[0253] Output: Report (SMS, app notification, email, etc.)

[0254] Details: The server analyzes the received alert, determines the appropriate contact point, and sends the report, for example, to the police or the user.

[0255] Step 7:

[0256] Phase Name: Real-time Notification and Actions

[0257] Operation: Users monitor and operate the system through a smartphone app.

[0258] Input: Reported alerts, real-time video data

[0259] Data processing: Displaying alerts, streaming video data

[0260] Output: User confirmation and action

[0261] Details: Users receive alerts of abnormal behavior via a smartphone app, check camera footage in real time, and take action such as saving the recording or reporting the incident to the police if necessary.

[0262] Prompt Sentence Examples

[0263] Generate AI model prompt:

[0264] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[0265] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[0266] 3. Detect abnormal behavior in public areas and notify in real time.

[0267] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0268] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. It is also designed to enable prompt and appropriate responses when an abnormality occurs.

[0269] 1. Initial Setup

[0270] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0271] 2. Data collection and learning

[0272] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0273] 3. Anomaly Detection

[0274] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0275] 4. Alerts and Notifications

[0276] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0277] 5. Sentiment Analysis and Notifications

[0278] When the server receives an alert that requires notification, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on the user's reactions and past data.

[0279] The server dynamically changes the content and display method of notifications based on the user's emotional state. For example, if the user is in a panic, the server will notify them of a message encouraging them to stay calm.

[0280] 6. Real-time notifications and actions

[0281] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[0282] 7. Collecting Emotional Data

[0283] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[0284] Specific examples

[0285] Example 1: Detecting a thief

[0286] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, sending it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the police. The server generates an adaptive notification, encouraging the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[0287] Example 2: Traffic accident detection

[0288] While monitoring an intersection, the device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0289] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, reducing the workload of police officers while improving local safety.

[0290] The processing flow will be explained below.

[0291] Step 1:

[0292] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[0293] Step 2:

[0294] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[0295] Step 3:

[0296] The device collects video data through the camera 24 hours a day and stores it in its internal memory.

[0297] Step 4:

[0298] The terminal transmits the collected video data to the server at regular intervals.

[0299] Step 5:

[0300] The server receives the transmitted video data, performs a basic analysis, and learns patterns of crimes and accidents.

[0301] Step 6:

[0302] The server performs data analysis based on the generative machine learning model and reflects the analysis results in the learning model.

[0303] Step 7:

[0304] The server sends the updated generative machine learning model to each terminal.

[0305] Step 8:

[0306] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[0307] Step 9:

[0308] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[0309] Step 10:

[0310] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[0311] Step 11:

[0312] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0313] Step 12:

[0314] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[0315] Step 13:

[0316] If the server determines that a report is necessary, it will make the report based on the alert information. Reports can be made via SMS, app notification, email, or other methods.

[0317] Step 14:

[0318] The server uses a sentiment analysis engine to analyze the user's emotions based on the user's past reactions and behavioral data.

[0319] Step 15:

[0320] Based on the results of the sentiment analysis, the server determines the notification content and display method that suits the user and generates the notification.

[0321] Step 16:

[0322] Users receive alert notifications via an app on their smartphones.

[0323] Step 17:

[0324] The user opens the app to see the notification and view the footage in real time.

[0325] Step 18:

[0326] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[0327] Step 19:

[0328] The server collects and stores user emotional data and uses it to improve the overall performance of the system. The collected emotional data is used to further improve the generative machine learning model and develop individualized countermeasures.

[0329] In this manner, specific embodiments of the invention can be effectively understood and implemented by performing specific actions at each step.

[0330] Example 2

[0331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0332] While conventional security and surveillance systems can detect abnormal behavior, crimes, and accidents in real time, it is difficult to provide optimal responses based on user emotions. Furthermore, they lack mechanisms for making appropriate reports immediately when an abnormality occurs. As a result, emergency responses can be delayed and users can panic.

[0333] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a video analysis means that detects abnormal behavior using a generative machine learning model, an alert generation and transmission means that generates and transmits an alert based on the detected abnormal behavior, a notification means that notifies a security agency or a user of information related to the occurrence of an abnormality, a sentiment analysis means that determines the user's sentiment using a sentiment analysis engine, and a notification adjustment means that dynamically changes the content and display method of the notification based on the user's emotional state. This enables crimes and accidents to be detected in real time, and optimal responses and prompt reporting based on the user's sentiments.

[0334] A "generative machine learning model" is a type of artificial intelligence trained to analyze video data and detect abnormal behavior and patterns.

[0335] "Video analysis means" refers to a means for processing and analyzing video data collected in real time to detect specific abnormal behavior.

[0336] "Alert generation and transmission means" refers to means for generating an alert based on detected anomalous behavior and transmitting the alert to appropriate recipients.

[0337] "Notification means" refers to a means for notifying security agencies and users of information related to the occurrence of an abnormality.

[0338] An "emotion analysis engine" is a program that analyzes and judges a user's emotional state based on their text data and behavioral patterns.

[0339] "Emotion analysis means" refers to means for determining the emotional state of a user using an emotion analysis engine.

[0340] The "notification adjustment means" is a means for dynamically changing the notification content and display method based on the emotional state of the user.

[0341] "Data analysis and learning means" refers to means for analyzing collected video data and updating the generative machine learning model.

[0342] The "learning model transmission means" is a means for transmitting the updated generated machine learning model to each terminal.

[0343] The "emotion data collection means" is a means for collecting and storing user emotion data.

[0344] The "notification means" is a means for sending a notification of abnormal behavior to the user's communication device.

[0345] The "video confirmation means" is a means for allowing the user to check the video in real time.

[0346] The "operation means" is a means by which the user can take necessary action after checking the notification.

[0347] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. This system is also designed to quickly take appropriate action when an abnormality occurs.

[0348] Initial Setup

[0349] The server sends an initial setting file to each security camera, which includes the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0350] Data collection and learning

[0351] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0352] Anomaly Detection

[0353] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0354] Alerts and Notifications

[0355] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies security agencies and relevant users as necessary. Notifications are made via SMS, app notifications, email, etc.

[0356] Sentiment Analysis and Notifications

[0357] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[0358] Real-time notifications and actions

[0359] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action, such as recording and saving the video or reporting the incident to security agencies.

[0360] Collecting Emotional Data

[0361] The server collects and stores user sentiment data, which is used to improve overall system performance. This data is used to further train machine learning models and develop personalized responses.

[0362] Specific examples

[0363] thief detection

[0364] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglar, generates an alert, and sends it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the security agency. The server generates an adaptive notification to encourage the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then work with the security agency to take further action.

[0365] Traffic accident detection

[0366] While monitoring an intersection, a device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0367] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, thereby reducing the workload of security agencies and improving local safety.

[0368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0369] Step 1:

[0370] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location, monitoring area, and monitoring rules.

[0371] Input: Initial setting data (unique ID, location information, monitoring area, monitoring rules)

[0372] Data processing: Generate initial setting file

[0373] Output: Initialization file

[0374] Step 2:

[0375] The terminal performs initial settings for the camera based on the received initial setting file.

[0376] Input: Initialization file from the server

[0377] Data processing: Analyze initial setting file and apply camera settings

[0378] Output: Configuration confirmation message

[0379] Step 3:

[0380] The device uses a camera to collect video data in real time.

[0381] Input: Video data (live feed from camera)

[0382] Data processing: Streaming and temporary storage of video data

[0383] Output: Streaming video data

[0384] Step 4:

[0385] The terminal periodically transmits the collected video data to the server.

[0386] Input: Collected video data

[0387] Data processing: compressing data and preparing it for transmission

[0388] Output: Send video data to the server

[0389] Step 5:

[0390] The server analyzes the received video data and learns patterns of crimes and accidents.

[0391] Input: Video data sent from the device

[0392] Data processing: Data analysis and pattern learning using machine learning algorithms

[0393] Output: Pattern data and trained model

[0394] Step 6:

[0395] The server periodically updates the generative machine learning model and sends the updated model to each terminal.

[0396] Input: A trained model

[0397] Data processing: model updating and format conversion

[0398] Output: Send updated model to device

[0399] Step 7:

[0400] The device analyzes video data in real time using a generative machine learning model.

[0401] Input: Updated generative machine learning model and real-time video data

[0402] Data processing: Analyzing video data using models

[0403] Output: Abnormal behavior detection results

[0404] Step 8:

[0405] When the device detects anomalous behavior (such as a suspicious person breaking open a window or a traffic accident), it determines whether the anomaly matches a known pattern.

[0406] Input: Abnormal behavior detection result

[0407] Data processing: Matching with known patterns

[0408] Output: The decision to generate an alert

[0409] Step 9:

[0410] If the abnormal behavior matches a known pattern, the device generates an alert and immediately sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0411] Input: Alert generation decision result and detailed information about the anomaly

[0412] Data processing: alert generation and information package creation

[0413] Output: Send alert to server

[0414] Step 10:

[0415] The server analyzes the received alerts and notifies security agencies and relevant users as necessary.

[0416] Input: Alert information from the terminal

[0417] Data processing: Alert analysis and notification prioritization

[0418] Output: Notify security agencies and users

[0419] Step 11:

[0420] The server uses a sentiment analysis engine to determine the user's sentiment.

[0421] Input: User reaction data and past data

[0422] Data processing: Applying sentiment analysis algorithms

[0423] Output: User's emotional state

[0424] Step 12:

[0425] The server dynamically changes the notification content and display method based on the user's emotional state.

[0426] Input: User's emotional state

[0427] Data processing: adjusting the content and presentation of notifications

[0428] Output: Adaptive notification to the user

[0429] Step 13:

[0430] Users will receive notifications of detected anomalies via a smartphone app and can check the footage in real time.

[0431] Input: Adaptive notification and anomaly video data from the server

[0432] Data processing: Displaying images within the app

[0433] Output: User confirmation action

[0434] Step 14:

[0435] The user can check the video and take necessary action (e.g., save the video, report the incident to a security agency).

[0436] Input: Anomalous video data

[0437] Data processing: Recording storage and additional reporting

[0438] Output: Recorded data and report information

[0439] Step 15:

[0440] The server collects and stores user emotional data and uses it to improve the overall performance of the system.

[0441] Input: User emotion data

[0442] Data processing: Data collection and storage

[0443] Output: Sentiment database

[0444] Based on the above steps, the system detects abnormal behavior in real time, and provides optimal responses and prompt reporting based on the user's emotions.

[0445] (Application example 2)

[0446] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0447] In addition to realizing real-time detection of crimes and accidents and optimal responses based on user emotions, there was a demand for a system that could provide faster and more appropriate security responses on the devices used by users (smartphones and smart glasses). Another issue is the lack of functionality to adapt notifications based on the user's emotional state and to check video footage in real time when an abnormality occurs. Current systems have difficulty effectively resolving these issues.

[0448] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0449] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating and transmitting an alert based on the detected abnormal behavior, a notification means for notifying the police and the user of information related to the occurrence of an abnormality, an emotion analysis means for analyzing the user's emotions and generating an adapted notification, and a notification means for sending a notification to the user's display terminal in real time. This enables the detection of crimes and accidents and a quick response, and further provides an environment in which the user can respond calmly and quickly by providing an adapted notification according to the user's emotional state.

[0450] A "generative machine learning model" is a model that uses algorithms to automatically detect patterns and anomalies based on collected data.

[0451] The "video analysis means" is a function that processes video data acquired from a camera in real time and detects abnormal behavior.

[0452] The "alert generation and transmission means" is a mechanism for generating an alert based on the detected abnormal behavior and transmitting that information to the user or relevant organizations.

[0453] The "reporting means" is a means for immediately notifying the police and users of relevant information when an abnormality occurs.

[0454] "Emotion analysis means" is a technology that analyzes a user's facial expressions, voice, etc., and determines their emotional state in real time.

[0455] The "notification means" is a means for sending information to the user in real time based on the results of abnormal behavior and emotion analysis.

[0456] "Data analysis and learning means" refers to methods for analyzing collected data and updating machine learning models.

[0457] The "learning model transmission means" is a function that transmits an updated generated machine learning model to each terminal to improve the performance of the entire system.

[0458] The "emotion data analysis means" is a means for optimizing the generative machine learning model by utilizing emotion data collected from users.

[0459] "Video confirmation means" is a function that allows users to check the video in real time through a smartphone or smart glasses.

[0460] The "video storage means" is a means for automatically recording video when abnormal behavior is detected and storing the data.

[0461] This invention relates to a security system for realizing real-time detection of crimes and accidents and optimal responses based on user emotions. This system uses a generative machine learning model to quickly detect abnormal behavior. Specific embodiments are described below.

[0462] Initial Setup

[0463] The server sends an initial configuration file to each security camera. This initial configuration file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial configuration file and begins monitoring according to the settings.

[0464] Data collection and learning

[0465] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0466] Anomaly Detection

[0467] The device analyzes video data in real time using generative machine learning models. When anomalous behavior is detected, the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0468] Alerts and Notifications

[0469] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0470] Sentiment Analysis and Notifications

[0471] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[0472] Real-time notifications and actions

[0473] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can view the video in real time. After viewing the video, users can take necessary action (e.g., save the video or report it to the police).

[0474] Collecting Emotional Data

[0475] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[0476] By using generative machine learning models and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on user sentiment, reducing the workload of police officers and improving community safety.

[0477] Specific examples

[0478] For example, a suspicious behavior could be multiple people trying to break into an office building at night. This system detects such abnormal behavior and notifies the user, "A suspicious person is trying to break into your office. Would you like to report this to the police immediately?" If the user begins to panic, the system notifies the user, "Please stay calm. We will begin the process of reporting this to the police."

[0479] Prompt Sentence Examples

[0480] "Please generate a program that analyzes surveillance camera footage in real time, detects abnormal behavior, and provides appropriate notifications based on the user's emotions."

[0481] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0482] Step 1:

[0483] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[0484] Input: Monitoring area, monitoring rule, camera unique ID, location information

[0485] Output: Initialization file

[0486] Specific operation: Based on the monitoring area and rules received from the administrator, the server generates an initial setting file suitable for each camera and sends it to each terminal.

[0487] Step 2:

[0488] The device receives this initial setting file and begins monitoring the camera according to the settings.

[0489] Input: Initialization file

[0490] Output: Ready to monitor

[0491] Specific operation: The device analyzes the initial configuration file and configures the camera. Once the configuration is complete, it is ready to start monitoring.

[0492] Step 3:

[0493] The device collects video data in real time through the camera and periodically transmits it to the server.

[0494] Input: Real-time video data

[0495] Output: Collected video data

[0496] Specific operation: Video data captured by the camera is sent to the terminal, which then sends the data to the server at regular intervals.

[0497] Step 4:

[0498] The server analyzes the transmitted video data and learns patterns of crimes and accidents. The generative machine learning model is regularly updated to improve the system's overall anomaly detection capabilities.

[0499] Input: Collected video data

[0500] Output: An updated generative machine learning model

[0501] How it works: The server inputs video data into the algorithm to learn and update a generative machine learning model for detecting abnormal behavioral patterns.

[0502] Step 5:

[0503] The device analyzes video data in real time using a generative machine learning model and generates an alert if abnormal behavior is detected.

[0504] Input: Generative machine learning model, real-time video data

[0505] Output: Alert

[0506] Specific operation: Video data acquired in real time is input into a generative machine learning model, and an alert is generated based on the results of abnormal behavior detection.

[0507] Step 6:

[0508] The device sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server and notifies the alert.

[0509] Input: Alert, detailed information about the anomaly

[0510] Output: Alert notification with detailed information

[0511] Specific operation: The generated alert and its detailed information are packaged and sent to the server.

[0512] Step 7:

[0513] The server analyzes the received alerts and notifies the police or relevant users as necessary via SMS, app notifications, email, etc.

[0514] Input: Alert notification with detailed information

[0515] Output: Report

[0516] Specific Actions: Analyze alert notifications and notify appropriate parties via SMS, app notification, or email.

[0517] Step 8:

[0518] When the server receives an alert that requires notification, it uses its emotion analysis function to determine the user's emotions. It analyzes the user's emotional state based on their reactions and past data.

[0519] Input: Alert notification and user reaction data

[0520] Output: User's emotional state

[0521] Specific operation: The sentiment analysis engine analyzes the user's reaction data and determines their current emotional state.

[0522] Step 9:

[0523] The server dynamically changes the content and display method of notifications based on the user's emotional state and sends appropriate notifications.

[0524] Input: User's emotional state, alert notification

[0525] Output: Personalized notifications

[0526] Specific behavior: Generate a notification message based on the emotional state and notify the user in a way that is appropriate for them.

[0527] Step 10:

[0528] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can check the video in real time and take necessary measures.

[0529] Inputs: Personalized notifications, real-time video

[0530] Output: User's response action

[0531] Specific operation: The user receives a notification, checks the real-time video, and takes action as necessary, such as saving the recording or reporting the incident to the police.

[0532] Step 11:

[0533] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the generative machine learning model and develop personalized responses.

[0534] Input: User emotion data

[0535] Output: A database of emotion data, an optimized machine learning model

[0536] What it does: Collects emotion data and stores it in a database. It uses that data to further train the generative machine learning model and optimize the system's performance.

[0537] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0538] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0539] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0540] [Second embodiment]

[0541] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0542] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0543] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0544] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0545] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0546] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0547] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0548] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0549] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0550] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0551] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0552] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0553] This invention relates to a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to enable rapid and appropriate responses when an abnormality occurs.

[0554] 1. Initial Setup

[0555] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0556] 2. Data collection and learning

[0557] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0558] 3. Anomaly Detection

[0559] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0560] 4. Alerts and Notifications

[0561] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0562] 5. Real-time notifications and actions

[0563] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[0564] Specific examples

[0565] Example 1: Detecting a thief

[0566] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[0567] Example 2: Traffic accident detection

[0568] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0569] In this way, this system uses generative machine learning models to detect crimes and accidents in real time and enable rapid response, reducing the workload of police officers while improving the safety of local communities.

[0570] The processing flow will be explained below.

[0571] Step 1:

[0572] The server sends an initial configuration file to each security camera (device), which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[0573] Step 2:

[0574] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[0575] Step 3:

[0576] The device collects video data through its camera 24 hours a day and stores it in its internal memory.

[0577] Step 4:

[0578] The terminal transmits the collected video data to the server at regular intervals.

[0579] Step 5:

[0580] The server receives the transmitted video data and performs an initial analysis to identify patterns of crimes and accidents.

[0581] Step 6:

[0582] The server updates the generative machine learning model based on the analyzed data.

[0583] Step 7:

[0584] The server sends the updated generative machine learning model to each terminal.

[0585] Step 8:

[0586] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[0587] Step 9:

[0588] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[0589] Step 10:

[0590] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[0591] Step 11:

[0592] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0593] Step 12:

[0594] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[0595] Step 13:

[0596] If the server determines that a report is necessary, it will make the report based on the alert information. Reports are made via SMS, app notification, email, etc.

[0597] Step 14:

[0598] Users receive alert notifications via an app on their smartphones.

[0599] Step 15:

[0600] The user opens the app to see the notification and view the footage in real time.

[0601] Step 16:

[0602] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[0603] In this way, by showing the specific operations and processing flow performed at each step, specific embodiments of the invention can be clearly understood.

[0604] Example 1

[0605] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0606] Conventional monitoring systems have difficulty detecting anomalies in real time and responding quickly, which means they are unable to adequately prevent crimes and accidents or respond quickly. Furthermore, the accuracy of anomaly detection is low, resulting in frequent false positives and oversights. Therefore, there is a need to improve the efficiency and accuracy of monitoring systems.

[0607] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0608] In this invention, the server includes means for transmitting an initial setting file, means for collecting video data, means for transmitting the collected video data to the server, means for analyzing the video data using a generative machine learning model to detect abnormal behavior, means for generating and transmitting an alert when abnormal behavior is detected, means for reporting related information to the police and users when an abnormality occurs, and means for transmitting notifications of abnormal behavior in real time, thereby enabling real-time detection of crimes and accidents and rapid response.

[0609] An "initial setting file" is a file that contains initial information such as the security camera's unique ID, location information, monitoring area, and monitoring rules.

[0610] "Means for collecting video data" refers to a function for acquiring video data in real time using security cameras.

[0611] The "means for transmitting to the server" is a communication means for periodically transmitting collected video data to the server.

[0612] A "generative machine learning model" is an algorithmic model that learns from collected data and detects abnormal behavior.

[0613] The "means for detecting abnormal behavior" is a function that uses a generative machine learning model to identify abnormal behavior from video data.

[0614] "Means for generating and sending alerts" refers to the function of creating and sending a warning message when abnormal behavior is detected.

[0615] "Means for reporting" is a function that notifies the police and users of relevant information when an abnormality occurs.

[0616] "Means for sending notifications of abnormal behavior in real time" is a function that immediately sends a notification to the user's smartphone, etc. when abnormal behavior is detected.

[0617] "Data analysis and learning means" is a function that analyzes collected video data and updates the generative machine learning model.

[0618] The "learning model transmission means" is a communication means for transmitting the updated generated machine learning model to each terminal.

[0619] The "video confirmation means" is a function that allows the user to check the video in real time.

[0620] This invention is a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to respond quickly and appropriately to abnormal events.

[0621] First, the basic elements that make up the system include a server, terminals (security cameras and their control devices), and users' smartphones. The hardware used includes security cameras, servers, and smartphones, while the software includes generative machine learning models, data transmission protocols, alert generation tools, and reporting systems.

[0622] The server collects and analyzes video data, and trains and updates the generative machine learning model. Specifically, the server analyzes the video data using machine learning frameworks such as TensorFlow and PyTorch to learn patterns of abnormal behavior. The trained generative machine learning model is periodically updated and sent to each device. This operation continuously improves the anomaly detection capabilities of the entire system.

[0623] The device collects video data in real time using security cameras. The collected video data is filtered and compressed, and then periodically sent to a server. The device's most important role is to analyze the video data in real time using a generative machine learning model to detect anomalous behavior. If anomalous behavior is detected, an alert is immediately generated and detailed information about the alert (such as the type of anomaly, its location, the time of occurrence, and video capture) is sent to the server.

[0624] The server analyzes the received alerts and notifies the police or relevant users as necessary. Notification methods include SMS, app notifications, and email. Users can also monitor and operate the system through a smartphone app. When an abnormality is detected, users can check the video in real time and take appropriate action, such as recording and saving the video or making an additional report to the police.

[0625] Specific examples

[0626] Example 1: Detecting a thief

[0627] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[0628] Example 2: Traffic accident detection

[0629] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0630] This system will detect crimes and accidents in real time and enable rapid response, which is expected to reduce the workload of police officers and improve local safety.

[0631] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0632] Step 1:

[0633] The server transmits an initial setting file to each security camera.

[0634] Input: Setting information (camera unique ID, location information, monitoring area, monitoring rules)

[0635] Output: Initialization file

[0636] Specific operation: The server obtains the necessary setting information for each camera from the management database, generates an initial setting file, and sends that file to each terminal via the network.

[0637] Step 2:

[0638] The device receives this initial setting file and configures the security camera.

[0639] Input: Initialization file

[0640] Output: Security camera with settings reflected

[0641] Specific operation: The device loads the initial setting file, sets the security camera's unique ID, and applies location information, monitoring area, and monitoring rules to the camera.

[0642] Step 3:

[0643] The device uses a camera to collect video data in real time.

[0644] Input: Real-time video

[0645] Output: Collected video data

[0646] Specific operation: The device's security camera starts operating and continues to capture video. The video data is temporarily stored in memory.

[0647] Step 4:

[0648] The terminal filters and compresses the collected video data and sends it to the server.

[0649] Input: Collected video data

[0650] Output: Compressed video data

[0651] Specific operation: The device filters the collected video data according to the specified format, applies compression algorithms as needed, and then packetizes the data and sends it to the server.

[0652] Step 5:

[0653] The server analyzes the received video data and uses a generative machine learning model to learn patterns of crimes and accidents.

[0654] Input: Video data

[0655] Output: An updated generative machine learning model

[0656] Specific operation: The server inputs the received video data into an analysis tool to extract patterns of crimes and accidents. It analyzes the data using TensorFlow and PyTorch and generates a new trained model.

[0657] Step 6:

[0658] The server sends the updated generative machine learning model to each terminal.

[0659] Input: Updated generative machine learning model

[0660] Output: A new model is distributed to each device.

[0661] How it works: The server packets the latest generative machine learning model for each device and sends it over the network. Each device receives the new model.

[0662] Step 7:

[0663] The device uses a generative machine learning model to analyze video data in real time and detect abnormal behavior.

[0664] Input: Real-time video data, generative machine learning model

[0665] Output: Abnormal behavior detection results

[0666] Specific operation: The device analyzes the captured video data in real time, applies the trained model, and generates an alert if abnormal behavior is detected.

[0667] Step 8:

[0668] When an abnormality is detected, the device generates an alert and sends detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0669] Input: Abnormal behavior detection results

[0670] Output: Alert details

[0671] Specific operation: When the terminal detects abnormal behavior, it generates alert information and packets the data to the server.

[0672] Step 9:

[0673] The server analyzes the received alerts and notifies the police or relevant users as necessary.

[0674] Input: Alert details

[0675] Output: Notification message

[0676] Specific operation: The server analyzes the alert and calls an API to send SMS, email, or app notifications to the police and relevant users.

[0677] Step 10:

[0678] Users can monitor and operate the system through a smartphone app, and when they receive a notification of an abnormality, they can check the video in real time.

[0679] Input: Anomaly detection notification

[0680] Output: Real-time video confirmation by the user

[0681] Specific operation: The user's smartphone app receives a notification and displays real-time video. The user can then use the app to check the video and take necessary action (e.g., save the recording, report additional incidents).

[0682] The above steps enable real-time detection of crimes and accidents and rapid response.

[0683] (Application example 1)

[0684] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] Conventional security and surveillance systems have insufficient accuracy in detecting abnormal behavior, resulting in delayed appropriate responses when an abnormality occurs. Furthermore, even after abnormal behavior is detected, the functions that allow users to respond quickly are limited, making it difficult to check and operate in real time. The present invention aims to solve these problems by providing a system that quickly and accurately detects abnormal behavior, generates and sends appropriate alerts, and allows users to check abnormalities in real time and take appropriate action.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0687] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating an alert based on the detected abnormal behavior and transmitting it to a third party, a notification means for notifying the third party of information related to the occurrence of an abnormality, a notification means for notifying the mobile terminal of the abnormal behavior in real time, and a video confirmation means for enabling the mobile terminal to check the real-time video. This allows abnormal behavior to be detected quickly, and enables the user to check the abnormality in real time and take appropriate action.

[0688] A "generative machine learning model" is an algorithm that learns patterns based on large amounts of data and makes predictions and judgments about new data.

[0689] "Video analysis means" refers to a means for processing and analyzing video data acquired from a camera to detect abnormal behavior.

[0690] "Alert generation and transmission means" refers to means for generating an alert based on detected abnormal behavior and transmitting it to a third party.

[0691] The "reporting means" is a means for reporting related information to a third party when an abnormality occurs.

[0692] A "mobile terminal" is a communication device that a user can carry with them (e.g., a smartphone or tablet).

[0693] The "notification means" is a means for notifying the mobile terminal of abnormal behavior in real time.

[0694] The "video confirmation means" is a means by which a user can check the video in real time using a mobile terminal.

[0695] "Data analysis and learning means" refers to means for analyzing collected data and updating generative machine learning models.

[0696] The "transmission means" is a means for transmitting the updated generative machine learning model to each end device.

[0697] The "confirmation means" is a means for the user of the mobile terminal who has been notified of abnormal behavior by the notification means to issue an alarm.

[0698] The "remote control means" is a means for accessing the mobile terminal at any time to check the details of the alarm and perform additional actions.

[0699] The present invention relates to a crime prevention and surveillance system that uses a generative machine learning model, and is a system that enables real-time detection of crimes and accidents and rapid response. Specific embodiments are described below.

[0700] Initial Setup

[0701] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0702] Data collection and learning

[0703] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0704] Anomaly Detection

[0705] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0706] Alerts and Notifications

[0707] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0708] Real-time notifications and actions

[0709] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[0710] Program processing description

[0711] The system's program consists of an engine that uses Python, OpenCV, and TensorFlow / Keras to detect abnormal behavior. Specifically, it performs the following processes:

[0712] Hardware and software used

[0713] Hardware: Security cameras

[0714] Software: Python, OpenCV, and TensorFlow

[0715] Communication tool: Twilio

[0716] Data processing and calculation

[0717] The server collects real-time video data sent from the camera and analyzes it using a generative machine learning model.

[0718] The terminal uses the generative machine learning model to perform real-time detection of anomalous behavior. When an anomaly is detected, the terminal generates an alert using the alert generation and transmission means and transmits the alert to the server.

[0719] The server analyzes detailed information about abnormal behavior and reports it to relevant organizations and users via SMS, app notifications, and email.

[0720] Specific examples

[0721] Example 1: Thief detection

[0722] While a security camera is monitoring the area around a house, it detects a suspicious person forcing open a window. It recognizes this behavior as a burglary and generates an alert that is sent to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user checks the notification and views the real-time video footage to confirm the burglar's actions. They then work with the police to take further action.

[0723] Example 2: Traffic accident detection

[0724] While monitoring an intersection, a collision between two vehicles is detected. This is determined to be a traffic accident, and an alert is immediately generated and sent to the server. The server receives the alert and notifies the emergency call center and relevant users. The users confirm the notification and view the video in real time. Based on this information, emergency responses can be implemented quickly.

[0725] Prompt Sentence Examples

[0726] Generate AI model prompt:

[0727] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[0728] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[0729] 3. Detect abnormal behavior in public areas and notify in real time.

[0730] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0731] Step 1:

[0732] Phase Name: Initial Setup

[0733] How it works: The server sends an initial configuration file to each security camera.

[0734] Input: Camera unique ID, location information, monitoring area, monitoring rules

[0735] Data processing: Assigning unique settings to each camera

[0736] Output: The configuration file is installed on the security camera.

[0737] Details: The server sends a configuration file specific to the security camera, and the device receives this configuration file and prepares for monitoring.

[0738] Step 2:

[0739] Phase Name: Data Collection and Learning

[0740] Operation: The device collects video data in real time through the camera and periodically transmits it to the server.

[0741] Input: Video data from the camera

[0742] Data processing: Collection of video data, pre-processing for analysis, and transmission

[0743] Output: Pre-processed video data is sent to the server

[0744] Details: The device converts the collected video data into a specific format and sends it to a server, where it is stored and analyzed.

[0745] Step 3:

[0746] Phase Name: Data Analysis and Model Update

[0747] How it works: The server analyzes the transmitted video data and updates the generative machine learning model.

[0748] Input: Video data sent from the device

[0749] Data processing: feature extraction from video data, learning of abnormal patterns, model updating

[0750] Output: An updated generative machine learning model

[0751] Details: The server analyzes the collected video data and learns new abnormal behavior patterns. Based on the learning results, it updates the generative machine learning model and sends the new model to each device.

[0752] Step 4:

[0753] Phase Name: Anomalous Behavior Detection

[0754] How it works: The device analyzes video data in real time using updated generative machine learning models.

[0755] Input: Real-time video data from the camera, updated model

[0756] Data processing: feature extraction, matching with abnormal patterns, and determining abnormal behavior

[0757] Output: Abnormal behavior detection result (abnormal / normal)

[0758] Details: The device analyzes the video data from the camera and detects abnormal behavior. If an abnormality is detected, it proceeds to the next step.

[0759] Step 5:

[0760] Phase Name: Alert Generation and Sending

[0761] Operation: If an abnormality is detected, the device immediately generates an alert and sends detailed information to the server.

[0762] Input: Abnormal behavior detection results, detailed information on the abnormality (type, location, time of occurrence, video capture)

[0763] Data processing: Packaging of abnormal behavior data, alert generation

[0764] Output: Generated alerts, sent alerts

[0765] Details: When abnormal behavior is detected, the device generates an alert with detailed information and sends it to the server.

[0766] Step 6:

[0767] Phase Name: Report and Notification

[0768] Operation: The server analyzes the received alert and notifies a third party if necessary.

[0769] Input: Alert received from the device

[0770] Data processing: Alert analysis, notification destination determination, notification content generation

[0771] Output: Report (SMS, app notification, email, etc.)

[0772] Details: The server analyzes the received alert, determines the appropriate contact point, and sends the report, for example, to the police or the user.

[0773] Step 7:

[0774] Phase Name: Real-time Notification and Actions

[0775] Operation: Users monitor and operate the system through a smartphone app.

[0776] Input: Reported alerts, real-time video data

[0777] Data processing: Displaying alerts, streaming video data

[0778] Output: User confirmation and action

[0779] Details: Users receive alerts of abnormal behavior via a smartphone app, check camera footage in real time, and take action such as saving the recording or reporting the incident to the police if necessary.

[0780] Prompt Sentence Examples

[0781] Generate AI model prompt:

[0782] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[0783] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[0784] 3. Detect abnormal behavior in public areas and notify in real time.

[0785] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0786] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. It is also designed to enable prompt and appropriate responses when an abnormality occurs.

[0787] 1. Initial Setup

[0788] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0789] 2. Data collection and learning

[0790] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0791] 3. Anomaly Detection

[0792] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0793] 4. Alerts and Notifications

[0794] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0795] 5. Sentiment Analysis and Notifications

[0796] When the server receives an alert that requires notification, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on the user's reactions and past data.

[0797] The server dynamically changes the content and display method of notifications based on the user's emotional state. For example, if the user is in a panic, the server will notify them of a message encouraging them to stay calm.

[0798] 6. Real-time notifications and actions

[0799] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[0800] 7. Collecting Emotional Data

[0801] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[0802] Specific examples

[0803] Example 1: Detecting a thief

[0804] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, sending it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the police. The server generates an adaptive notification, encouraging the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[0805] Example 2: Traffic accident detection

[0806] While monitoring an intersection, the device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0807] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, reducing the workload of police officers while improving local safety.

[0808] The processing flow will be explained below.

[0809] Step 1:

[0810] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[0811] Step 2:

[0812] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[0813] Step 3:

[0814] The device collects video data through the camera 24 hours a day and stores it in its internal memory.

[0815] Step 4:

[0816] The terminal transmits the collected video data to the server at regular intervals.

[0817] Step 5:

[0818] The server receives the transmitted video data, performs a basic analysis, and learns patterns of crimes and accidents.

[0819] Step 6:

[0820] The server performs data analysis based on the generative machine learning model and reflects the analysis results in the learning model.

[0821] Step 7:

[0822] The server sends the updated generative machine learning model to each terminal.

[0823] Step 8:

[0824] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[0825] Step 9:

[0826] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[0827] Step 10:

[0828] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[0829] Step 11:

[0830] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0831] Step 12:

[0832] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[0833] Step 13:

[0834] If the server determines that a report is necessary, it will make the report based on the alert information. Reports can be made via SMS, app notification, email, or other methods.

[0835] Step 14:

[0836] The server uses a sentiment analysis engine to analyze the user's emotions based on the user's past reactions and behavioral data.

[0837] Step 15:

[0838] Based on the results of the sentiment analysis, the server determines the notification content and display method that suits the user and generates the notification.

[0839] Step 16:

[0840] Users receive alert notifications via an app on their smartphones.

[0841] Step 17:

[0842] The user opens the app to see the notification and view the footage in real time.

[0843] Step 18:

[0844] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[0845] Step 19:

[0846] The server collects and stores user emotional data and uses it to improve the overall performance of the system. The collected emotional data is used to further improve the generative machine learning model and develop individualized countermeasures.

[0847] In this manner, specific embodiments of the invention can be effectively understood and implemented by performing specific actions at each step.

[0848] Example 2

[0849] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0850] While conventional security and surveillance systems can detect abnormal behavior, crimes, and accidents in real time, it is difficult to provide optimal responses based on user emotions. Furthermore, they lack mechanisms for making appropriate reports immediately when an abnormality occurs. As a result, emergency responses can be delayed and users can panic.

[0851] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a video analysis means that detects abnormal behavior using a generative machine learning model, an alert generation and transmission means that generates and transmits an alert based on the detected abnormal behavior, a notification means that notifies a security agency or a user of information related to the occurrence of an abnormality, a sentiment analysis means that determines the user's sentiment using a sentiment analysis engine, and a notification adjustment means that dynamically changes the content and display method of the notification based on the user's emotional state. This enables crimes and accidents to be detected in real time, and optimal responses and prompt reporting based on the user's sentiments.

[0852] A "generative machine learning model" is a type of artificial intelligence trained to analyze video data and detect abnormal behavior and patterns.

[0853] "Video analysis means" refers to a means for processing and analyzing video data collected in real time to detect specific abnormal behavior.

[0854] "Alert generation and transmission means" refers to means for generating an alert based on detected anomalous behavior and transmitting the alert to appropriate recipients.

[0855] "Notification means" refers to a means for notifying security agencies and users of information related to the occurrence of an abnormality.

[0856] An "emotion analysis engine" is a program that analyzes and judges a user's emotional state based on their text data and behavioral patterns.

[0857] "Emotion analysis means" refers to means for determining the emotional state of a user using an emotion analysis engine.

[0858] The "notification adjustment means" is a means for dynamically changing the notification content and display method based on the emotional state of the user.

[0859] "Data analysis and learning means" refers to means for analyzing collected video data and updating the generative machine learning model.

[0860] The "learning model transmission means" is a means for transmitting the updated generated machine learning model to each terminal.

[0861] The "emotion data collection means" is a means for collecting and storing user emotion data.

[0862] The "notification means" is a means for sending a notification of abnormal behavior to the user's communication device.

[0863] The "video confirmation means" is a means for allowing the user to check the video in real time.

[0864] The "operation means" is a means by which the user can take necessary action after checking the notification.

[0865] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. This system is also designed to quickly take appropriate action when an abnormality occurs.

[0866] Initial Setup

[0867] The server sends an initial setting file to each security camera, which includes the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[0868] Data collection and learning

[0869] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0870] Anomaly Detection

[0871] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0872] Alerts and Notifications

[0873] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies security agencies and relevant users as necessary. Notifications are made via SMS, app notifications, email, etc.

[0874] Sentiment Analysis and Notifications

[0875] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[0876] Real-time notifications and actions

[0877] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action, such as recording and saving the video or reporting the incident to security agencies.

[0878] Collecting Emotional Data

[0879] The server collects and stores user sentiment data, which is used to improve overall system performance. This data is used to further train machine learning models and develop personalized responses.

[0880] Specific examples

[0881] thief detection

[0882] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglar, generates an alert, and sends it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the security agency. The server generates an adaptive notification to encourage the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then work with the security agency to take further action.

[0883] Traffic accident detection

[0884] While monitoring an intersection, a device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[0885] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, thereby reducing the workload of security agencies and improving local safety.

[0886] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0887] Step 1:

[0888] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location, monitoring area, and monitoring rules.

[0889] Input: Initial setting data (unique ID, location information, monitoring area, monitoring rules)

[0890] Data processing: Generate initial setting file

[0891] Output: Initialization file

[0892] Step 2:

[0893] The terminal performs initial settings for the camera based on the received initial setting file.

[0894] Input: Initialization file from the server

[0895] Data processing: Analyze initial setting file and apply camera settings

[0896] Output: Configuration confirmation message

[0897] Step 3:

[0898] The device uses a camera to collect video data in real time.

[0899] Input: Video data (live feed from camera)

[0900] Data processing: Streaming and temporary storage of video data

[0901] Output: Streaming video data

[0902] Step 4:

[0903] The terminal periodically transmits the collected video data to the server.

[0904] Input: Collected video data

[0905] Data processing: compressing data and preparing it for transmission

[0906] Output: Send video data to the server

[0907] Step 5:

[0908] The server analyzes the received video data and learns patterns of crimes and accidents.

[0909] Input: Video data sent from the device

[0910] Data processing: Data analysis and pattern learning using machine learning algorithms

[0911] Output: Pattern data and trained model

[0912] Step 6:

[0913] The server periodically updates the generative machine learning model and sends the updated model to each terminal.

[0914] Input: A trained model

[0915] Data processing: model updating and format conversion

[0916] Output: Send updated model to device

[0917] Step 7:

[0918] The device analyzes video data in real time using a generative machine learning model.

[0919] Input: Updated generative machine learning model and real-time video data

[0920] Data processing: Analyzing video data using models

[0921] Output: Abnormal behavior detection results

[0922] Step 8:

[0923] When the device detects anomalous behavior (such as a suspicious person breaking open a window or a traffic accident), it determines whether the anomaly matches a known pattern.

[0924] Input: Abnormal behavior detection result

[0925] Data processing: Matching with known patterns

[0926] Output: The decision to generate an alert

[0927] Step 9:

[0928] If the abnormal behavior matches a known pattern, the device generates an alert and immediately sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[0929] Input: Alert generation decision result and detailed information about the anomaly

[0930] Data processing: alert generation and information package creation

[0931] Output: Send alert to server

[0932] Step 10:

[0933] The server analyzes the received alerts and notifies security agencies and relevant users as necessary.

[0934] Input: Alert information from the terminal

[0935] Data processing: Alert analysis and notification prioritization

[0936] Output: Notify security agencies and users

[0937] Step 11:

[0938] The server uses a sentiment analysis engine to determine the user's sentiment.

[0939] Input: User reaction data and past data

[0940] Data processing: Applying sentiment analysis algorithms

[0941] Output: User's emotional state

[0942] Step 12:

[0943] The server dynamically changes the notification content and display method based on the user's emotional state.

[0944] Input: User's emotional state

[0945] Data processing: adjusting the content and presentation of notifications

[0946] Output: Adaptive notification to the user

[0947] Step 13:

[0948] Users will receive notifications of detected anomalies via a smartphone app and can check the footage in real time.

[0949] Input: Adaptive notification and anomaly video data from the server

[0950] Data processing: Displaying images within the app

[0951] Output: User confirmation action

[0952] Step 14:

[0953] The user can check the video and take necessary action (e.g., save the video, report the incident to a security agency).

[0954] Input: Anomalous video data

[0955] Data processing: Recording storage and additional reporting

[0956] Output: Recorded data and report information

[0957] Step 15:

[0958] The server collects and stores user emotional data and uses it to improve the overall performance of the system.

[0959] Input: User emotion data

[0960] Data processing: Data collection and storage

[0961] Output: Sentiment database

[0962] Based on the above steps, the system detects abnormal behavior in real time, and provides optimal responses and prompt reporting based on the user's emotions.

[0963] (Application example 2)

[0964] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0965] In addition to realizing real-time detection of crimes and accidents and optimal responses based on user emotions, there was a demand for a system that could provide faster and more appropriate security responses on the devices used by users (smartphones and smart glasses). Another issue is the lack of functionality to adapt notifications based on the user's emotional state and to check video footage in real time when an abnormality occurs. Current systems have difficulty effectively resolving these issues.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0967] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating and transmitting an alert based on the detected abnormal behavior, a notification means for notifying the police and the user of information related to the occurrence of an abnormality, an emotion analysis means for analyzing the user's emotions and generating an adapted notification, and a notification means for sending a notification to the user's display terminal in real time. This enables the detection of crimes and accidents and a quick response, and further provides an environment in which the user can respond calmly and quickly by providing an adapted notification according to the user's emotional state.

[0968] A "generative machine learning model" is a model that uses algorithms to automatically detect patterns and anomalies based on collected data.

[0969] The "video analysis means" is a function that processes video data acquired from a camera in real time and detects abnormal behavior.

[0970] The "alert generation and transmission means" is a mechanism for generating an alert based on the detected abnormal behavior and transmitting that information to the user or relevant organizations.

[0971] The "reporting means" is a means for immediately notifying the police and users of relevant information when an abnormality occurs.

[0972] "Emotion analysis means" is a technology that analyzes a user's facial expressions, voice, etc., and determines their emotional state in real time.

[0973] The "notification means" is a means for sending information to the user in real time based on the results of abnormal behavior and emotion analysis.

[0974] "Data analysis and learning means" refers to methods for analyzing collected data and updating machine learning models.

[0975] The "learning model transmission means" is a function that transmits an updated generated machine learning model to each terminal to improve the performance of the entire system.

[0976] The "emotion data analysis means" is a means for optimizing the generative machine learning model by utilizing emotion data collected from users.

[0977] "Video confirmation means" is a function that allows users to check the video in real time through a smartphone or smart glasses.

[0978] The "video storage means" is a means for automatically recording video when abnormal behavior is detected and storing the data.

[0979] This invention relates to a security system for realizing real-time detection of crimes and accidents and optimal responses based on user emotions. This system uses a generative machine learning model to quickly detect abnormal behavior. Specific embodiments are described below.

[0980] Initial Setup

[0981] The server sends an initial configuration file to each security camera. This initial configuration file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial configuration file and begins monitoring according to the settings.

[0982] Data collection and learning

[0983] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[0984] Anomaly Detection

[0985] The device analyzes video data in real time using generative machine learning models. When anomalous behavior is detected, the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[0986] Alerts and Notifications

[0987] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[0988] Sentiment Analysis and Notifications

[0989] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[0990] Real-time notifications and actions

[0991] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can view the video in real time. After viewing the video, users can take necessary action (e.g., save the video or report it to the police).

[0992] Collecting Emotional Data

[0993] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[0994] By using generative machine learning models and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on user sentiment, reducing the workload of police officers and improving community safety.

[0995] Specific examples

[0996] For example, a suspicious behavior could be multiple people trying to break into an office building at night. This system detects such abnormal behavior and notifies the user, "A suspicious person is trying to break into your office. Would you like to report this to the police immediately?" If the user begins to panic, the system notifies the user, "Please stay calm. We will begin the process of reporting this to the police."

[0997] Prompt Sentence Examples

[0998] "Please generate a program that analyzes surveillance camera footage in real time, detects abnormal behavior, and provides appropriate notifications based on the user's emotions."

[0999] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1000] Step 1:

[1001] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[1002] Input: Monitoring area, monitoring rule, camera unique ID, location information

[1003] Output: Initialization file

[1004] Specific operation: Based on the monitoring area and rules received from the administrator, the server generates an initial setting file suitable for each camera and sends it to each terminal.

[1005] Step 2:

[1006] The device receives this initial setting file and begins monitoring the camera according to the settings.

[1007] Input: Initialization file

[1008] Output: Ready to monitor

[1009] Specific operation: The device analyzes the initial configuration file and configures the camera. Once the configuration is complete, it is ready to start monitoring.

[1010] Step 3:

[1011] The device collects video data in real time through the camera and periodically transmits it to the server.

[1012] Input: Real-time video data

[1013] Output: Collected video data

[1014] Specific operation: Video data captured by the camera is sent to the terminal, which then sends the data to the server at regular intervals.

[1015] Step 4:

[1016] The server analyzes the transmitted video data and learns patterns of crimes and accidents. The generative machine learning model is regularly updated to improve the system's overall anomaly detection capabilities.

[1017] Input: Collected video data

[1018] Output: An updated generative machine learning model

[1019] How it works: The server inputs video data into the algorithm to learn and update a generative machine learning model for detecting abnormal behavioral patterns.

[1020] Step 5:

[1021] The device analyzes video data in real time using a generative machine learning model and generates an alert if abnormal behavior is detected.

[1022] Input: Generative machine learning model, real-time video data

[1023] Output: Alert

[1024] Specific operation: Video data acquired in real time is input into a generative machine learning model, and an alert is generated based on the results of abnormal behavior detection.

[1025] Step 6:

[1026] The device sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server and notifies the alert.

[1027] Input: Alert, detailed information about the anomaly

[1028] Output: Alert notification with detailed information

[1029] Specific operation: The generated alert and its detailed information are packaged and sent to the server.

[1030] Step 7:

[1031] The server analyzes the received alerts and notifies the police or relevant users as necessary via SMS, app notifications, email, etc.

[1032] Input: Alert notification with detailed information

[1033] Output: Report

[1034] Specific Actions: Analyze alert notifications and notify appropriate parties via SMS, app notification, or email.

[1035] Step 8:

[1036] When the server receives an alert that requires notification, it uses its emotion analysis function to determine the user's emotions. It analyzes the user's emotional state based on their reactions and past data.

[1037] Input: Alert notification and user reaction data

[1038] Output: User's emotional state

[1039] Specific operation: The sentiment analysis engine analyzes the user's reaction data and determines their current emotional state.

[1040] Step 9:

[1041] The server dynamically changes the content and display method of notifications based on the user's emotional state and sends appropriate notifications.

[1042] Input: User's emotional state, alert notification

[1043] Output: Personalized notifications

[1044] Specific behavior: Generate a notification message based on the emotional state and notify the user in a way that is appropriate for them.

[1045] Step 10:

[1046] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can check the video in real time and take necessary measures.

[1047] Inputs: Personalized notifications, real-time video

[1048] Output: User's response action

[1049] Specific operation: The user receives a notification, checks the real-time video, and takes action as necessary, such as saving the recording or reporting the incident to the police.

[1050] Step 11:

[1051] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the generative machine learning model and develop personalized responses.

[1052] Input: User emotion data

[1053] Output: A database of emotion data, an optimized machine learning model

[1054] What it does: Collects emotion data and stores it in a database. It uses that data to further train the generative machine learning model and optimize the system's performance.

[1055] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1056] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1057] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1058] [Third embodiment]

[1059] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1060] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1061] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1062] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1063] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1064] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1065] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1066] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1067] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1068] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1069] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1070] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1071] This invention relates to a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to enable rapid and appropriate responses when an abnormality occurs.

[1072] 1. Initial Setup

[1073] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1074] 2. Data collection and learning

[1075] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1076] 3. Anomaly Detection

[1077] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1078] 4. Alerts and Notifications

[1079] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[1080] 5. Real-time notifications and actions

[1081] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[1082] Specific examples

[1083] Example 1: Detecting a thief

[1084] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[1085] Example 2: Traffic accident detection

[1086] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1087] In this way, this system uses generative machine learning models to detect crimes and accidents in real time and enable rapid response, reducing the workload of police officers while improving the safety of local communities.

[1088] The processing flow will be explained below.

[1089] Step 1:

[1090] The server sends an initial configuration file to each security camera (device), which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[1091] Step 2:

[1092] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[1093] Step 3:

[1094] The device collects video data through its camera 24 hours a day and stores it in its internal memory.

[1095] Step 4:

[1096] The terminal transmits the collected video data to the server at regular intervals.

[1097] Step 5:

[1098] The server receives the transmitted video data and performs an initial analysis to identify patterns of crimes and accidents.

[1099] Step 6:

[1100] The server updates the generative machine learning model based on the analyzed data.

[1101] Step 7:

[1102] The server sends the updated generative machine learning model to each terminal.

[1103] Step 8:

[1104] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[1105] Step 9:

[1106] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[1107] Step 10:

[1108] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[1109] Step 11:

[1110] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1111] Step 12:

[1112] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[1113] Step 13:

[1114] If the server determines that a report is necessary, it will make the report based on the alert information. Reports are made via SMS, app notification, email, etc.

[1115] Step 14:

[1116] Users receive alert notifications via an app on their smartphones.

[1117] Step 15:

[1118] The user opens the app to see the notification and view the footage in real time.

[1119] Step 16:

[1120] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[1121] In this way, by showing the specific operations and processing flow performed at each step, specific embodiments of the invention can be clearly understood.

[1122] Example 1

[1123] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1124] Conventional monitoring systems have difficulty detecting anomalies in real time and responding quickly, which means they are unable to adequately prevent crimes and accidents or respond quickly. Furthermore, the accuracy of anomaly detection is low, resulting in frequent false positives and oversights. Therefore, there is a need to improve the efficiency and accuracy of monitoring systems.

[1125] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1126] In this invention, the server includes means for transmitting an initial setting file, means for collecting video data, means for transmitting the collected video data to the server, means for analyzing the video data using a generative machine learning model to detect abnormal behavior, means for generating and transmitting an alert when abnormal behavior is detected, means for reporting related information to the police and users when an abnormality occurs, and means for transmitting notifications of abnormal behavior in real time, thereby enabling real-time detection of crimes and accidents and rapid response.

[1127] An "initial setting file" is a file that contains initial information such as the security camera's unique ID, location information, monitoring area, and monitoring rules.

[1128] "Means for collecting video data" refers to a function for acquiring video data in real time using security cameras.

[1129] The "means for transmitting to the server" is a communication means for periodically transmitting collected video data to the server.

[1130] A "generative machine learning model" is an algorithmic model that learns from collected data and detects abnormal behavior.

[1131] The "means for detecting abnormal behavior" is a function that uses a generative machine learning model to identify abnormal behavior from video data.

[1132] "Means for generating and sending alerts" refers to the function of creating and sending a warning message when abnormal behavior is detected.

[1133] "Means for reporting" is a function that notifies the police and users of relevant information when an abnormality occurs.

[1134] "Means for sending notifications of abnormal behavior in real time" is a function that immediately sends a notification to the user's smartphone, etc. when abnormal behavior is detected.

[1135] "Data analysis and learning means" is a function that analyzes collected video data and updates the generative machine learning model.

[1136] The "learning model transmission means" is a communication means for transmitting the updated generated machine learning model to each terminal.

[1137] The "video confirmation means" is a function that allows the user to check the video in real time.

[1138] This invention is a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to respond quickly and appropriately to abnormal events.

[1139] First, the basic elements that make up the system include a server, terminals (security cameras and their control devices), and users' smartphones. The hardware used includes security cameras, servers, and smartphones, while the software includes generative machine learning models, data transmission protocols, alert generation tools, and reporting systems.

[1140] The server collects and analyzes video data, and trains and updates the generative machine learning model. Specifically, the server analyzes the video data using machine learning frameworks such as TensorFlow and PyTorch to learn patterns of abnormal behavior. The trained generative machine learning model is periodically updated and sent to each device. This operation continuously improves the anomaly detection capabilities of the entire system.

[1141] The device collects video data in real time using security cameras. The collected video data is filtered and compressed, and then periodically sent to a server. The device's most important role is to analyze the video data in real time using a generative machine learning model to detect anomalous behavior. If anomalous behavior is detected, an alert is immediately generated and detailed information about the alert (such as the type of anomaly, its location, the time of occurrence, and video capture) is sent to the server.

[1142] The server analyzes the received alerts and notifies the police or relevant users as necessary. Notification methods include SMS, app notifications, and email. Users can also monitor and operate the system through a smartphone app. When an abnormality is detected, users can check the video in real time and take appropriate action, such as recording and saving the video or making an additional report to the police.

[1143] Specific examples

[1144] Example 1: Detecting a thief

[1145] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[1146] Example 2: Traffic accident detection

[1147] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1148] This system will detect crimes and accidents in real time and enable rapid response, which is expected to reduce the workload of police officers and improve local safety.

[1149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1150] Step 1:

[1151] The server transmits an initial setting file to each security camera.

[1152] Input: Setting information (camera unique ID, location information, monitoring area, monitoring rules)

[1153] Output: Initialization file

[1154] Specific operation: The server obtains the necessary setting information for each camera from the management database, generates an initial setting file, and sends that file to each terminal via the network.

[1155] Step 2:

[1156] The device receives this initial setting file and configures the security camera.

[1157] Input: Initialization file

[1158] Output: Security camera with settings reflected

[1159] Specific operation: The device loads the initial setting file, sets the security camera's unique ID, and applies location information, monitoring area, and monitoring rules to the camera.

[1160] Step 3:

[1161] The device uses a camera to collect video data in real time.

[1162] Input: Real-time video

[1163] Output: Collected video data

[1164] Specific operation: The device's security camera starts operating and continues to capture video. The video data is temporarily stored in memory.

[1165] Step 4:

[1166] The terminal filters and compresses the collected video data and sends it to the server.

[1167] Input: Collected video data

[1168] Output: Compressed video data

[1169] Specific operation: The device filters the collected video data according to the specified format, applies compression algorithms as needed, and then packetizes the data and sends it to the server.

[1170] Step 5:

[1171] The server analyzes the received video data and uses a generative machine learning model to learn patterns of crimes and accidents.

[1172] Input: Video data

[1173] Output: An updated generative machine learning model

[1174] Specific operation: The server inputs the received video data into an analysis tool to extract patterns of crimes and accidents. It analyzes the data using TensorFlow and PyTorch and generates a new trained model.

[1175] Step 6:

[1176] The server sends the updated generative machine learning model to each terminal.

[1177] Input: Updated generative machine learning model

[1178] Output: A new model is distributed to each device.

[1179] How it works: The server packets the latest generative machine learning model for each device and sends it over the network. Each device receives the new model.

[1180] Step 7:

[1181] The device uses a generative machine learning model to analyze video data in real time and detect abnormal behavior.

[1182] Input: Real-time video data, generative machine learning model

[1183] Output: Abnormal behavior detection results

[1184] Specific operation: The device analyzes the captured video data in real time, applies the trained model, and generates an alert if abnormal behavior is detected.

[1185] Step 8:

[1186] When an abnormality is detected, the device generates an alert and sends detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1187] Input: Abnormal behavior detection results

[1188] Output: Alert details

[1189] Specific operation: When the terminal detects abnormal behavior, it generates alert information and packets the data to the server.

[1190] Step 9:

[1191] The server analyzes the received alerts and notifies the police or relevant users as necessary.

[1192] Input: Alert details

[1193] Output: Notification message

[1194] Specific operation: The server analyzes the alert and calls an API to send SMS, email, or app notifications to the police and relevant users.

[1195] Step 10:

[1196] Users can monitor and operate the system through a smartphone app, and when they receive a notification of an abnormality, they can check the video in real time.

[1197] Input: Anomaly detection notification

[1198] Output: Real-time video confirmation by the user

[1199] Specific operation: The user's smartphone app receives a notification and displays real-time video. The user can then use the app to check the video and take necessary action (e.g., save the recording, report additional incidents).

[1200] The above steps enable real-time detection of crimes and accidents and rapid response.

[1201] (Application example 1)

[1202] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1203] Conventional security and surveillance systems have insufficient accuracy in detecting abnormal behavior, resulting in delayed appropriate responses when an abnormality occurs. Furthermore, even after abnormal behavior is detected, the functions that allow users to respond quickly are limited, making it difficult to check and operate in real time. The present invention aims to solve these problems by providing a system that quickly and accurately detects abnormal behavior, generates and sends appropriate alerts, and allows users to check abnormalities in real time and take appropriate action.

[1204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1205] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating an alert based on the detected abnormal behavior and transmitting it to a third party, a notification means for notifying the third party of information related to the occurrence of an abnormality, a notification means for notifying the mobile terminal of the abnormal behavior in real time, and a video confirmation means for enabling the mobile terminal to check the real-time video. This allows abnormal behavior to be detected quickly, and enables the user to check the abnormality in real time and take appropriate action.

[1206] A "generative machine learning model" is an algorithm that learns patterns based on large amounts of data and makes predictions and judgments about new data.

[1207] "Video analysis means" refers to a means for processing and analyzing video data acquired from a camera to detect abnormal behavior.

[1208] "Alert generation and transmission means" refers to means for generating an alert based on detected abnormal behavior and transmitting it to a third party.

[1209] The "reporting means" is a means for reporting related information to a third party when an abnormality occurs.

[1210] A "mobile terminal" is a communication device that a user can carry with them (e.g., a smartphone or tablet).

[1211] The "notification means" is a means for notifying the mobile terminal of abnormal behavior in real time.

[1212] The "video confirmation means" is a means by which a user can check the video in real time using a mobile terminal.

[1213] "Data analysis and learning means" refers to means for analyzing collected data and updating generative machine learning models.

[1214] The "transmission means" is a means for transmitting the updated generative machine learning model to each end device.

[1215] The "confirmation means" is a means for the user of the mobile terminal who has been notified of abnormal behavior by the notification means to issue an alarm.

[1216] The "remote control means" is a means for accessing the mobile terminal at any time to check the details of the alarm and perform additional actions.

[1217] The present invention relates to a crime prevention and surveillance system that uses a generative machine learning model, and is a system that enables real-time detection of crimes and accidents and rapid response. Specific embodiments are described below.

[1218] Initial Setup

[1219] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1220] Data collection and learning

[1221] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1222] Anomaly Detection

[1223] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1224] Alerts and Notifications

[1225] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[1226] Real-time notifications and actions

[1227] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[1228] Program processing description

[1229] The system's program consists of an engine that uses Python, OpenCV, and TensorFlow / Keras to detect abnormal behavior. Specifically, it performs the following processes:

[1230] Hardware and software used

[1231] Hardware: Security cameras

[1232] Software: Python, OpenCV, and TensorFlow

[1233] Communication tool: Twilio

[1234] Data processing and calculation

[1235] The server collects real-time video data sent from the camera and analyzes it using a generative machine learning model.

[1236] The terminal uses the generative machine learning model to perform real-time detection of anomalous behavior. When an anomaly is detected, the terminal generates an alert using the alert generation and transmission means and transmits the alert to the server.

[1237] The server analyzes detailed information about abnormal behavior and reports it to relevant organizations and users via SMS, app notifications, and email.

[1238] Specific examples

[1239] Example 1: Thief detection

[1240] While a security camera is monitoring the area around a house, it detects a suspicious person forcing open a window. It recognizes this behavior as a burglary and generates an alert that is sent to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user checks the notification and views the real-time video footage to confirm the burglar's actions. They then work with the police to take further action.

[1241] Example 2: Traffic accident detection

[1242] While monitoring an intersection, a collision between two vehicles is detected. This is determined to be a traffic accident, and an alert is immediately generated and sent to the server. The server receives the alert and notifies the emergency call center and relevant users. The users confirm the notification and view the video in real time. Based on this information, emergency responses can be implemented quickly.

[1243] Prompt Sentence Examples

[1244] Generate AI model prompt:

[1245] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[1246] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[1247] 3. Detect abnormal behavior in public areas and notify in real time.

[1248] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1249] Step 1:

[1250] Phase Name: Initial Setup

[1251] How it works: The server sends an initial configuration file to each security camera.

[1252] Input: Camera unique ID, location information, monitoring area, monitoring rules

[1253] Data processing: Assigning unique settings to each camera

[1254] Output: The configuration file is installed on the security camera.

[1255] Details: The server sends a configuration file specific to the security camera, and the device receives this configuration file and prepares for monitoring.

[1256] Step 2:

[1257] Phase Name: Data Collection and Learning

[1258] Operation: The device collects video data in real time through the camera and periodically transmits it to the server.

[1259] Input: Video data from the camera

[1260] Data processing: Collection of video data, pre-processing for analysis, and transmission

[1261] Output: Pre-processed video data is sent to the server

[1262] Details: The device converts the collected video data into a specific format and sends it to a server, where it is stored and analyzed.

[1263] Step 3:

[1264] Phase Name: Data Analysis and Model Update

[1265] How it works: The server analyzes the transmitted video data and updates the generative machine learning model.

[1266] Input: Video data sent from the device

[1267] Data processing: feature extraction from video data, learning of abnormal patterns, model updating

[1268] Output: An updated generative machine learning model

[1269] Details: The server analyzes the collected video data and learns new abnormal behavior patterns. Based on the learning results, it updates the generative machine learning model and sends the new model to each device.

[1270] Step 4:

[1271] Phase Name: Anomalous Behavior Detection

[1272] How it works: The device analyzes video data in real time using updated generative machine learning models.

[1273] Input: Real-time video data from the camera, updated model

[1274] Data processing: feature extraction, matching with abnormal patterns, and determining abnormal behavior

[1275] Output: Abnormal behavior detection result (abnormal / normal)

[1276] Details: The device analyzes the video data from the camera and detects abnormal behavior. If an abnormality is detected, it proceeds to the next step.

[1277] Step 5:

[1278] Phase Name: Alert Generation and Sending

[1279] Operation: If an abnormality is detected, the device immediately generates an alert and sends detailed information to the server.

[1280] Input: Abnormal behavior detection results, detailed information on the abnormality (type, location, time of occurrence, video capture)

[1281] Data processing: Packaging of abnormal behavior data, alert generation

[1282] Output: Generated alerts, sent alerts

[1283] Details: When abnormal behavior is detected, the device generates an alert with detailed information and sends it to the server.

[1284] Step 6:

[1285] Phase Name: Report and Notification

[1286] Operation: The server analyzes the received alert and notifies a third party if necessary.

[1287] Input: Alert received from the device

[1288] Data processing: Alert analysis, notification destination determination, notification content generation

[1289] Output: Report (SMS, app notification, email, etc.)

[1290] Details: The server analyzes the received alert, determines the appropriate contact point, and sends the report, for example, to the police or the user.

[1291] Step 7:

[1292] Phase Name: Real-time Notification and Actions

[1293] Operation: Users monitor and operate the system through a smartphone app.

[1294] Input: Reported alerts, real-time video data

[1295] Data processing: Displaying alerts, streaming video data

[1296] Output: User confirmation and action

[1297] Details: Users receive alerts of abnormal behavior via a smartphone app, check camera footage in real time, and take action such as saving the recording or reporting the incident to the police if necessary.

[1298] Prompt Sentence Examples

[1299] Generate AI model prompt:

[1300] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[1301] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[1302] 3. Detect abnormal behavior in public areas and notify in real time.

[1303] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1304] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. It is also designed to enable prompt and appropriate responses when an abnormality occurs.

[1305] 1. Initial Setup

[1306] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1307] 2. Data collection and learning

[1308] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1309] 3. Anomaly Detection

[1310] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1311] 4. Alerts and Notifications

[1312] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[1313] 5. Sentiment Analysis and Notifications

[1314] When the server receives an alert that requires notification, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on the user's reactions and past data.

[1315] The server dynamically changes the content and display method of notifications based on the user's emotional state. For example, if the user is in a panic, the server will notify them of a message encouraging them to stay calm.

[1316] 6. Real-time notifications and actions

[1317] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[1318] 7. Collecting Emotional Data

[1319] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[1320] Specific examples

[1321] Example 1: Detecting a thief

[1322] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, sending it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the police. The server generates an adaptive notification, encouraging the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[1323] Example 2: Traffic accident detection

[1324] While monitoring an intersection, the device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1325] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, reducing the workload of police officers while improving local safety.

[1326] The processing flow will be explained below.

[1327] Step 1:

[1328] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[1329] Step 2:

[1330] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[1331] Step 3:

[1332] The device collects video data through the camera 24 hours a day and stores it in its internal memory.

[1333] Step 4:

[1334] The terminal transmits the collected video data to the server at regular intervals.

[1335] Step 5:

[1336] The server receives the transmitted video data, performs a basic analysis, and learns patterns of crimes and accidents.

[1337] Step 6:

[1338] The server performs data analysis based on the generative machine learning model and reflects the analysis results in the learning model.

[1339] Step 7:

[1340] The server sends the updated generative machine learning model to each terminal.

[1341] Step 8:

[1342] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[1343] Step 9:

[1344] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[1345] Step 10:

[1346] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[1347] Step 11:

[1348] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1349] Step 12:

[1350] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[1351] Step 13:

[1352] If the server determines that a report is necessary, it will make the report based on the alert information. Reports can be made via SMS, app notification, email, or other methods.

[1353] Step 14:

[1354] The server uses a sentiment analysis engine to analyze the user's emotions based on the user's past reactions and behavioral data.

[1355] Step 15:

[1356] Based on the results of the sentiment analysis, the server determines the notification content and display method that suits the user and generates the notification.

[1357] Step 16:

[1358] Users receive alert notifications via an app on their smartphones.

[1359] Step 17:

[1360] The user opens the app to see the notification and view the footage in real time.

[1361] Step 18:

[1362] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[1363] Step 19:

[1364] The server collects and stores user emotional data and uses it to improve the overall performance of the system. The collected emotional data is used to further improve the generative machine learning model and develop individualized countermeasures.

[1365] In this manner, specific embodiments of the invention can be effectively understood and implemented by performing specific actions at each step.

[1366] Example 2

[1367] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1368] While conventional security and surveillance systems can detect abnormal behavior, crimes, and accidents in real time, it is difficult to provide optimal responses based on user emotions. Furthermore, they lack mechanisms for making appropriate reports immediately when an abnormality occurs. As a result, emergency responses can be delayed and users can panic.

[1369] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a video analysis means that detects abnormal behavior using a generative machine learning model, an alert generation and transmission means that generates and transmits an alert based on the detected abnormal behavior, a notification means that notifies a security agency or a user of information related to the occurrence of an abnormality, a sentiment analysis means that determines the user's sentiment using a sentiment analysis engine, and a notification adjustment means that dynamically changes the content and display method of the notification based on the user's emotional state. This enables crimes and accidents to be detected in real time, and optimal responses and prompt reporting based on the user's sentiments.

[1370] A "generative machine learning model" is a type of artificial intelligence trained to analyze video data and detect abnormal behavior and patterns.

[1371] "Video analysis means" refers to a means for processing and analyzing video data collected in real time to detect specific abnormal behavior.

[1372] "Alert generation and transmission means" refers to means for generating an alert based on detected anomalous behavior and transmitting the alert to appropriate recipients.

[1373] "Notification means" refers to a means for notifying security agencies and users of information related to the occurrence of an abnormality.

[1374] An "emotion analysis engine" is a program that analyzes and judges a user's emotional state based on their text data and behavioral patterns.

[1375] "Emotion analysis means" refers to means for determining the emotional state of a user using an emotion analysis engine.

[1376] The "notification adjustment means" is a means for dynamically changing the notification content and display method based on the emotional state of the user.

[1377] "Data analysis and learning means" refers to means for analyzing collected video data and updating the generative machine learning model.

[1378] The "learning model transmission means" is a means for transmitting the updated generated machine learning model to each terminal.

[1379] The "emotion data collection means" is a means for collecting and storing user emotion data.

[1380] The "notification means" is a means for sending a notification of abnormal behavior to the user's communication device.

[1381] The "video confirmation means" is a means for allowing the user to check the video in real time.

[1382] The "operation means" is a means by which the user can take necessary action after checking the notification.

[1383] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. This system is also designed to quickly take appropriate action when an abnormality occurs.

[1384] Initial Setup

[1385] The server sends an initial setting file to each security camera, which includes the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1386] Data collection and learning

[1387] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1388] Anomaly Detection

[1389] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1390] Alerts and Notifications

[1391] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies security agencies and relevant users as necessary. Notifications are made via SMS, app notifications, email, etc.

[1392] Sentiment Analysis and Notifications

[1393] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[1394] Real-time notifications and actions

[1395] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action, such as recording and saving the video or reporting the incident to security agencies.

[1396] Collecting Emotional Data

[1397] The server collects and stores user sentiment data, which is used to improve overall system performance. This data is used to further train machine learning models and develop personalized responses.

[1398] Specific examples

[1399] thief detection

[1400] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglar, generates an alert, and sends it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the security agency. The server generates an adaptive notification to encourage the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then work with the security agency to take further action.

[1401] Traffic accident detection

[1402] While monitoring an intersection, a device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1403] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, thereby reducing the workload of security agencies and improving local safety.

[1404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1405] Step 1:

[1406] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location, monitoring area, and monitoring rules.

[1407] Input: Initial setting data (unique ID, location information, monitoring area, monitoring rules)

[1408] Data processing: Generate initial setting file

[1409] Output: Initialization file

[1410] Step 2:

[1411] The terminal performs initial settings for the camera based on the received initial setting file.

[1412] Input: Initialization file from the server

[1413] Data processing: Analyze initial setting file and apply camera settings

[1414] Output: Configuration confirmation message

[1415] Step 3:

[1416] The device uses a camera to collect video data in real time.

[1417] Input: Video data (live feed from camera)

[1418] Data processing: Streaming and temporary storage of video data

[1419] Output: Streaming video data

[1420] Step 4:

[1421] The terminal periodically transmits the collected video data to the server.

[1422] Input: Collected video data

[1423] Data processing: compressing data and preparing it for transmission

[1424] Output: Send video data to the server

[1425] Step 5:

[1426] The server analyzes the received video data and learns patterns of crimes and accidents.

[1427] Input: Video data sent from the device

[1428] Data processing: Data analysis and pattern learning using machine learning algorithms

[1429] Output: Pattern data and trained model

[1430] Step 6:

[1431] The server periodically updates the generative machine learning model and sends the updated model to each terminal.

[1432] Input: A trained model

[1433] Data processing: model updating and format conversion

[1434] Output: Send updated model to device

[1435] Step 7:

[1436] The device analyzes video data in real time using a generative machine learning model.

[1437] Input: Updated generative machine learning model and real-time video data

[1438] Data processing: Analyzing video data using models

[1439] Output: Abnormal behavior detection results

[1440] Step 8:

[1441] When the device detects anomalous behavior (such as a suspicious person breaking open a window or a traffic accident), it determines whether the anomaly matches a known pattern.

[1442] Input: Abnormal behavior detection result

[1443] Data processing: Matching with known patterns

[1444] Output: The decision to generate an alert

[1445] Step 9:

[1446] If the abnormal behavior matches a known pattern, the device generates an alert and immediately sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1447] Input: Alert generation decision result and detailed information about the anomaly

[1448] Data processing: alert generation and information package creation

[1449] Output: Send alert to server

[1450] Step 10:

[1451] The server analyzes the received alerts and notifies security agencies and relevant users as necessary.

[1452] Input: Alert information from the terminal

[1453] Data processing: Alert analysis and notification prioritization

[1454] Output: Notify security agencies and users

[1455] Step 11:

[1456] The server uses a sentiment analysis engine to determine the user's sentiment.

[1457] Input: User reaction data and past data

[1458] Data processing: Applying sentiment analysis algorithms

[1459] Output: User's emotional state

[1460] Step 12:

[1461] The server dynamically changes the notification content and display method based on the user's emotional state.

[1462] Input: User's emotional state

[1463] Data processing: adjusting the content and presentation of notifications

[1464] Output: Adaptive notification to the user

[1465] Step 13:

[1466] Users will receive notifications of detected anomalies via a smartphone app and can check the footage in real time.

[1467] Input: Adaptive notification and anomaly video data from the server

[1468] Data processing: Displaying images within the app

[1469] Output: User confirmation action

[1470] Step 14:

[1471] The user can check the video and take necessary action (e.g., save the video, report the incident to a security agency).

[1472] Input: Anomalous video data

[1473] Data processing: Recording storage and additional reporting

[1474] Output: Recorded data and report information

[1475] Step 15:

[1476] The server collects and stores user emotional data and uses it to improve the overall performance of the system.

[1477] Input: User emotion data

[1478] Data processing: Data collection and storage

[1479] Output: Sentiment database

[1480] Based on the above steps, the system detects abnormal behavior in real time, and provides optimal responses and prompt reporting based on the user's emotions.

[1481] (Application example 2)

[1482] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1483] In addition to realizing real-time detection of crimes and accidents and optimal responses based on user emotions, there was a demand for a system that could provide faster and more appropriate security responses on the devices used by users (smartphones and smart glasses). Another issue is the lack of functionality to adapt notifications based on the user's emotional state and to check video footage in real time when an abnormality occurs. Current systems have difficulty effectively resolving these issues.

[1484] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1485] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating and transmitting an alert based on the detected abnormal behavior, a notification means for notifying the police and the user of information related to the occurrence of an abnormality, an emotion analysis means for analyzing the user's emotions and generating an adapted notification, and a notification means for sending a notification to the user's display terminal in real time. This enables the detection of crimes and accidents and a quick response, and further provides an environment in which the user can respond calmly and quickly by providing an adapted notification according to the user's emotional state.

[1486] A "generative machine learning model" is a model that uses algorithms to automatically detect patterns and anomalies based on collected data.

[1487] The "video analysis means" is a function that processes video data acquired from a camera in real time and detects abnormal behavior.

[1488] The "alert generation and transmission means" is a mechanism for generating an alert based on the detected abnormal behavior and transmitting that information to the user or relevant organizations.

[1489] The "reporting means" is a means for immediately notifying the police and users of relevant information when an abnormality occurs.

[1490] "Emotion analysis means" is a technology that analyzes a user's facial expressions, voice, etc., and determines their emotional state in real time.

[1491] The "notification means" is a means for sending information to the user in real time based on the results of abnormal behavior and emotion analysis.

[1492] "Data analysis and learning means" refers to methods for analyzing collected data and updating machine learning models.

[1493] The "learning model transmission means" is a function that transmits an updated generated machine learning model to each terminal to improve the performance of the entire system.

[1494] The "emotion data analysis means" is a means for optimizing the generative machine learning model by utilizing emotion data collected from users.

[1495] "Video confirmation means" is a function that allows users to check the video in real time through a smartphone or smart glasses.

[1496] The "video storage means" is a means for automatically recording video when abnormal behavior is detected and storing the data.

[1497] This invention relates to a security system for realizing real-time detection of crimes and accidents and optimal responses based on user emotions. This system uses a generative machine learning model to quickly detect abnormal behavior. Specific embodiments are described below.

[1498] Initial Setup

[1499] The server sends an initial configuration file to each security camera. This initial configuration file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial configuration file and begins monitoring according to the settings.

[1500] Data collection and learning

[1501] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1502] Anomaly Detection

[1503] The device analyzes video data in real time using generative machine learning models. When anomalous behavior is detected, the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1504] Alerts and Notifications

[1505] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[1506] Sentiment Analysis and Notifications

[1507] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[1508] Real-time notifications and actions

[1509] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can view the video in real time. After viewing the video, users can take necessary action (e.g., save the video or report it to the police).

[1510] Collecting Emotional Data

[1511] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[1512] By using generative machine learning models and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on user sentiment, reducing the workload of police officers and improving community safety.

[1513] Specific examples

[1514] For example, a suspicious behavior could be multiple people trying to break into an office building at night. This system detects such abnormal behavior and notifies the user, "A suspicious person is trying to break into your office. Would you like to report this to the police immediately?" If the user begins to panic, the system notifies the user, "Please stay calm. We will begin the process of reporting this to the police."

[1515] Prompt Sentence Examples

[1516] "Please generate a program that analyzes surveillance camera footage in real time, detects abnormal behavior, and provides appropriate notifications based on the user's emotions."

[1517] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1518] Step 1:

[1519] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[1520] Input: Monitoring area, monitoring rule, camera unique ID, location information

[1521] Output: Initialization file

[1522] Specific operation: Based on the monitoring area and rules received from the administrator, the server generates an initial setting file suitable for each camera and sends it to each terminal.

[1523] Step 2:

[1524] The device receives this initial setting file and begins monitoring the camera according to the settings.

[1525] Input: Initialization file

[1526] Output: Ready to monitor

[1527] Specific operation: The device analyzes the initial configuration file and configures the camera. Once the configuration is complete, it is ready to start monitoring.

[1528] Step 3:

[1529] The device collects video data in real time through the camera and periodically transmits it to the server.

[1530] Input: Real-time video data

[1531] Output: Collected video data

[1532] Specific operation: Video data captured by the camera is sent to the terminal, which then sends the data to the server at regular intervals.

[1533] Step 4:

[1534] The server analyzes the transmitted video data and learns patterns of crimes and accidents. The generative machine learning model is regularly updated to improve the system's overall anomaly detection capabilities.

[1535] Input: Collected video data

[1536] Output: An updated generative machine learning model

[1537] How it works: The server inputs video data into the algorithm to learn and update a generative machine learning model for detecting abnormal behavioral patterns.

[1538] Step 5:

[1539] The device analyzes video data in real time using a generative machine learning model and generates an alert if abnormal behavior is detected.

[1540] Input: Generative machine learning model, real-time video data

[1541] Output: Alert

[1542] Specific operation: Video data acquired in real time is input into a generative machine learning model, and an alert is generated based on the results of abnormal behavior detection.

[1543] Step 6:

[1544] The device sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server and notifies the alert.

[1545] Input: Alert, detailed information about the anomaly

[1546] Output: Alert notification with detailed information

[1547] Specific operation: The generated alert and its detailed information are packaged and sent to the server.

[1548] Step 7:

[1549] The server analyzes the received alerts and notifies the police or relevant users as necessary via SMS, app notifications, email, etc.

[1550] Input: Alert notification with detailed information

[1551] Output: Report

[1552] Specific Actions: Analyze alert notifications and notify appropriate parties via SMS, app notification, or email.

[1553] Step 8:

[1554] When the server receives an alert that requires notification, it uses its emotion analysis function to determine the user's emotions. It analyzes the user's emotional state based on their reactions and past data.

[1555] Input: Alert notification and user reaction data

[1556] Output: User's emotional state

[1557] Specific operation: The sentiment analysis engine analyzes the user's reaction data and determines their current emotional state.

[1558] Step 9:

[1559] The server dynamically changes the content and display method of notifications based on the user's emotional state and sends appropriate notifications.

[1560] Input: User's emotional state, alert notification

[1561] Output: Personalized notifications

[1562] Specific behavior: Generate a notification message based on the emotional state and notify the user in a way that is appropriate for them.

[1563] Step 10:

[1564] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can check the video in real time and take necessary measures.

[1565] Inputs: Personalized notifications, real-time video

[1566] Output: User's response action

[1567] Specific operation: The user receives a notification, checks the real-time video, and takes action as necessary, such as saving the recording or reporting the incident to the police.

[1568] Step 11:

[1569] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the generative machine learning model and develop personalized responses.

[1570] Input: User emotion data

[1571] Output: A database of emotion data, an optimized machine learning model

[1572] What it does: Collects emotion data and stores it in a database. It uses that data to further train the generative machine learning model and optimize the system's performance.

[1573] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1574] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1575] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1576] [Fourth embodiment]

[1577] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1578] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1579] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1580] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1581] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1582] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1583] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1584] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1585] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1586] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1587] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1588] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1589] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] This invention relates to a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to enable rapid and appropriate responses when an abnormality occurs.

[1591] 1. Initial Setup

[1592] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1593] 2. Data collection and learning

[1594] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1595] 3. Anomaly Detection

[1596] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1597] 4. Alerts and Notifications

[1598] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[1599] 5. Real-time notifications and actions

[1600] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[1601] Specific examples

[1602] Example 1: Detecting a thief

[1603] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[1604] Example 2: Traffic accident detection

[1605] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1606] In this way, this system uses generative machine learning models to detect crimes and accidents in real time and enable rapid response, reducing the workload of police officers while improving the safety of local communities.

[1607] The processing flow will be explained below.

[1608] Step 1:

[1609] The server sends an initial configuration file to each security camera (device), which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[1610] Step 2:

[1611] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[1612] Step 3:

[1613] The device collects video data through its camera 24 hours a day and stores it in its internal memory.

[1614] Step 4:

[1615] The terminal transmits the collected video data to the server at regular intervals.

[1616] Step 5:

[1617] The server receives the transmitted video data and performs an initial analysis to identify patterns of crimes and accidents.

[1618] Step 6:

[1619] The server updates the generative machine learning model based on the analyzed data.

[1620] Step 7:

[1621] The server sends the updated generative machine learning model to each terminal.

[1622] Step 8:

[1623] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[1624] Step 9:

[1625] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[1626] Step 10:

[1627] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[1628] Step 11:

[1629] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1630] Step 12:

[1631] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[1632] Step 13:

[1633] If the server determines that a report is necessary, it will make the report based on the alert information. Reports are made via SMS, app notification, email, etc.

[1634] Step 14:

[1635] Users receive alert notifications via an app on their smartphones.

[1636] Step 15:

[1637] The user opens the app to see the notification and view the footage in real time.

[1638] Step 16:

[1639] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[1640] In this way, by showing the specific operations and processing flow performed at each step, specific embodiments of the invention can be clearly understood.

[1641] Example 1

[1642] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1643] Conventional monitoring systems have difficulty detecting anomalies in real time and responding quickly, which means they are unable to adequately prevent crimes and accidents or respond quickly. Furthermore, the accuracy of anomaly detection is low, resulting in frequent false positives and oversights. Therefore, there is a need to improve the efficiency and accuracy of monitoring systems.

[1644] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1645] In this invention, the server includes means for transmitting an initial setting file, means for collecting video data, means for transmitting the collected video data to the server, means for analyzing the video data using a generative machine learning model to detect abnormal behavior, means for generating and transmitting an alert when abnormal behavior is detected, means for reporting related information to the police and users when an abnormality occurs, and means for transmitting notifications of abnormal behavior in real time, thereby enabling real-time detection of crimes and accidents and rapid response.

[1646] An "initial setting file" is a file that contains initial information such as the security camera's unique ID, location information, monitoring area, and monitoring rules.

[1647] "Means for collecting video data" refers to a function for acquiring video data in real time using security cameras.

[1648] The "means for transmitting to the server" is a communication means for periodically transmitting collected video data to the server.

[1649] A "generative machine learning model" is an algorithmic model that learns from collected data and detects abnormal behavior.

[1650] The "means for detecting abnormal behavior" is a function that uses a generative machine learning model to identify abnormal behavior from video data.

[1651] "Means for generating and sending alerts" refers to the function of creating and sending a warning message when abnormal behavior is detected.

[1652] "Means for reporting" is a function that notifies the police and users of relevant information when an abnormality occurs.

[1653] "Means for sending notifications of abnormal behavior in real time" is a function that immediately sends a notification to the user's smartphone, etc. when abnormal behavior is detected.

[1654] "Data analysis and learning means" is a function that analyzes collected video data and updates the generative machine learning model.

[1655] The "learning model transmission means" is a communication means for transmitting the updated generated machine learning model to each terminal.

[1656] The "video confirmation means" is a function that allows the user to check the video in real time.

[1657] This invention is a crime prevention and surveillance system that uses generative machine learning models to enable real-time detection of crimes and accidents, and is designed to respond quickly and appropriately to abnormal events.

[1658] First, the basic elements that make up the system include a server, terminals (security cameras and their control devices), and users' smartphones. The hardware used includes security cameras, servers, and smartphones, while the software includes generative machine learning models, data transmission protocols, alert generation tools, and reporting systems.

[1659] The server collects and analyzes video data, and trains and updates the generative machine learning model. Specifically, the server analyzes the video data using machine learning frameworks such as TensorFlow and PyTorch to learn patterns of abnormal behavior. The trained generative machine learning model is periodically updated and sent to each device. This operation continuously improves the anomaly detection capabilities of the entire system.

[1660] The device collects video data in real time using security cameras. The collected video data is filtered and compressed, and then periodically sent to a server. The device's most important role is to analyze the video data in real time using a generative machine learning model to detect anomalous behavior. If anomalous behavior is detected, an alert is immediately generated and detailed information about the alert (such as the type of anomaly, its location, the time of occurrence, and video capture) is sent to the server.

[1661] The server analyzes the received alerts and notifies the police or relevant users as necessary. Notification methods include SMS, app notifications, and email. Users can also monitor and operate the system through a smartphone app. When an abnormality is detected, users can check the video in real time and take appropriate action, such as recording and saving the video or making an additional report to the police.

[1662] Specific examples

[1663] Example 1: Detecting a thief

[1664] While the device is monitoring the area around the house, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, which it sends to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user can then check the notification and view real-time footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[1665] Example 2: Traffic accident detection

[1666] While monitoring an intersection, the device detects a collision between two vehicles. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1667] This system will detect crimes and accidents in real time and enable rapid response, which is expected to reduce the workload of police officers and improve local safety.

[1668] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1669] Step 1:

[1670] The server transmits an initial setting file to each security camera.

[1671] Input: Setting information (camera unique ID, location information, monitoring area, monitoring rules)

[1672] Output: Initialization file

[1673] Specific operation: The server obtains the necessary setting information for each camera from the management database, generates an initial setting file, and sends that file to each terminal via the network.

[1674] Step 2:

[1675] The device receives this initial setting file and configures the security camera.

[1676] Input: Initialization file

[1677] Output: Security camera with settings reflected

[1678] Specific operation: The device loads the initial setting file, sets the security camera's unique ID, and applies location information, monitoring area, and monitoring rules to the camera.

[1679] Step 3:

[1680] The device uses a camera to collect video data in real time.

[1681] Input: Real-time video

[1682] Output: Collected video data

[1683] Specific operation: The device's security camera starts operating and continues to capture video. The video data is temporarily stored in memory.

[1684] Step 4:

[1685] The terminal filters and compresses the collected video data and sends it to the server.

[1686] Input: Collected video data

[1687] Output: Compressed video data

[1688] Specific operation: The device filters the collected video data according to the specified format, applies compression algorithms as needed, and then packetizes the data and sends it to the server.

[1689] Step 5:

[1690] The server analyzes the received video data and uses a generative machine learning model to learn patterns of crimes and accidents.

[1691] Input: Video data

[1692] Output: An updated generative machine learning model

[1693] Specific operation: The server inputs the received video data into an analysis tool to extract patterns of crimes and accidents. It analyzes the data using TensorFlow and PyTorch and generates a new trained model.

[1694] Step 6:

[1695] The server sends the updated generative machine learning model to each terminal.

[1696] Input: Updated generative machine learning model

[1697] Output: A new model is distributed to each device.

[1698] How it works: The server packets the latest generative machine learning model for each device and sends it over the network. Each device receives the new model.

[1699] Step 7:

[1700] The device uses a generative machine learning model to analyze video data in real time and detect abnormal behavior.

[1701] Input: Real-time video data, generative machine learning model

[1702] Output: Abnormal behavior detection results

[1703] Specific operation: The device analyzes the captured video data in real time, applies the trained model, and generates an alert if abnormal behavior is detected.

[1704] Step 8:

[1705] When an abnormality is detected, the device generates an alert and sends detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1706] Input: Abnormal behavior detection results

[1707] Output: Alert details

[1708] Specific operation: When the terminal detects abnormal behavior, it generates alert information and packets the data to the server.

[1709] Step 9:

[1710] The server analyzes the received alerts and notifies the police or relevant users as necessary.

[1711] Input: Alert details

[1712] Output: Notification message

[1713] Specific operation: The server analyzes the alert and calls an API to send SMS, email, or app notifications to the police and relevant users.

[1714] Step 10:

[1715] Users can monitor and operate the system through a smartphone app, and when they receive a notification of an abnormality, they can check the video in real time.

[1716] Input: Anomaly detection notification

[1717] Output: Real-time video confirmation by the user

[1718] Specific operation: The user's smartphone app receives a notification and displays real-time video. The user can then use the app to check the video and take necessary action (e.g., save the recording, report additional incidents).

[1719] The above steps enable real-time detection of crimes and accidents and rapid response.

[1720] (Application example 1)

[1721] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1722] Conventional security and surveillance systems have insufficient accuracy in detecting abnormal behavior, resulting in delayed appropriate responses when an abnormality occurs. Furthermore, even after abnormal behavior is detected, the functions that allow users to respond quickly are limited, making it difficult to check and operate in real time. The present invention aims to solve these problems by providing a system that quickly and accurately detects abnormal behavior, generates and sends appropriate alerts, and allows users to check abnormalities in real time and take appropriate action.

[1723] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1724] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating an alert based on the detected abnormal behavior and transmitting it to a third party, a notification means for notifying the third party of information related to the occurrence of an abnormality, a notification means for notifying the mobile terminal of the abnormal behavior in real time, and a video confirmation means for enabling the mobile terminal to check the real-time video. This allows abnormal behavior to be detected quickly, and enables the user to check the abnormality in real time and take appropriate action.

[1725] A "generative machine learning model" is an algorithm that learns patterns based on large amounts of data and makes predictions and judgments about new data.

[1726] "Video analysis means" refers to a means for processing and analyzing video data acquired from a camera to detect abnormal behavior.

[1727] "Alert generation and transmission means" refers to means for generating an alert based on detected abnormal behavior and transmitting it to a third party.

[1728] The "reporting means" is a means for reporting related information to a third party when an abnormality occurs.

[1729] A "mobile terminal" is a communication device that a user can carry with them (e.g., a smartphone or tablet).

[1730] The "notification means" is a means for notifying the mobile terminal of abnormal behavior in real time.

[1731] The "video confirmation means" is a means by which a user can check the video in real time using a mobile terminal.

[1732] "Data analysis and learning means" refers to means for analyzing collected data and updating generative machine learning models.

[1733] The "transmission means" is a means for transmitting the updated generative machine learning model to each end device.

[1734] The "confirmation means" is a means for the user of the mobile terminal who has been notified of abnormal behavior by the notification means to issue an alarm.

[1735] The "remote control means" is a means for accessing the mobile terminal at any time to check the details of the alarm and perform additional actions.

[1736] The present invention relates to a crime prevention and surveillance system that uses a generative machine learning model, and is a system that enables real-time detection of crimes and accidents and rapid response. Specific embodiments are described below.

[1737] Initial Setup

[1738] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1739] Data collection and learning

[1740] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1741] Anomaly Detection

[1742] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1743] Alerts and Notifications

[1744] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[1745] Real-time notifications and actions

[1746] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[1747] Program processing description

[1748] The system's program consists of an engine that uses Python, OpenCV, and TensorFlow / Keras to detect abnormal behavior. Specifically, it performs the following processes:

[1749] Hardware and software used

[1750] Hardware: Security cameras

[1751] Software: Python, OpenCV, and TensorFlow

[1752] Communication tool: Twilio

[1753] Data processing and calculation

[1754] The server collects real-time video data sent from the camera and analyzes it using a generative machine learning model.

[1755] The terminal uses the generative machine learning model to perform real-time detection of anomalous behavior. When an anomaly is detected, the terminal generates an alert using the alert generation and transmission means and transmits the alert to the server.

[1756] The server analyzes detailed information about abnormal behavior and reports it to relevant organizations and users via SMS, app notifications, and email.

[1757] Specific examples

[1758] Example 1: Thief detection

[1759] While a security camera is monitoring the area around a house, it detects a suspicious person forcing open a window. It recognizes this behavior as a burglary and generates an alert that is sent to the server. After receiving the alert, the server sends a notification to the smartphone of the user (the homeowner) and also notifies the police. The user checks the notification and views the real-time video footage to confirm the burglar's actions. They then work with the police to take further action.

[1760] Example 2: Traffic accident detection

[1761] While monitoring an intersection, a collision between two vehicles is detected. This is determined to be a traffic accident, and an alert is immediately generated and sent to the server. The server receives the alert and notifies the emergency call center and relevant users. The users confirm the notification and view the video in real time. Based on this information, emergency responses can be implemented quickly.

[1762] Prompt Sentence Examples

[1763] Generate AI model prompt:

[1764] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[1765] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[1766] 3. Detect abnormal behavior in public areas and notify in real time.

[1767] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1768] Step 1:

[1769] Phase Name: Initial Setup

[1770] How it works: The server sends an initial configuration file to each security camera.

[1771] Input: Camera unique ID, location information, monitoring area, monitoring rules

[1772] Data processing: Assigning unique settings to each camera

[1773] Output: The configuration file is installed on the security camera.

[1774] Details: The server sends a configuration file specific to the security camera, and the device receives this configuration file and prepares for monitoring.

[1775] Step 2:

[1776] Phase Name: Data Collection and Learning

[1777] Operation: The device collects video data in real time through the camera and periodically transmits it to the server.

[1778] Input: Video data from the camera

[1779] Data processing: Collection of video data, pre-processing for analysis, and transmission

[1780] Output: Pre-processed video data is sent to the server

[1781] Details: The device converts the collected video data into a specific format and sends it to a server, where it is stored and analyzed.

[1782] Step 3:

[1783] Phase Name: Data Analysis and Model Update

[1784] How it works: The server analyzes the transmitted video data and updates the generative machine learning model.

[1785] Input: Video data sent from the device

[1786] Data processing: feature extraction from video data, learning of abnormal patterns, model updating

[1787] Output: An updated generative machine learning model

[1788] Details: The server analyzes the collected video data and learns new abnormal behavior patterns. Based on the learning results, it updates the generative machine learning model and sends the new model to each device.

[1789] Step 4:

[1790] Phase Name: Anomalous Behavior Detection

[1791] How it works: The device analyzes video data in real time using updated generative machine learning models.

[1792] Input: Real-time video data from the camera, updated model

[1793] Data processing: feature extraction, matching with abnormal patterns, and determining abnormal behavior

[1794] Output: Abnormal behavior detection result (abnormal / normal)

[1795] Details: The device analyzes the video data from the camera and detects abnormal behavior. If an abnormality is detected, it proceeds to the next step.

[1796] Step 5:

[1797] Phase Name: Alert Generation and Sending

[1798] Operation: If an abnormality is detected, the device immediately generates an alert and sends detailed information to the server.

[1799] Input: Abnormal behavior detection results, detailed information on the abnormality (type, location, time of occurrence, video capture)

[1800] Data processing: Packaging of abnormal behavior data, alert generation

[1801] Output: Generated alerts, sent alerts

[1802] Details: When abnormal behavior is detected, the device generates an alert with detailed information and sends it to the server.

[1803] Step 6:

[1804] Phase Name: Report and Notification

[1805] Operation: The server analyzes the received alert and notifies a third party if necessary.

[1806] Input: Alert received from the device

[1807] Data processing: Alert analysis, notification destination determination, notification content generation

[1808] Output: Report (SMS, app notification, email, etc.)

[1809] Details: The server analyzes the received alert, determines the appropriate contact point, and sends the report, for example, to the police or the user.

[1810] Step 7:

[1811] Phase Name: Real-time Notification and Actions

[1812] Operation: Users monitor and operate the system through a smartphone app.

[1813] Input: Reported alerts, real-time video data

[1814] Data processing: Displaying alerts, streaming video data

[1815] Output: User confirmation and action

[1816] Details: Users receive alerts of abnormal behavior via a smartphone app, check camera footage in real time, and take action such as saving the recording or reporting the incident to the police if necessary.

[1817] Prompt Sentence Examples

[1818] Generate AI model prompt:

[1819] 1. Monitor roads to detect vehicle collisions and risky driving behavior.

[1820] 2. Analyze security camera footage from within the office building to identify suspicious activity.

[1821] 3. Detect abnormal behavior in public areas and notify in real time.

[1822] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1823] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. It is also designed to enable prompt and appropriate responses when an abnormality occurs.

[1824] 1. Initial Setup

[1825] The server sends an initial setting file to each security camera. This file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1826] 2. Data collection and learning

[1827] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1828] 3. Anomaly Detection

[1829] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1830] 4. Alerts and Notifications

[1831] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[1832] 5. Sentiment Analysis and Notifications

[1833] When the server receives an alert that requires notification, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on the user's reactions and past data.

[1834] The server dynamically changes the content and display method of notifications based on the user's emotional state. For example, if the user is in a panic, the server will notify them of a message encouraging them to stay calm.

[1835] 6. Real-time notifications and actions

[1836] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action (e.g., save the video or report it to the police).

[1837] 7. Collecting Emotional Data

[1838] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[1839] Specific examples

[1840] Example 1: Detecting a thief

[1841] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglary and generates an alert, sending it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the police. The server generates an adaptive notification, encouraging the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then cooperate with the police to take further action.

[1842] Example 2: Traffic accident detection

[1843] While monitoring an intersection, the device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1844] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, reducing the workload of police officers while improving local safety.

[1845] The processing flow will be explained below.

[1846] Step 1:

[1847] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[1848] Step 2:

[1849] The device reads the received initial setting file, sets its own location information and monitoring area, and prepares to start monitoring based on this information.

[1850] Step 3:

[1851] The device collects video data through the camera 24 hours a day and stores it in its internal memory.

[1852] Step 4:

[1853] The terminal transmits the collected video data to the server at regular intervals.

[1854] Step 5:

[1855] The server receives the transmitted video data, performs a basic analysis, and learns patterns of crimes and accidents.

[1856] Step 6:

[1857] The server performs data analysis based on the generative machine learning model and reflects the analysis results in the learning model.

[1858] Step 7:

[1859] The server sends the updated generative machine learning model to each terminal.

[1860] Step 8:

[1861] The device receives the updated generative machine learning model and begins analyzing the video data in real time.

[1862] Step 9:

[1863] When a device detects anomalous behavior (such as a theft or a traffic accident), it determines whether the anomaly matches a known pattern.

[1864] Step 10:

[1865] The device checks for pattern matches and immediately generates an alert if an anomaly is detected.

[1866] Step 11:

[1867] The terminal sends the generated alert along with detailed information (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1868] Step 12:

[1869] The server analyzes the received alert and determines whether or not the police or relevant users need to be notified.

[1870] Step 13:

[1871] If the server determines that a report is necessary, it will make the report based on the alert information. Reports can be made via SMS, app notification, email, or other methods.

[1872] Step 14:

[1873] The server uses a sentiment analysis engine to analyze the user's emotions based on the user's past reactions and behavioral data.

[1874] Step 15:

[1875] Based on the results of the sentiment analysis, the server determines the notification content and display method that suits the user and generates the notification.

[1876] Step 16:

[1877] Users receive alert notifications via an app on their smartphones.

[1878] Step 17:

[1879] The user opens the app to see the notification and view the footage in real time.

[1880] Step 18:

[1881] Users can check the real-time video and take additional action if necessary, such as contacting the police directly.

[1882] Step 19:

[1883] The server collects and stores user emotional data and uses it to improve the overall performance of the system. The collected emotional data is used to further improve the generative machine learning model and develop individualized countermeasures.

[1884] In this manner, specific embodiments of the invention can be effectively understood and implemented by performing specific actions at each step.

[1885] Example 2

[1886] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1887] While conventional security and surveillance systems can detect abnormal behavior, crimes, and accidents in real time, it is difficult to provide optimal responses based on user emotions. Furthermore, they lack mechanisms for making appropriate reports immediately when an abnormality occurs. As a result, emergency responses can be delayed and users can panic.

[1888] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a video analysis means that detects abnormal behavior using a generative machine learning model, an alert generation and transmission means that generates and transmits an alert based on the detected abnormal behavior, a notification means that notifies a security agency or a user of information related to the occurrence of an abnormality, a sentiment analysis means that determines the user's sentiment using a sentiment analysis engine, and a notification adjustment means that dynamically changes the content and display method of the notification based on the user's emotional state. This enables crimes and accidents to be detected in real time, and optimal responses and prompt reporting based on the user's sentiments.

[1889] A "generative machine learning model" is a type of artificial intelligence trained to analyze video data and detect abnormal behavior and patterns.

[1890] "Video analysis means" refers to a means for processing and analyzing video data collected in real time to detect specific abnormal behavior.

[1891] "Alert generation and transmission means" refers to means for generating an alert based on detected anomalous behavior and transmitting the alert to appropriate recipients.

[1892] "Notification means" refers to a means for notifying security agencies and users of information related to the occurrence of an abnormality.

[1893] An "emotion analysis engine" is a program that analyzes and judges a user's emotional state based on their text data and behavioral patterns.

[1894] "Emotion analysis means" refers to means for determining the emotional state of a user using an emotion analysis engine.

[1895] The "notification adjustment means" is a means for dynamically changing the notification content and display method based on the emotional state of the user.

[1896] "Data analysis and learning means" refers to means for analyzing collected video data and updating the generative machine learning model.

[1897] The "learning model transmission means" is a means for transmitting the updated generated machine learning model to each terminal.

[1898] The "emotion data collection means" is a means for collecting and storing user emotion data.

[1899] The "notification means" is a means for sending a notification of abnormal behavior to the user's communication device.

[1900] The "video confirmation means" is a means for allowing the user to check the video in real time.

[1901] The "operation means" is a means by which the user can take necessary action after checking the notification.

[1902] This invention relates to a crime prevention and surveillance system that uses a generative machine learning model and a sentiment analysis engine, enabling real-time detection of crimes and accidents and optimal responses based on user sentiment. This system is also designed to quickly take appropriate action when an abnormality occurs.

[1903] Initial Setup

[1904] The server sends an initial setting file to each security camera, which includes the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial setting file and prepares to start monitoring according to the settings.

[1905] Data collection and learning

[1906] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[1907] Anomaly Detection

[1908] The device analyzes video data in real time using a generative machine learning model. When anomalous behavior is detected (for example, a suspicious individual breaking into a window or a traffic accident), the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[1909] Alerts and Notifications

[1910] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies security agencies and relevant users as necessary. Notifications are made via SMS, app notifications, email, etc.

[1911] Sentiment Analysis and Notifications

[1912] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[1913] Real-time notifications and actions

[1914] Users can monitor and operate the system through a smartphone app. When an abnormality is detected, they can check the video in real time. After checking the video, users can take necessary action, such as recording and saving the video or reporting the incident to security agencies.

[1915] Collecting Emotional Data

[1916] The server collects and stores user sentiment data, which is used to improve overall system performance. This data is used to further train machine learning models and develop personalized responses.

[1917] Specific examples

[1918] thief detection

[1919] While the device is monitoring the area around the home, it detects a suspicious person trying to break open a window. The device recognizes this behavior as a burglar, generates an alert, and sends it to the server. After receiving the alert, the server sends a notification to the homeowner's smartphone and also notifies the security agency. The server generates an adaptive notification to encourage the user to remain calm. The user can then confirm the notification and view real-time video footage to confirm the burglar's actions. They can then work with the security agency to take further action.

[1920] Traffic accident detection

[1921] While monitoring an intersection, a device detects a vehicle collision. The device determines this to be a traffic accident and immediately generates an alert and sends it to the server. The server receives the alert and notifies the emergency call center and relevant users. The server analyzes the user's emotional state and personalizes the content of the immediate notification. The user confirms the notification and views the video in real time. Based on this information, emergency response can be carried out quickly.

[1922] In this way, by using a generative machine learning model and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on the user's emotions, thereby reducing the workload of security agencies and improving local safety.

[1923] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1924] Step 1:

[1925] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location, monitoring area, and monitoring rules.

[1926] Input: Initial setting data (unique ID, location information, monitoring area, monitoring rules)

[1927] Data processing: Generate initial setting file

[1928] Output: Initialization file

[1929] Step 2:

[1930] The terminal performs initial settings for the camera based on the received initial setting file.

[1931] Input: Initialization file from the server

[1932] Data processing: Analyze initial setting file and apply camera settings

[1933] Output: Configuration confirmation message

[1934] Step 3:

[1935] The device uses a camera to collect video data in real time.

[1936] Input: Video data (live feed from camera)

[1937] Data processing: Streaming and temporary storage of video data

[1938] Output: Streaming video data

[1939] Step 4:

[1940] The terminal periodically transmits the collected video data to the server.

[1941] Input: Collected video data

[1942] Data processing: compressing data and preparing it for transmission

[1943] Output: Send video data to the server

[1944] Step 5:

[1945] The server analyzes the received video data and learns patterns of crimes and accidents.

[1946] Input: Video data sent from the device

[1947] Data processing: Data analysis and pattern learning using machine learning algorithms

[1948] Output: Pattern data and trained model

[1949] Step 6:

[1950] The server periodically updates the generative machine learning model and sends the updated model to each terminal.

[1951] Input: A trained model

[1952] Data processing: model updating and format conversion

[1953] Output: Send updated model to device

[1954] Step 7:

[1955] The device analyzes video data in real time using a generative machine learning model.

[1956] Input: Updated generative machine learning model and real-time video data

[1957] Data processing: Analyzing video data using models

[1958] Output: Abnormal behavior detection results

[1959] Step 8:

[1960] When the device detects anomalous behavior (such as a suspicious person breaking open a window or a traffic accident), it determines whether the anomaly matches a known pattern.

[1961] Input: Abnormal behavior detection result

[1962] Data processing: Matching with known patterns

[1963] Output: The decision to generate an alert

[1964] Step 9:

[1965] If the abnormal behavior matches a known pattern, the device generates an alert and immediately sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server.

[1966] Input: Alert generation decision result and detailed information about the anomaly

[1967] Data processing: alert generation and information package creation

[1968] Output: Send alert to server

[1969] Step 10:

[1970] The server analyzes the received alerts and notifies security agencies and relevant users as necessary.

[1971] Input: Alert information from the terminal

[1972] Data processing: Alert analysis and notification prioritization

[1973] Output: Notify security agencies and users

[1974] Step 11:

[1975] The server uses a sentiment analysis engine to determine the user's sentiment.

[1976] Input: User reaction data and past data

[1977] Data processing: Applying sentiment analysis algorithms

[1978] Output: User's emotional state

[1979] Step 12:

[1980] The server dynamically changes the notification content and display method based on the user's emotional state.

[1981] Input: User's emotional state

[1982] Data processing: adjusting the content and presentation of notifications

[1983] Output: Adaptive notification to the user

[1984] Step 13:

[1985] Users will receive notifications of detected anomalies via a smartphone app and can check the footage in real time.

[1986] Input: Adaptive notification and anomaly video data from the server

[1987] Data processing: Displaying images within the app

[1988] Output: User confirmation action

[1989] Step 14:

[1990] The user can check the video and take necessary action (e.g., save the video, report the incident to a security agency).

[1991] Input: Anomalous video data

[1992] Data processing: Recording storage and additional reporting

[1993] Output: Recorded data and report information

[1994] Step 15:

[1995] The server collects and stores user emotional data and uses it to improve the overall performance of the system.

[1996] Input: User emotion data

[1997] Data processing: Data collection and storage

[1998] Output: Sentiment database

[1999] Based on the above steps, the system detects abnormal behavior in real time, and provides optimal responses and prompt reporting based on the user's emotions.

[2000] (Application example 2)

[2001] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2002] In addition to realizing real-time detection of crimes and accidents and optimal responses based on user emotions, there was a demand for a system that could provide faster and more appropriate security responses on the devices used by users (smartphones and smart glasses). Another issue is the lack of functionality to adapt notifications based on the user's emotional state and to check video footage in real time when an abnormality occurs. Current systems have difficulty effectively resolving these issues.

[2003] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2004] In this invention, the server includes a video analysis means for detecting abnormal behavior using a generative machine learning model, an alert generation and transmission means for generating and transmitting an alert based on the detected abnormal behavior, a notification means for notifying the police and the user of information related to the occurrence of an abnormality, an emotion analysis means for analyzing the user's emotions and generating an adapted notification, and a notification means for sending a notification to the user's display terminal in real time. This enables the detection of crimes and accidents and a quick response, and further provides an environment in which the user can respond calmly and quickly by providing an adapted notification according to the user's emotional state.

[2005] A "generative machine learning model" is a model that uses algorithms to automatically detect patterns and anomalies based on collected data.

[2006] The "video analysis means" is a function that processes video data acquired from a camera in real time and detects abnormal behavior.

[2007] The "alert generation and transmission means" is a mechanism for generating an alert based on the detected abnormal behavior and transmitting that information to the user or relevant organizations.

[2008] The "reporting means" is a means for immediately notifying the police and users of relevant information when an abnormality occurs.

[2009] "Emotion analysis means" is a technology that analyzes a user's facial expressions, voice, etc., and determines their emotional state in real time.

[2010] The "notification means" is a means for sending information to the user in real time based on the results of abnormal behavior and emotion analysis.

[2011] "Data analysis and learning means" refers to methods for analyzing collected data and updating machine learning models.

[2012] The "learning model transmission means" is a function that transmits an updated generated machine learning model to each terminal to improve the performance of the entire system.

[2013] The "emotion data analysis means" is a means for optimizing the generative machine learning model by utilizing emotion data collected from users.

[2014] "Video confirmation means" is a function that allows users to check the video in real time through a smartphone or smart glasses.

[2015] The "video storage means" is a means for automatically recording video when abnormal behavior is detected and storing the data.

[2016] This invention relates to a security system for realizing real-time detection of crimes and accidents and optimal responses based on user emotions. This system uses a generative machine learning model to quickly detect abnormal behavior. Specific embodiments are described below.

[2017] Initial Setup

[2018] The server sends an initial configuration file to each security camera. This initial configuration file contains the camera's unique ID, location information, monitoring area, and monitoring rules. The device receives this initial configuration file and begins monitoring according to the settings.

[2019] Data collection and learning

[2020] The devices collect video data in real time via cameras. The collected video data is periodically sent to a server. The server analyzes the data and learns patterns of crimes and accidents. The generative machine learning model is periodically updated, and this updated model is sent to each device to improve the anomaly detection capabilities of the entire system.

[2021] Anomaly Detection

[2022] The device analyzes video data in real time using generative machine learning models. When anomalous behavior is detected, the device determines whether the anomaly matches a known pattern. If so, it generates an alert.

[2023] Alerts and Notifications

[2024] If an abnormality is detected, the device immediately generates an alert and sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server. The server analyzes the received alert and notifies the police or relevant users as necessary. Reports are made via SMS, app notification, email, etc.

[2025] Sentiment Analysis and Notifications

[2026] When the server receives an alert that requires reporting, it uses a sentiment analysis engine to determine the user's emotions. It analyzes the user's current emotional state based on their reactions and past data. The server dynamically changes the content and display method of the notification based on the user's emotional state. For example, if the user is in a panic, it will notify them of a message encouraging them to remain calm.

[2027] Real-time notifications and actions

[2028] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can view the video in real time. After viewing the video, users can take necessary action (e.g., save the video or report it to the police).

[2029] Collecting Emotional Data

[2030] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the machine learning model and develop personalized responses.

[2031] By using generative machine learning models and a sentiment analysis engine, this system can detect crimes and accidents in real time and provide optimal responses based on user sentiment, reducing the workload of police officers and improving community safety.

[2032] Specific examples

[2033] For example, a suspicious behavior could be multiple people trying to break into an office building at night. This system detects such abnormal behavior and notifies the user, "A suspicious person is trying to break into your office. Would you like to report this to the police immediately?" If the user begins to panic, the system notifies the user, "Please stay calm. We will begin the process of reporting this to the police."

[2034] Prompt Sentence Examples

[2035] "Please generate a program that analyzes surveillance camera footage in real time, detects abnormal behavior, and provides appropriate notifications based on the user's emotions."

[2036] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2037] Step 1:

[2038] The server sends an initial configuration file to each security camera, which contains the camera's unique ID, location information, monitoring area, and monitoring rules.

[2039] Input: Monitoring area, monitoring rule, camera unique ID, location information

[2040] Output: Initialization file

[2041] Specific operation: Based on the monitoring area and rules received from the administrator, the server generates an initial setting file suitable for each camera and sends it to each terminal.

[2042] Step 2:

[2043] The device receives this initial setting file and begins monitoring the camera according to the settings.

[2044] Input: Initialization file

[2045] Output: Ready to monitor

[2046] Specific operation: The device analyzes the initial configuration file and configures the camera. Once the configuration is complete, it is ready to start monitoring.

[2047] Step 3:

[2048] The device collects video data in real time through the camera and periodically transmits it to the server.

[2049] Input: Real-time video data

[2050] Output: Collected video data

[2051] Specific operation: Video data captured by the camera is sent to the terminal, which then sends the data to the server at regular intervals.

[2052] Step 4:

[2053] The server analyzes the transmitted video data and learns patterns of crimes and accidents. The generative machine learning model is regularly updated to improve the system's overall anomaly detection capabilities.

[2054] Input: Collected video data

[2055] Output: An updated generative machine learning model

[2056] How it works: The server inputs video data into the algorithm to learn and update a generative machine learning model for detecting abnormal behavioral patterns.

[2057] Step 5:

[2058] The device analyzes video data in real time using a generative machine learning model and generates an alert if abnormal behavior is detected.

[2059] Input: Generative machine learning model, real-time video data

[2060] Output: Alert

[2061] Specific operation: Video data acquired in real time is input into a generative machine learning model, and an alert is generated based on the results of abnormal behavior detection.

[2062] Step 6:

[2063] The device sends detailed information about the abnormality (type of abnormality, location, time of occurrence, video capture, etc.) to the server and notifies the alert.

[2064] Input: Alert, detailed information about the anomaly

[2065] Output: Alert notification with detailed information

[2066] Specific operation: The generated alert and its detailed information are packaged and sent to the server.

[2067] Step 7:

[2068] The server analyzes the received alerts and notifies the police or relevant users as necessary via SMS, app notifications, email, etc.

[2069] Input: Alert notification with detailed information

[2070] Output: Report

[2071] Specific Actions: Analyze alert notifications and notify appropriate parties via SMS, app notification, or email.

[2072] Step 8:

[2073] When the server receives an alert that requires notification, it uses its emotion analysis function to determine the user's emotions. It analyzes the user's emotional state based on their reactions and past data.

[2074] Input: Alert notification and user reaction data

[2075] Output: User's emotional state

[2076] Specific operation: The sentiment analysis engine analyzes the user's reaction data and determines their current emotional state.

[2077] Step 9:

[2078] The server dynamically changes the content and display method of notifications based on the user's emotional state and sends appropriate notifications.

[2079] Input: User's emotional state, alert notification

[2080] Output: Personalized notifications

[2081] Specific behavior: Generate a notification message based on the emotional state and notify the user in a way that is appropriate for them.

[2082] Step 10:

[2083] Users can monitor and operate the system through a smartphone or smart glasses app. When an abnormality is detected, they can check the video in real time and take necessary measures.

[2084] Inputs: Personalized notifications, real-time video

[2085] Output: User's response action

[2086] Specific operation: The user receives a notification, checks the real-time video, and takes action as necessary, such as saving the recording or reporting the incident to the police.

[2087] Step 11:

[2088] The server collects and stores user emotional data and uses it to improve the overall performance of the system. This data is used to further train the generative machine learning model and develop personalized responses.

[2089] Input: User emotion data

[2090] Output: A database of emotion data, an optimized machine learning model

[2091] What it does: Collects emotion data and stores it in a database. It uses that data to further train the generative machine learning model and optimize the system's performance.

[2092] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2093] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2094] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2095] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2096] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2097] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2098] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2099] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2100] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2101] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2102] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2103] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2104] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2105] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2106] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2107] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2108] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2109] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2110] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific p...

Claims

1. a video analysis means for detecting abnormal behavior using a generative machine learning model; an alert generating and transmitting means for generating and transmitting an alert based on the detected abnormal behavior; A reporting means for reporting relevant information to the police and users when an abnormality occurs; A system including:

2. a data analysis and learning means for updating the generative machine learning model; a learning model transmission means for transmitting the updated learning model to each terminal; The system of claim 1 further comprising:

3. a notification means for sending a notification of abnormal behavior to a user's smartphone in real time; a video confirmation means for allowing a user to check the video in real time; The system of claim 1 further comprising:

Images