System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting crime and security information across the country; means for analyzing the collected information and identifying dangerous areas; means for monitoring location information of a user and providing notification when the user approaches a dangerous area; means for starting video and audio recording in response to a user instruction; means for detecting vibration and shock; means for emitting light when detecting vibration; and means for automatically providing an emergency notification when detecting shock.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, when individuals go out at night, there are many situations where the risk of crime increases and their safety is threatened. However, current crime prevention measures are not always effective, and individuals are particularly likely to become victims of crime while walking. In response to this, there is a need for a system that can detect danger in advance and respond quickly when a crime actually occurs. The object of the present invention is to provide a system that can grasp the risk of crime in advance, record it in real time, and respond immediately in an emergency. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by the following means. First, it comprises a means for collecting crime information and crime prevention information from all over the country and identifying risk areas based on this information. Next, it comprises a means for monitoring the user's location information and notifying the user when the user approaches a risk area. After receiving the notification, the user can start video and audio recording. It also comprises a means for detecting vibrations and impacts, emitting a light when strong vibrations are detected and automatically making an emergency call when strong impacts are detected. It also comprises a means for analyzing the generated video and audio data and sending an emergency notification based on the analysis results. This system increases personal safety and enables a rapid response when a crime occurs.

[0006] "Crime information" refers to the occurrence of crimes, their methods, the areas where crimes are committed, and data related thereto.

[0007] "Crime prevention information" refers to data for crime prevention measures, including information on alert areas, preventive measures, crime trends, and information provided by public institutions such as the police.

[0008] "User" refers to an individual who uses this system.

[0009] "Location information" is data that indicates the user's current location, and is mainly obtained using GPS.

[0010] "Notification" refers to a message sent from the system to the user to warn or inform them.

[0011] "Video and audio recording" refers to the act of recording video and audio using a camera or microphone.

[0012] "Vibration" refers to physical vibrations that occur on the user's body or on a device, and are detected by a sensor.

[0013] "Impact" refers to a sudden force applied to the user's body or a device, and is detected by an acceleration sensor or the like.

[0014] "Emitting light" refers to emitting strong light using a light source such as an LED or flashlight.

[0015] "Emergency call" refers to the act of automatically contacting emergency services such as the police.

[0016] "Analysis" refers to the process of using collected data to analyze a situation and make a decision.

[0017] "Emergency Notification" refers to a warning message sent to pre-defined contacts when the system detects a dangerous situation. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention is a security system for ensuring personal safety, primarily when out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it automatically starts recording and recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[0040] Program processing

[0041] Collection and notification of crime prevention information

[0042] server

[0043] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. This includes programs that retrieve data via APIs. The server analyzes this data and runs algorithms to identify crime trends and high-risk areas. The identified risk areas are stored in a database that is continually updated.

[0044] User location monitoring and notification

[0045] server

[0046] The server receives location information periodically sent from the user's smartphone. The server-side program monitors this location information in real time, and if it determines that the user's current location is approaching a risky area, it immediately sends a warning notification to the user's smartphone. The notification includes the message, "You are currently approaching a high-risk area."

[0047] Start of video and audio recording

[0048] User

[0049] The user checks the warning notification on their smartphone and taps the recording button on the smartphone app. This activates the Walking Recorder's high-resolution camera and microphone, recording video and audio of the surrounding area. The recorded data is saved to the device's internal storage or a cloud service.

[0050] Vibration and shock detection

[0051] Terminal

[0052] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will immediately flash at maximum brightness to scare off suspicious individuals. At the same time, if a strong impact is detected again, an emergency signal will be automatically sent to your smartphone.

[0053] Automatic reporting

[0054] Terminal

[0055] When a smartphone receives an emergency signal, it automatically calls 110 according to pre-defined emergency protocols, and the call includes the user's current location and real-time video recording.

[0056] emergency notification

[0057] Terminal

[0058] The AI ​​analyzes the camera footage and audio sequentially, and if it determines that the user is in danger, it sends an emergency notification to pre-defined contacts such as family and close friends. The notification includes a message saying "The user is currently in danger," along with location information and a link to the video data. This enables rapid assistance in an emergency.

[0059] Specific examples

[0060] Night Out Scenario

[0061] When a user is walking on a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone. The user receives the notification and starts recording on the Walking Recorder. After a while, a suspicious person approaches from behind and grabs the user's shoulder. The Walking Recorder detects a strong vibration and immediately lights up its LED light. At the same time, recording continues, and if the suspicious person delivers a stronger shock, the Walking Recorder automatically reports the incident and sends the video data to the police. The generating AI then analyzes the situation and sends an emergency notification to the user's family.

[0062] This system not only allows users to go out safely at night, but also enables them to respond quickly in the event of an unforeseen incident.

[0063] The processing flow will be explained below.

[0064] Step 1: Gather crime prevention information

[0065] Subject: Server

[0066] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, specifically collecting data from government crime databases and public data from local police stations.

[0067] Step 2: Data analysis

[0068] Subject: Server

[0069] The server analyzes the collected crime and crime prevention information, using machine learning algorithms to calculate crime trends and risk levels for each area, allowing risk areas to be identified.

[0070] Step 3: Location Monitoring

[0071] Subject: Server

[0072] The server receives current location information periodically sent from the user's smartphone. The user's smartphone uses GPS to identify its location and periodically sends location data to the server.

[0073] Step 4: Risk Area Notification

[0074] Subject: Server

[0075] If the server determines that the user's current location is approaching a high-risk area, it sends a warning notification to the user's smartphone. This notification includes the message, "You are currently approaching a high-risk area."

[0076] Step 5: Start recording

[0077] Subject: User

[0078] The user checks the warning notification on their smartphone, opens the smartphone app, and taps the video and audio recording button, which activates the walking recorder and starts recording video and audio of the surrounding area.

[0079] Step 6: Vibration and shock sensing

[0080] Subject: Terminal

[0081] The Walking Recorder uses a built-in acceleration sensor to constantly detect vibrations. If a strong vibration is detected, the LED light will immediately flash at maximum brightness. If a stronger impact is subsequently detected, an emergency signal will be sent to your smartphone.

[0082] Step 7: Automatic reporting

[0083] Subject: Terminal

[0084] When a smartphone receives an emergency signal, it automatically initiates the 110 emergency call process, which includes the user's current location and recorded video data, allowing the police to immediately be notified of the situation at the scene.

[0085] Step 8: Emergency Notification

[0086] Subject: Terminal

[0087] The AI ​​analyzes the camera footage and audio in real time, and if it determines that the user is in danger, an emergency notification is sent to pre-defined contacts (such as family members). This notification includes a message saying "The user is currently in danger," along with location information and a link to the video data.

[0088] This series of processes improves the safety of the user and allows for a quick response in dangerous situations.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] In modern society, crime prevention measures are extremely important when people are out at night or alone. However, current crime prevention systems have difficulty notifying users of crime information and dangerous areas in real time, and they have limited means of taking prompt and appropriate action in emergencies. For these reasons, a system that can ensure even greater safety is needed.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, and means for monitoring the user's location information and notifying the user when the user approaches the risk area. This allows the user to receive real-time notifications when the user approaches a risk area. The server also includes means for starting video and audio recording in response to a user's instruction, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results, thereby enabling the user to respond quickly and appropriately when they encounter an unexpected situation. The system includes a means for monitoring a user's location information in real time and detecting approach to a dangerous area, a means for sending a warning notification as a push notification, a means for detecting vibrations using an acceleration sensor in the security system and controlling an LED light when the vibration exceeds a threshold, a means for storing the generated video data in a cloud service, a means for sending an emergency signal to a smart device using Bluetooth or Wi-Fi Direct, a means for using a telephone API to automatically make an emergency call, and a means for sending an emergency notification using SMS or a notification API. This allows users to act safely even when out at night or in isolated situations, and enables quick response in the event of an emergency.

[0094] "Means of collecting crime information and crime prevention information from all over the country" refers to the function of regularly obtaining crime information and crime prevention information provided by police stations and public institutions from all over the country and storing it in a database.

[0095] "Means for analyzing the collected information and identifying risk areas" refers to the function of analyzing collected crime information and crime prevention information and identifying crime trends and high-risk areas.

[0096] "Means for monitoring the user's location information and notifying when the user approaches the dangerous area" refers to a function that monitors the location information sent from the user's smart device and sends a warning notification when the user approaches the dangerous area.

[0097] The term "means for starting video and audio recording in response to a user's instruction" refers to a function that allows a user to start video and audio recording by operating an application.

[0098] "Means for detecting vibrations and impacts" refers to the function of detecting vibrations and impacts using an acceleration sensor built into the security device.

[0099] "Means for emitting light when vibration is detected" refers to the function of illuminating the security device's LED light at maximum brightness when vibration is detected by the acceleration sensor.

[0100] "Means for automatically making an emergency call when an impact is detected" refers to a function that automatically makes an emergency call when a strong impact is detected by the acceleration sensor.

[0101] "Means for analyzing generated video and audio data" refers to the function of analyzing video and audio data recorded by a user's smart device.

[0102] "Means for sending an emergency notification based on the analysis results" refers to the function in which the generative AI model analyzes video and audio data and sends a notification if it determines that an emergency exists.

[0103] "Means for monitoring the user's location information in real time and detecting when the user approaches a dangerous area" refers to a function for monitoring the user's location information in real time and detecting when the user approaches a dangerous area.

[0104] "Means for sending a warning notification as a push notification" refers to a function that sends a warning using a push notification when a user approaches a dangerous area.

[0105] "Means for detecting vibrations using an acceleration sensor in a security system and controlling an LED light when the vibrations exceed a threshold" refers to a function that detects vibrations using an acceleration sensor and controls an LED light to turn on when the vibrations exceed a certain threshold.

[0106] "Means for storing generated video data in a cloud service" refers to a function for storing video data recorded on a user's device in a cloud service.

[0107] "Means for transmitting emergency signals to a smart device using Bluetooth or Wi-Fi Direct" refers to the function of transmitting emergency signals to a user's smart device using Bluetooth or Wi-Fi Direct.

[0108] "Means of using a telephone API to automatically make an emergency call" refers to a function that automatically makes an emergency call using a telephone API when an emergency signal is received.

[0109] "Means of sending emergency notifications using SMS or notification APIs" refers to the ability to send emergency notifications to pre-defined contacts using SMS or notification APIs.

[0110] This invention is a security system for ensuring personal safety, primarily when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. It also automatically starts recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[0111] Server Operation

[0112] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. The collection process uses an API, including a program that sends HTTP requests to retrieve data in JSON format and stores it in a database. The server analyzes the collected data and runs machine learning algorithms to identify crime trends and high-risk areas. The results are stored in a database, providing constantly up-to-date information.

[0113] The server also receives location information periodically sent from the user's smartphone and monitors it in real time. When the user approaches a risky area, the server immediately generates a warning notification and sends it to the user's smartphone using a notification service such as Firebase Cloud Messaging (FCM). The specific notification message may say, "You are currently approaching a high-risk area."

[0114] User smartphone behavior

[0115] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. This action calls the smartphone's camera API and microphone API, which starts recording the surrounding video and audio. The recorded data is saved in the device's internal storage or on a cloud service.

[0116] Walking Recorder Operation

[0117] The Walking Recorder constantly detects vibrations using a built-in accelerometer. If a strong vibration is detected, the system immediately turns on the LED light at maximum brightness. This is achieved by acquiring vibration data using the sensor API and calling the LED light control API when a threshold is exceeded. If an even stronger impact is detected, the Walking Recorder will send an emergency signal to your smartphone. This is done using Bluetooth or Wi-Fi Direct.

[0118] Handling emergency calls and notifications

[0119] When a smartphone receives an emergency signal, it automatically makes an emergency call using a phone API according to pre-defined emergency protocols. This call includes the user's current location and recorded video. If the generative AI model analyzes the video and audio and determines that the user is in danger, it sends a notification to pre-defined emergency contacts via SMS or a notification API. The notification includes the user's current location and a link to the video data.

[0120] Specific examples

[0121] For example, if a user is walking down a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone saying, "You are approaching a high-risk area." The user confirms the notification and taps the recording button to begin recording. If a suspicious person approaches and grabs the user's shoulder, the Walking Recorder detects the strong vibration and immediately illuminates its LED light. If an even stronger impact is applied, the Walking Recorder sends an emergency signal to the smartphone, which then automatically reports the incident and sends the video data and current location to the police. The generative AI model then analyzes the situation and sends an emergency notification to the user's family.

[0122] Prompt Sentence Examples

[0123] Could you please explain the details of the nighttime security system? Location monitoring, danger zone notification, sudden vibration and shock detection, video and audio recording start, automatic reporting, and emergency notification.

[0124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0125] Step 1:

[0126] server

[0127] The server collects crime and crime prevention information provided by police stations and public institutions across the country. Specifically, the server periodically sends HTTP requests to the API endpoint and retrieves data in JSON format. This data includes information such as the date, time, location, and type of crime. The collected data is stored in a database and used for subsequent analysis. (Input): Response data from the API. (Output): Crime information stored in the database.

[0128] Step 2:

[0129] server

[0130] The server analyzes the collected crime information using a machine learning algorithm to identify high-risk areas. Specifically, it applies a clustering algorithm (e.g., K-means algorithm) based on past crime data to identify areas where crimes are concentrated. The results of this analysis are stored in a database and used to match users' location information. (Input): Crime information stored in the database. (Output): High-risk areas identified through analysis.

[0131] Step 3:

[0132] server

[0133] The server receives real-time location information periodically sent from the user's smartphone. The received location information is sent to the server as GPS data, and monitoring is performed based on that data. If the user's current location approaches a high-risk area, the server generates a warning notification. (Input): GPS data from the smartphone. (Output): Generated warning notification.

[0134] Step 4:

[0135] server

[0136] The server sends the generated warning notification to the user's smartphone. Specifically, it sends a push notification using a notification service such as Firebase Cloud Messaging (FCM). This notification contains the message "You are currently approaching a high-risk area." (Input): Warning notification generated by the server. (Output): Push notification sent to the user's smartphone.

[0137] Step 5:

[0138] User

[0139] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. The smartphone app calls the camera API and microphone API to begin recording video and audio. This recorded data is saved in the internal storage or on a cloud service. (Input): User's tap operation. (Output): Video and audio data saved on the smartphone.

[0140] Step 6:

[0141] Terminal

[0142] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will turn on at maximum brightness to scare off suspicious people. The sensor API is used to obtain vibration data, and if the vibration exceeds a threshold, the LED control API is called to turn on the light. (Input): Vibration data from the acceleration sensor. (Output): The lit LED light.

[0143] Step 7:

[0144] Terminal

[0145] If the Walking Recorder detects another strong impact, it immediately sends an emergency signal to your smartphone. The emergency signal is transmitted using Bluetooth or Wi-Fi Direct. This signal contains information about the strength of the impact and your location. (Input): Strong impact data from the acceleration sensor. (Output): Emergency signal sent to your smartphone.

[0146] Step 8:

[0147] Smartphone

[0148] When the smartphone receives an emergency signal, it automatically calls 110 using the phone API according to the pre-set emergency protocol. The call includes the user's current location and real-time video recording. (Input): Emergency signal from the walking recorder. (Output): Current location and video data sent along with the automatic call.

[0149] Step 9:

[0150] Generative AI Models

[0151] The generating AI sequentially analyzes camera footage and audio, and if it determines that the user is in danger, it sends an emergency notification to pre-set contacts such as family and close friends. Specifically, it uses a notification API and SMS service to send a message saying "The user is currently in danger," along with location information and a link to the video data. (Input): Camera footage and audio data. (Output): Emergency notification sent to family and pre-set contacts.

[0152] (Application example 1)

[0153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] Ensuring personal safety when out at night is extremely important, but conventional security systems often lack sufficient risk area notification and emergency response capabilities, requiring users to operate the system directly. Furthermore, users lack a means to receive visual information, making it difficult to quickly recognize danger. Furthermore, automation of emergency calls and video and audio recording is insufficient, potentially preventing effective action even in situations requiring a rapid response.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0156] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country and monitoring user location information, means for analyzing the collected information to identify risk areas, means for notifying the user when they approach a risk area, and means for starting video and audio recording in response to a user's instruction.In this invention, by providing an easy-to-use device and interface, it is possible to quickly and effectively recognize danger and take appropriate action, by detecting vibrations and impacts using a built-in sensor and emitting light to automatically start video and audio recording, means for analyzing the generated video and audio data and sending an emergency notification based on the analysis results, means for providing a visual display device with a notification function to warn the user in real time, and means for implementing an interface function for the user to start video and audio recording.

[0157] "Crime information and crime prevention information" refers to data on crime occurrence and prevention measures provided by police stations and public institutions across the country.

[0158] "Risk areas" refer to areas that have been identified as having a high risk of crime occurring based on the results of an analysis of collected crime and crime prevention information.

[0159] "Location Information" means data that indicates a user's current geographic location, obtained through GPS or other location-determining technology.

[0160] "Notification" refers to the action of displaying or sending a warning or information to a user's device when the user approaches a dangerous area.

[0161] "Video and audio recording" refers to the act of activating the camera and microphone, either at the user's command or automatically, to record video and audio of the surrounding area.

[0162] "Vibration and shock" refers to physical shaking or impact detected by an accelerometer.

[0163] "Emitting light" refers to the act of turning on an LED light or similar as a visual warning when vibration is detected.

[0164] "Emergency call" refers to the action of automatically notifying pre-defined contacts or the police to request help.

[0165] "Analyzing the generated video and audio data" refers to the process of analyzing video and audio data using AI models, etc., to determine dangerous situations.

[0166] "Send emergency notification" refers to the action of notifying set contacts, such as family and friends, of the user's situation based on the analysis results.

[0167] "Visual display device" refers to a device (e.g., smart glasses) worn by a user that displays information in real time.

[0168] The "interface function" refers to a function that provides an operation method for the user to start video and audio recording.

[0169] This invention is a security system to ensure the safety of users when they are out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies them when they approach a dangerous area. It also provides a series of processes that automatically start recording and recording when it detects vibration or impact, and automatically reports and notifies users in the event of an emergency.

[0170] System configuration

[0171] The system consists of the following components:

[0172] 1. Server:

[0173] Collection of crime and crime prevention information: Data provided by police stations and public institutions across the country is periodically obtained via API. The server analyzes this data to identify crime trends and high-risk areas.

[0174] Monitoring user location information: The server periodically receives location information from the user's smart device and sends a warning notification to the user if the device approaches a dangerous area.

[0175] 2. User terminal (smart glasses):

[0176] Video and audio recording function: When the user confirms the alert notification, the smart glasses' built-in camera and microphone will activate and record the surrounding video and audio. The recorded data will be saved in the internal storage or cloud service.

[0177] Vibration and impact detection: The smart glasses' built-in acceleration sensor constantly detects vibrations, and when a strong vibration is detected, the LED light will turn on. At the same time, if a strong impact is detected, an emergency signal will be automatically sent to the server.

[0178] Automatic notification function: When the smart glasses receive an emergency signal, they will automatically make an emergency call according to the emergency protocol pre-set on the server side. The call will include the user's current location information and real-time video recording data.

[0179] Emergency Notification: If the generative AI model analyzes camera footage and audio and determines that the user is in danger, it will send an emergency notification to pre-defined contacts, including location information and a link to the video recording.

[0180] Hardware and Software Details

[0181] Hardware:

[0182] Smart Glasses: Uses a display (e.g., smart glasses) that displays information in real time.

[0183] Acceleration sensor: Built-in sensor for detecting vibration and impact.

[0184] High-resolution camera and microphone: Equipped with a device for recording video and audio of the surroundings.

[0185] LED light: Visual warning when vibration is detected.

[0186] software:

[0187] Server: Uses cloud services such as AWS (registered trademark) and Google (registered trademark) Cloud Platform to collect and analyze data and execute emergency protocols.

[0188] Generative AI model: OpenAI (registered trademark) GPT-4 (registered trademark) and others are used to analyze video and audio data and determine emergency situations.

[0189] Notification app: An application for iOS and ANDROID devices that monitors the user's location, controls video and audio recording, and sends emergency notifications.

[0190] Specific examples

[0191] A user puts on smart glasses while walking down a street at night. The server detects that the user is approaching a dangerous area, and a warning message is displayed on the smart glasses. The user confirms the warning and starts recording. If a suspicious person approaches and grabs the user's shoulder, the smart glasses detect a strong vibration, turn on an LED light, and automatically start recording. If the suspicious person causes further harm, the generating AI analyzes the situation and makes an emergency call via the server. In addition, an emergency notification is sent to family and friends.

[0192] Example prompts for generative AI models

[0193] "I would like to develop the following security application. A system in which users wear smart glasses and are warned of approaching dangerous areas when going out at night. The glasses can record video and audio using the camera and microphone, and will turn on an LED light if a suspicious person comes into contact with the user. If vibration or impact is detected, the glasses will automatically report the user and send an emergency notification."

[0194] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0195] Step 1:

[0196] The server collects crime and crime prevention information from all over the country. The input is data provided by police stations and public institutions. This data is obtained via API and stored in a database on the server. The output is a database of the collected crime and crime prevention information.

[0197] Step 2:

[0198] The server analyzes the collected crime and crime prevention information and identifies risk areas. The input is crime information in the database. The server performs data analysis to identify high-risk areas using statistical models and machine learning algorithms. The output is information on areas identified as high-risk areas.

[0199] Step 3:

[0200] The server receives location information periodically sent from the user's smart glasses. The input is the user's location information. The server monitors this information in real time, and if it determines that the user is approaching a dangerous area, it sends a warning notification to the user's smart glasses. The output is a warning message displayed on the user's smart glasses.

[0201] Step 4:

[0202] The user confirms the warning notification and taps the video / audio button on the smart glasses. The input is the warning notification. This action activates the camera and microphone on the smart glasses and starts recording the surrounding video and audio. The output is the video and audio data.

[0203] Step 5:

[0204] The smart glasses' accelerometer constantly detects vibrations. The input is the accelerometer data. If a strong vibration is detected, the smart glasses will turn on the LED light to scare off suspicious people. The output is the LED light being turned on and the glasses will continue recording.

[0205] Step 6:

[0206] If a strong impact is detected again, the smart glasses will automatically send an emergency signal to the server. The input is the impact data from the acceleration sensor. The output is the emergency signal sent to the server.

[0207] Step 7:

[0208] When the server receives an emergency signal, it automatically makes an emergency call according to a pre-defined emergency protocol. The input is the emergency signal and the user's current location information. The call includes the user's current location information and real-time video recording data. The output is a call to 110.

[0209] Step 8:

[0210] The generative AI model sequentially analyzes camera footage and audio and determines whether the user is in a dangerous situation. The input is video and audio data. The output is the analysis result that determines whether the situation is an emergency.

[0211] Step 9:

[0212] The server sends emergency notifications to pre-defined contacts, such as family members and close friends, based on the analysis results of the generative AI model. The inputs are the analysis results, the user's location information, and a link to the video recording data. The output is an emergency notification message sent to the emergency contacts.

[0213] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0214] This invention is a crime prevention system for ensuring personal safety, primarily when out at night, that collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, this system uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[0215] Program processing

[0216] Collection and notification of crime prevention information

[0217] server

[0218] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. This data is collected and machine learning algorithms are used to calculate crime trends and risk levels for each area. Identified risk areas are stored in a database and updated in real time.

[0219] User location monitoring and notification

[0220] server

[0221] The server receives location information periodically sent from the user's smartphone. The user's smartphone identifies its location using GPS and periodically sends the location data to the server. When the user approaches a dangerous area, a warning notification is sent to the smartphone.

[0222] Emotion recognition by emotion engine

[0223] Terminal

[0224] The emotion engine recognizes emotions such as fear and anxiety by monitoring biometric indicators such as the user's heart rate and galvanic skin response. These biometric indicators are acquired through a wearable device and transmitted to the emotion engine.

[0225] Automatic response through emotion recognition

[0226] Terminal

[0227] If the emotion engine detects that the user is feeling fear or anxiety, the following steps are automatically performed:

[0228] 1. Start recording and audio: When the emotion engine recognizes the user's fear or anxiety, the walking recorder will automatically start and record video and audio of the surrounding area.

[0229] 2. Emergency Call: The emotion engine recognizes the user's fear and anxiety and, if it determines that a crisis is occurring, automatically initiates a call to 110. The call will include the user's current location and recorded video.

[0230] 3. Sending emergency notifications: If the emotion engine determines that the user is experiencing fear or anxiety, it will immediately send an emergency notification to pre-defined contacts (e.g., family members). The notification will include the message "The user is in danger," along with location information and a link to the video data.

[0231] Specific examples

[0232] Night Out Scenario

[0233] When a user is walking on a street at night, the server monitors the user's current location and detects that they are approaching a dangerous area, sending a warning notification to their smartphone. The user receives the notification, but begins to feel anxious or scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the walking recorder detects strong vibrations and immediately lights up an LED light. Furthermore, if the user feels extreme fear or is shocked, the emotion engine immediately calls 110 and sends an emergency notification to their family.

[0234] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[0235] The processing flow will be explained below.

[0236] Step 1: Gather crime prevention information

[0237] Subject: Server

[0238] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, and this data is stored in the server's database.

[0239] Step 2: Data analysis

[0240] Subject: Server

[0241] The server analyzes the collected crime and crime prevention information, and uses machine learning algorithms to calculate crime trends and risk levels for each area. The analysis results are stored in a database.

[0242] Step 3: Location Monitoring

[0243] Subject: Server

[0244] The server receives location information periodically sent from the user's smartphone. The smartphone uses GPS to identify the user's location and sends that data to the server.

[0245] Step 4: Risk Area Notification

[0246] Subject: Server

[0247] When the server detects that the user is approaching a high-risk area based on the user's current location and the analysis results, it sends a danger notification to the user's smartphone. The notification includes a message saying, "You are currently approaching a high-risk area."

[0248] Step 5: Emotion Recognition with the Emotion Engine

[0249] Subject: Terminal

[0250] The emotion engine built into the device receives biometric data such as the user's heart rate and skin galvanic response from the wearable device, analyzes this data, and evaluates the user's emotional state (especially fear and anxiety) in real time.

[0251] Step 6: Start recording and recording with emotion recognition

[0252] Subject: Terminal

[0253] When the emotion engine recognizes the user's fear or anxiety, the device automatically starts recording and audio recording. The Walking Recorder's camera and microphone are activated to capture video and audio.

[0254] Step 7: Emotion-Recognition Automatic Reporting

[0255] Subject: Terminal

[0256] If the emotion engine recognizes the user's fear or anxiety, and detects a strong impact or fall, the device will automatically call 110. The call will include the user's current location and recorded video.

[0257] Step 8: Send emergency notifications

[0258] Subject: Terminal

[0259] If the emotion engine detects that the user is in danger, the device will immediately send an emergency notification to pre-defined contacts, containing the message "The user is in danger," along with location information and a link to the video data.

[0260] Specific examples

[0261] Night Out Scenario

[0262] When a user is walking on a street at night, the server monitors the user's current location and sends a warning notification to their smartphone if it detects that they are approaching a dangerous area. The user receives the notification, and the emotion engine begins monitoring their heart rate and skin galvanic response. If the user begins to feel anxious, the emotion engine recognizes this and the walking recorder automatically begins recording and audio recording. If a suspicious person approaches, the device detects vibrations and impacts and automatically reports the incident. An emergency notification is also sent to family members.

[0263] Example 2

[0264] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0265] Personal safety when out at night is an important issue in modern society. To address this issue, a system that can detect danger in real time and respond quickly is needed. However, current security systems do not take the user's emotional state into account, making it difficult to respond appropriately. There is also a need for a system that can automatically respond when the user feels fear or anxiety.

[0266] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting crime information and crime prevention information from all over the country; means for analyzing the collected information and identifying risk areas; means for monitoring the user's location information and notifying the user when the user approaches the risk area; means for collecting the user's biometric indicators and analyzing their emotional state; means for starting video and audio recording when it is determined that the user is feeling fear or anxiety; means for monitoring the biometric indicators in real time and automatically making an emergency call when fear or anxiety is detected; means for detecting vibrations and impacts; means for emitting light when the vibrations are detected; means for analyzing the generated video and audio data; and means for sending an emergency notification based on the analysis results. This enables danger to be detected in real time and appropriate responses taking into account the user's emotional state.

[0267] "Means for collecting crime information and crime prevention information nationwide" refers to the means for collecting data on the number of crimes and crime prevention information obtained from each region of the country and storing it on a server.

[0268] "Means for analyzing and identifying risk areas" refers to means for analyzing collected crime information and crime prevention information and identifying areas where there is a high risk of crime occurring.

[0269] The "means for monitoring the user's location information and notifying when the user approaches the risk area" is a means for monitoring the user's current location using GPS and sending a warning notification when the user approaches an identified risk area.

[0270] "Means for collecting biometric indicators of a user and analyzing their emotional state" refers to means for analyzing a user's emotional state based on data such as the user's heart rate and skin galvanic response obtained from a wearable device or the like.

[0271] The "means for starting video and audio recording when it is determined that the user is feeling fear or anxiety" refers to a means for automatically activating the smartphone's camera and microphone to record video and audio of the surrounding area when the emotion engine determines that the user's emotional state is fear or anxiety.

[0272] The "means for automatically making an emergency call when fear or anxiety is detected" is a means for automatically making an emergency call when fear or anxiety is detected from the biometric indicators of the user.

[0273] The "means for detecting vibration and impact" is a means for detecting vibration or impact when the user receives vibration or impact from the outside.

[0274] The "means for emitting light when vibration is detected" is a means for notifying the surroundings by immediately emitting light based on the detected vibration.

[0275] The "means for analyzing the generated video and audio data" refers to a means for analyzing the video and audio data that has been recorded and detecting abnormalities or emergencies.

[0276] The "means for sending an emergency notification based on the analysis result" is a means for sending an emergency notification to the set contacts when an emergency is recognized based on the analysis result.

[0277] This invention is a crime prevention system for ensuring personal safety when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[0278] Collection and analysis of crime prevention information

[0279] server

[0280] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. For example, it uses government-provided public safety APIs and private crime data providers. The collected data is stored in a database. The databases used include MySQL (registered trademark) and PostgreSQL.

[0281] The server uses machine learning algorithms such as TENSORFLOW (registered trademark) and Scikit-learn to analyze crime trends and risk levels in each area and identify risk areas, which are updated in real time.

[0282] Collection and monitoring of user location information

[0283] Terminal

[0284] The user's smartphone periodically obtains location information using the GPS sensor and the smartphone's standard location services (such as Google Location Services).

[0285] The smartphone periodically sends the acquired location information to the server. The HTTPS protocol is used for transmission, ensuring secure data transmission.

[0286] server

[0287] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area, and maps the user's current location using a map service such as Google Maps API.

[0288] If the user is determined to be approaching a dangerous area, a warning notification is sent to the smartphone. The notification is sent using a notification service such as Firebase Cloud Messaging (FCM).

[0289] Biometric data collection for emotion recognition

[0290] Terminal

[0291] Users wear a wearable device such as an Apple Watch or Fitbit to periodically measure vital signs such as heart rate and galvanic skin response, and this data is sent to a smartphone via Bluetooth.

[0292] The smartphone transmits the received biometric data to the emotion engine.

[0293] Automatic response through emotion recognition

[0294] Terminal

[0295] The emotion engine uses libraries such as OpenCV and DeepFace to analyze biometric data and determine whether the user is feeling fear or anxiety.

[0296] 1. If the user's state is judged to be fearful or anxious, the smartphone's camera and microphone will automatically activate and record video and audio of the surrounding area.

[0297] 2. Additionally, the emotion engine uses the Twilio API to automatically place emergency calls, including current location information and recorded video.

[0298] 3. It also uses SendGrid or Firebase Cloud Messaging (FCM) to send emergency notifications to configured contacts, including a message saying "The user is in danger," along with location information and a link to video data.

[0299] Specific examples of operation

[0300] When a user is walking down a street at night, the server monitors the user's current location and sends a warning notification to the smartphone if it detects that the user is approaching a dangerous area. The user receives the notification, but begins to feel anxious and scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the smartphone's camera will record video and the microphone will record audio. The emotion engine also analyzes the user's condition and, if it determines that the situation is critical, will immediately call emergency services and send an emergency notification to family members.

[0301] Prompt Sentence Examples

[0302] "Please explain the process of a security system that sends a warning notification to the user's smartphone when the user approaches a dangerous area while out at night, and automatically starts video and audio recording if the emotion engine recognizes fear or anxiety."

[0303] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[0304] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0305] Step 1:

[0306] Collection of crime prevention information

[0307] The server periodically obtains nationwide crime and crime prevention information via API. As input, it receives data from APIs provided by police stations and public institutions. The server analyzes this data and stores it in a database. Specifically, it uses public safety APIs provided by the government and private crime data providers. The output is data in a format that stores the number of crimes and crime prevention information for each region in a database.

[0308] Step 2:

[0309] Crime information analysis

[0310] The server analyzes the stored crime information using machine learning algorithms. It uses the crime information stored in the database as input. It calculates crime trends and risk levels for each area using TensorFlow, Scikit-learn, etc. The output is data showing the crime risk level for each area. The analysis results are then stored back in the database, and risk areas are identified.

[0311] Step 3:

[0312] Collection of user location information

[0313] The user's smartphone periodically acquires location information using a GPS sensor. The input is GPS data, which is acquired using the smartphone's location service (e.g., Google Location Services). The acquired location information is sent to the server using the HTTPS protocol. The output is the user's current location data sent to the server.

[0314] Step 4:

[0315] Location monitoring and notifications

[0316] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area. The input is the user's current location data, which is mapped on a map using Google Maps API. If the server determines that the user is approaching a dangerous area, it sends a warning notification to the smartphone. The output is the warning notification sent to the smartphone.

[0317] Step 5:

[0318] Biometric data collection

[0319] A user wears a wearable device and periodically measures their heart rate and electrodermal response. The input is the biometric data measured by the wearable device. For example, a device such as an Apple Watch or Fitbit collects this data and transmits it to a smartphone via Bluetooth. The output is the biometric data transmitted to the smartphone.

[0320] Step 6:

[0321] Emotional state analysis

[0322] The smartphone sends the received biometric data to the emotion engine. The input is the biometric data sent from the wearable device. The emotion engine analyzes this data using libraries such as OpenCV and DeepFace to determine whether the user is feeling fear or anxiety. The output is the user's emotional state data as a result of the analysis.

[0323] Step 7:

[0324] Start of video and audio recording

[0325] When the emotion engine recognizes the user's fear or anxiety, it automatically activates the smartphone's camera and microphone. The input is emotional state data, and it is determined that the user is feeling fear or anxiety. Video and audio recording is started, and surrounding video and audio are recorded. The output is the video and audio data.

[0326] Step 8:

[0327] Handling emergency calls

[0328] If the emotion engine recognizes the user's fear or anxiety and determines that the situation is critical, the server automatically makes an emergency call. The input is recorded video and audio data, as well as current location information. Using the Twilio API, this data is included in the call content. The output is data indicating that an emergency call has been made.

[0329] Step 9:

[0330] Sending emergency notifications

[0331] The server sends an emergency notification based on the analysis results. The input is the emotional state data as the analysis result, the video and audio data recorded, and the current location information. The emergency notification is sent to the specified contacts (e.g., family members) using SendGrid or Firebase Cloud Messaging (FCM). The output is the emergency notification sent to the specified contacts.

[0332] (Application example 2)

[0333] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0334] Conventional security systems only have the function of notifying users when they approach a dangerous area, but do not take into account the user's emotional state or biometric data to respond immediately. This can result in a delay in appropriate response when the user senses danger. In addition, video and audio recording must be started manually, which can lead to situations where the user cannot operate the system in an emergency, making effective security measures ineffective.

[0335] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0336] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, means for monitoring the user's location information and notifying the user when the user approaches the risk area, means for recognizing the user's emotions by monitoring the user's heart rate and galvanic skin response, means for automatically starting video and audio recording when fear or anxiety is recognized, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results. This allows the user's emotional state to be reflected in real time, enabling quick and accurate crime prevention measures.

[0337] "Crime information" refers to the number of crimes occurring across the country and detailed information about crimes, and is data provided by police stations and public institutions.

[0338] "Crime prevention information" is data that includes information on crime prevention measures and warning information about areas where danger is expected, and is information to ensure safety and security.

[0339] "Risk areas" refer to areas where crimes are frequent or where caution is required for crime prevention, and are identified based on collected crime and crime prevention information.

[0340] "Location Information" means data that indicates a user's current geographic location, obtained using GPS or other location-determining technology.

[0341] An "emotion engine" is software or algorithm that analyzes biometric indicators such as a user's heart rate and galvanic skin response to recognize the user's emotional state.

[0342] "Video and audio recording" refers to the function of recording video and audio using a camera or microphone, and is used to generate evidence for crime prevention purposes.

[0343] "Vibration" is a physical movement caused by a device being shaken or struck by a user, and is detected by a sensor.

[0344] "Impact" is a physical action caused by the application of sudden force such as strong vibration or impact, and is detected by a sensor.

[0345] "Emergency Notification" is a feature that instantly notifies appropriate authorities, such as police, of a crisis situation and is performed automatically when certain events occur.

[0346] "Generated video and audio data" refers to video and audio data recorded by the video and audio recording functions.

[0347] "Emergency Notification" is a feature that allows you to quickly notify designated contacts (e.g., family and friends) if you are in a dangerous situation.

[0348] This invention is implemented as a security application called "Safe Night Guardian" that is installed on smartphones. This system ensures the safety of users who go out at night or other unsafe times. It collects and analyzes crime and crime prevention information from all over the country in real time, and links the user's current location and emotional state to enable quick response.

[0349] Hardware used

[0350] 1. Smartphones: Uses location information (GPS), camera and microphone. Examples: Apple iPhone® and Samsung Galaxy.

[0351] 2. Wearable devices: monitor heart rate and galvanic skin response. Examples: Fitbit, Apple Watch.

[0352] Software used

[0353] 1. Server side:

[0354] AWS EC2: Cloud server for running programs.

[0355] AWS RDS: Database management for crime and crime prevention information.

[0356] Python: Data processing and machine learning algorithm implementation.

[0357] Scikit-Learn: A machine learning library for analyzing crime trends.

[0358] 2. Client side:

[0359] Android / iOS app: User interface and location data collection.

[0360] TensorFlow: A machine learning platform for building emotion engines.

[0361] Firebase Cloud Messaging: Real-time notifications to users.

[0362] 3. Notification system:

[0363] Twilio API: For emergency calling and notification.

[0364] System operation explanation

[0365] Collection and analysis of crime prevention information

[0366] The server periodically receives crime and crime prevention information from police and public institutions across the country via API. Based on this data, machine learning algorithms (using Python and Scikit-Learn) are used to calculate crime trends and risk levels for each region. This information is stored in AWS RDS and updated in real time.

[0367] User location monitoring and notification

[0368] The smartphone's GPS function is used to obtain the user's location information, which is then periodically sent to the server. The server compares this location information with information about dangerous areas stored in a database, and sends a warning notification to the smartphone if the user approaches a dangerous area (using Firebase Cloud Messaging).

[0369] Emotion recognition by emotion engine

[0370] Biometric data such as heart rate and skin galvanic response is collected from wearable devices (e.g., Fitbit, Apple Watch), and analyzed by an emotion engine using TensorFlow. The emotion engine recognizes the user's state of fear or anxiety in real time and takes action, such as automatically starting video or audio recording, as needed.

[0371] Auto-response function

[0372] If the emotion engine detects that the user is feeling fear or anxiety, it will automatically:

[0373] 1. Automatically start recording and recording: Record the situation using your smartphone's camera and microphone.

[0374] 2. Emergency call: A call to 110 is made using the Twilio API, and the user's current location and recorded video are provided.

[0375] 3. Emergency notification: Notify pre-defined contacts (e.g., family members) that the user is in danger and send them location information and a link to video data.

[0376] Specific examples

[0377] When a user is walking down a street at night, the app monitors their location in the background and displays a warning if they approach a dangerous area. If the user feels anxious, the smartwatch detects an increase in heart rate and galvanic skin response. The emotion engine analyzes this and, if it recognizes a state of fear, automatically starts video and audio recording and simultaneously calls 110. An emergency notification is also sent to family members.

[0378] Example prompts to input to the generative AI model

[0379] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[0380] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0381] Step 1:

[0382] Collection of crime prevention information

[0383] The server regularly obtains crime and crime prevention information from police and public institutions across the country via API.

[0384] Input: Crime and crime prevention data provided by police departments and public agency APIs.

[0385] Output: The acquired data is saved on the server. The server saves the acquired data in a database (e.g. AWS RDS).

[0386] Step 2:

[0387] Crime information analysis

[0388] The server analyzes the collected crime and crime prevention data using machine learning algorithms (e.g., Python and Scikit-Learn) to calculate crime trends and risk levels for each area.

[0389] Input: Crime and crime prevention data stored in a database.

[0390] Output: Analysis report data showing crime trends and risk levels. The server updates this data and stores it as risk area information.

[0391] Step 3:

[0392] Location acquisition and monitoring

[0393] The device (smartphone) periodically obtains the user's current location using its GPS function and sends it to the server.

[0394] Input: User location data obtained from the smartphone's GPS.

[0395] Output: The user's current location information is sent to the server, which monitors the user's location in real time and prepares a warning notification if the user approaches a dangerous area.

[0396] Step 4:

[0397] Notification when approaching a dangerous area

[0398] The server compares the user's location information with the danger area information, and if the user approaches a danger area, it sends a warning notification to the device (smartphone).

[0399] Input: User's current location information and danger area information.

[0400] Output: A warning notification is sent to your smartphone. Use a notification system (e.g., Firebase Cloud Messaging) to display real-time warnings.

[0401] Step 5:

[0402] Biometric data acquisition and emotion recognition

[0403] A wearable device (e.g., Fitbit, Apple Watch) worn by the user captures heart rate and electrodermal response data and sends them to the emotion engine, which uses TensorFlow to analyze this data and recognize the user's emotional state (e.g., fear, anxiety).

[0404] Input: Biometric data such as heart rate and galvanic skin response obtained from wearable devices.

[0405] Output: The user's emotional state (e.g., fear, anxiety) as a result of the emotion engine's analysis. If the emotion engine detects fear or anxiety, it prepares the next step.

[0406] Step 6:

[0407] Automatic start of video and audio recording

[0408] When fear or anxiety is detected from the user's biometric data, the device (smartphone) automatically starts recording and recording audio using its camera and microphone.

[0409] Input: Signal from emotion engine (user's emotional state).

[0410] Output: Start recording. The device will record video and audio and save the data.

[0411] Step 7:

[0412] Making an emergency call

[0413] If the device's emotion engine detects fear or anxiety, it will automatically call 110 using an emergency notification system (e.g., Twilio API) and provide current location information and recorded video.

[0414] Input: Emotion engine signals, video and audio data, and current location information.

[0415] Output: Report, current location and recorded video sent to police. The device will automatically make an emergency call.

[0416] Step 8:

[0417] Sending emergency notifications

[0418] If the emotion engine detects fear or anxiety, the device will send an emergency notification to pre-defined contacts (e.g., family members) informing them that they are in danger, along with their current location and a link to the recorded video.

[0419] Input: Emotion engine signal, current location information, and recorded video link.

[0420] Output: Emergency notification sent to configured contacts. Notifications are sent to family members using Firebase Cloud Messaging.

[0421] Example prompts to input to the generative AI model

[0422] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[0423] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0424] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0425] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0426] [Second embodiment]

[0427] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0428] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0429] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0430] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0431] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0433] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0434] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0435] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0436] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0437] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0438] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0439] This invention is a security system for ensuring personal safety, primarily when out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it automatically starts recording and recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[0440] Program processing

[0441] Collection and notification of crime prevention information

[0442] server

[0443] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. This includes programs that retrieve data via APIs. The server analyzes this data and runs algorithms to identify crime trends and high-risk areas. The identified risk areas are stored in a database that is continually updated.

[0444] User location monitoring and notification

[0445] server

[0446] The server receives location information periodically sent from the user's smartphone. The server-side program monitors this location information in real time, and if it determines that the user's current location is approaching a risky area, it immediately sends a warning notification to the user's smartphone. The notification includes the message, "You are currently approaching a high-risk area."

[0447] Start of video and audio recording

[0448] User

[0449] The user checks the warning notification on their smartphone and taps the recording button on the smartphone app. This activates the Walking Recorder's high-resolution camera and microphone, recording video and audio of the surrounding area. The recorded data is saved to the device's internal storage or a cloud service.

[0450] Vibration and shock detection

[0451] Terminal

[0452] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will immediately flash at maximum brightness to scare off suspicious individuals. At the same time, if a strong impact is detected again, an emergency signal will be automatically sent to your smartphone.

[0453] Automatic reporting

[0454] Terminal

[0455] When a smartphone receives an emergency signal, it automatically calls 110 according to pre-defined emergency protocols, and the call includes the user's current location and real-time video recording.

[0456] emergency notification

[0457] Terminal

[0458] The AI ​​analyzes the camera footage and audio sequentially, and if it determines that the user is in danger, it sends an emergency notification to pre-defined contacts such as family and close friends. The notification includes a message saying "The user is currently in danger," along with location information and a link to the video data. This enables rapid assistance in an emergency.

[0459] Specific examples

[0460] Night Out Scenario

[0461] When a user is walking on a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone. The user receives the notification and starts recording on the Walking Recorder. After a while, a suspicious person approaches from behind and grabs the user's shoulder. The Walking Recorder detects a strong vibration and immediately lights up its LED light. At the same time, recording continues, and if the suspicious person delivers a stronger shock, the Walking Recorder automatically reports the incident and sends the video data to the police. The generating AI then analyzes the situation and sends an emergency notification to the user's family.

[0462] This system not only allows users to go out safely at night, but also enables them to respond quickly in the event of an unforeseen incident.

[0463] The processing flow will be explained below.

[0464] Step 1: Gather crime prevention information

[0465] Subject: Server

[0466] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, specifically collecting data from government crime databases and public data from local police stations.

[0467] Step 2: Data analysis

[0468] Subject: Server

[0469] The server analyzes the collected crime and crime prevention information, using machine learning algorithms to calculate crime trends and risk levels for each area, allowing risk areas to be identified.

[0470] Step 3: Location Monitoring

[0471] Subject: Server

[0472] The server receives current location information periodically sent from the user's smartphone. The user's smartphone uses GPS to identify its location and periodically sends location data to the server.

[0473] Step 4: Risk Area Notification

[0474] Subject: Server

[0475] If the server determines that the user's current location is approaching a high-risk area, it sends a warning notification to the user's smartphone. This notification includes the message, "You are currently approaching a high-risk area."

[0476] Step 5: Start recording

[0477] Subject: User

[0478] The user checks the warning notification on their smartphone, opens the smartphone app, and taps the video and audio recording button, which activates the walking recorder and starts recording video and audio of the surrounding area.

[0479] Step 6: Vibration and shock sensing

[0480] Subject: Terminal

[0481] The Walking Recorder uses a built-in acceleration sensor to constantly detect vibrations. If a strong vibration is detected, the LED light will immediately flash at maximum brightness. If a stronger impact is subsequently detected, an emergency signal will be sent to your smartphone.

[0482] Step 7: Automatic reporting

[0483] Subject: Terminal

[0484] When a smartphone receives an emergency signal, it automatically initiates the 110 emergency call process, which includes the user's current location and recorded video data, allowing the police to immediately be notified of the situation at the scene.

[0485] Step 8: Emergency Notification

[0486] Subject: Terminal

[0487] The AI ​​analyzes the camera footage and audio in real time, and if it determines that the user is in danger, an emergency notification is sent to pre-defined contacts (such as family members). This notification includes a message saying "The user is currently in danger," along with location information and a link to the video data.

[0488] This series of processes improves the safety of the user and allows for a quick response in dangerous situations.

[0489] Example 1

[0490] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0491] In modern society, crime prevention measures are extremely important when people are out at night or alone. However, current crime prevention systems have difficulty notifying users of crime information and dangerous areas in real time, and they have limited means of taking prompt and appropriate action in emergencies. For these reasons, a system that can ensure even greater safety is needed.

[0492] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0493] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, and means for monitoring the user's location information and notifying the user when the user approaches the risk area. This allows the user to receive real-time notifications when the user approaches a risk area. The server also includes means for starting video and audio recording in response to a user's instruction, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results, thereby enabling the user to respond quickly and appropriately when they encounter an unexpected situation. The system includes a means for monitoring a user's location information in real time and detecting approach to a dangerous area, a means for sending a warning notification as a push notification, a means for detecting vibrations using an acceleration sensor in the security system and controlling an LED light when the vibration exceeds a threshold, a means for storing the generated video data in a cloud service, a means for sending an emergency signal to a smart device using Bluetooth or Wi-Fi Direct, a means for using a telephone API to automatically make an emergency call, and a means for sending an emergency notification using SMS or a notification API. This allows users to act safely even when out at night or in isolated situations, and enables quick response in the event of an emergency.

[0494] "Means of collecting crime information and crime prevention information from all over the country" refers to the function of regularly obtaining crime information and crime prevention information provided by police stations and public institutions from all over the country and storing it in a database.

[0495] "Means for analyzing the collected information and identifying risk areas" refers to the function of analyzing collected crime information and crime prevention information and identifying crime trends and high-risk areas.

[0496] "Means for monitoring the user's location information and notifying when the user approaches the dangerous area" refers to a function that monitors the location information sent from the user's smart device and sends a warning notification when the user approaches the dangerous area.

[0497] The term "means for starting video and audio recording in response to a user's instruction" refers to a function that allows a user to start video and audio recording by operating an application.

[0498] "Means for detecting vibrations and impacts" refers to the function of detecting vibrations and impacts using an acceleration sensor built into the security device.

[0499] "Means for emitting light when vibration is detected" refers to the function of illuminating the security device's LED light at maximum brightness when vibration is detected by the acceleration sensor.

[0500] "Means for automatically making an emergency call when an impact is detected" refers to a function that automatically makes an emergency call when a strong impact is detected by the acceleration sensor.

[0501] "Means for analyzing generated video and audio data" refers to the function of analyzing video and audio data recorded by a user's smart device.

[0502] "Means for sending an emergency notification based on the analysis results" refers to the function in which the generative AI model analyzes video and audio data and sends a notification if it determines that an emergency exists.

[0503] "Means for monitoring the user's location information in real time and detecting when the user approaches a dangerous area" refers to a function for monitoring the user's location information in real time and detecting when the user approaches a dangerous area.

[0504] "Means for sending a warning notification as a push notification" refers to a function that sends a warning using a push notification when a user approaches a dangerous area.

[0505] "Means for detecting vibrations using an acceleration sensor in a security system and controlling an LED light when the vibrations exceed a threshold" refers to a function that detects vibrations using an acceleration sensor and controls an LED light to turn on when the vibrations exceed a certain threshold.

[0506] "Means for storing generated video data in a cloud service" refers to a function for storing video data recorded on a user's device in a cloud service.

[0507] "Means for transmitting emergency signals to a smart device using Bluetooth or Wi-Fi Direct" refers to the function of transmitting emergency signals to a user's smart device using Bluetooth or Wi-Fi Direct.

[0508] "Means of using a telephone API to automatically make an emergency call" refers to a function that automatically makes an emergency call using a telephone API when an emergency signal is received.

[0509] "Means of sending emergency notifications using SMS or notification APIs" refers to the ability to send emergency notifications to pre-defined contacts using SMS or notification APIs.

[0510] This invention is a security system for ensuring personal safety, primarily when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. It also automatically starts recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[0511] Server Operation

[0512] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. The collection process uses an API, including a program that sends HTTP requests to retrieve data in JSON format and stores it in a database. The server analyzes the collected data and runs machine learning algorithms to identify crime trends and high-risk areas. The results are stored in a database, providing constantly up-to-date information.

[0513] The server also receives location information periodically sent from the user's smartphone and monitors it in real time. When the user approaches a risky area, the server immediately generates a warning notification and sends it to the user's smartphone using a notification service such as Firebase Cloud Messaging (FCM). The specific notification message may say, "You are currently approaching a high-risk area."

[0514] User smartphone behavior

[0515] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. This action calls the smartphone's camera API and microphone API, which starts recording the surrounding video and audio. The recorded data is saved in the device's internal storage or on a cloud service.

[0516] Walking Recorder Operation

[0517] The Walking Recorder constantly detects vibrations using a built-in accelerometer. If a strong vibration is detected, the system immediately turns on the LED light at maximum brightness. This is achieved by acquiring vibration data using the sensor API and calling the LED light control API when a threshold is exceeded. If an even stronger impact is detected, the Walking Recorder will send an emergency signal to your smartphone. This is done using Bluetooth or Wi-Fi Direct.

[0518] Handling emergency calls and notifications

[0519] When a smartphone receives an emergency signal, it automatically makes an emergency call using a phone API according to pre-defined emergency protocols. This call includes the user's current location and recorded video. If the generative AI model analyzes the video and audio and determines that the user is in danger, it sends a notification to pre-defined emergency contacts via SMS or a notification API. The notification includes the user's current location and a link to the video data.

[0520] Specific examples

[0521] For example, if a user is walking down a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone saying, "You are approaching a high-risk area." The user confirms the notification and taps the recording button to begin recording. If a suspicious person approaches and grabs the user's shoulder, the Walking Recorder detects the strong vibration and immediately illuminates its LED light. If an even stronger impact is applied, the Walking Recorder sends an emergency signal to the smartphone, which then automatically reports the incident and sends the video data and current location to the police. The generative AI model then analyzes the situation and sends an emergency notification to the user's family.

[0522] Prompt Sentence Examples

[0523] Could you please explain the details of the nighttime security system? Location monitoring, danger zone notification, sudden vibration and shock detection, video and audio recording start, automatic reporting, and emergency notification.

[0524] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0525] Step 1:

[0526] server

[0527] The server collects crime and crime prevention information provided by police stations and public institutions across the country. Specifically, the server periodically sends HTTP requests to the API endpoint and retrieves data in JSON format. This data includes information such as the date, time, location, and type of crime. The collected data is stored in a database and used for subsequent analysis. (Input): Response data from the API. (Output): Crime information stored in the database.

[0528] Step 2:

[0529] server

[0530] The server analyzes the collected crime information using a machine learning algorithm to identify high-risk areas. Specifically, it applies a clustering algorithm (e.g., K-means algorithm) based on past crime data to identify areas where crimes are concentrated. The results of this analysis are stored in a database and used to match users' location information. (Input): Crime information stored in the database. (Output): High-risk areas identified through analysis.

[0531] Step 3:

[0532] server

[0533] The server receives real-time location information periodically sent from the user's smartphone. The received location information is sent to the server as GPS data, and monitoring is performed based on that data. If the user's current location approaches a high-risk area, the server generates a warning notification. (Input): GPS data from the smartphone. (Output): Generated warning notification.

[0534] Step 4:

[0535] server

[0536] The server sends the generated warning notification to the user's smartphone. Specifically, it sends a push notification using a notification service such as Firebase Cloud Messaging (FCM). This notification contains the message "You are currently approaching a high-risk area." (Input): Warning notification generated by the server. (Output): Push notification sent to the user's smartphone.

[0537] Step 5:

[0538] User

[0539] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. The smartphone app calls the camera API and microphone API to begin recording video and audio. This recorded data is saved in the internal storage or on a cloud service. (Input): User's tap operation. (Output): Video and audio data saved on the smartphone.

[0540] Step 6:

[0541] Terminal

[0542] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will turn on at maximum brightness to scare off suspicious people. The sensor API is used to obtain vibration data, and if the vibration exceeds a threshold, the LED control API is called to turn on the light. (Input): Vibration data from the acceleration sensor. (Output): The lit LED light.

[0543] Step 7:

[0544] Terminal

[0545] If the Walking Recorder detects another strong impact, it immediately sends an emergency signal to your smartphone. The emergency signal is transmitted using Bluetooth or Wi-Fi Direct. This signal contains information about the strength of the impact and your location. (Input): Strong impact data from the acceleration sensor. (Output): Emergency signal sent to your smartphone.

[0546] Step 8:

[0547] Smartphone

[0548] When the smartphone receives an emergency signal, it automatically calls 110 using the phone API according to the pre-set emergency protocol. The call includes the user's current location and real-time video recording. (Input): Emergency signal from the walking recorder. (Output): Current location and video data sent along with the automatic call.

[0549] Step 9:

[0550] Generative AI Models

[0551] The generating AI sequentially analyzes camera footage and audio, and if it determines that the user is in danger, it sends an emergency notification to pre-set contacts such as family and close friends. Specifically, it uses a notification API and SMS service to send a message saying "The user is currently in danger," along with location information and a link to the video data. (Input): Camera footage and audio data. (Output): Emergency notification sent to family and pre-set contacts.

[0552] (Application example 1)

[0553] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0554] Ensuring personal safety when out at night is extremely important, but conventional security systems often lack sufficient risk area notification and emergency response capabilities, requiring users to operate the system directly. Furthermore, users lack a means to receive visual information, making it difficult to quickly recognize danger. Furthermore, automation of emergency calls and video and audio recording is insufficient, potentially preventing effective action even in situations requiring a rapid response.

[0555] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0556] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country and monitoring user location information, means for analyzing the collected information to identify risk areas, means for notifying the user when they approach a risk area, and means for starting video and audio recording in response to a user's instruction.In this invention, by providing an easy-to-use device and interface, it is possible to quickly and effectively recognize danger and take appropriate action, by detecting vibrations and impacts using a built-in sensor and emitting light to automatically start video and audio recording, means for analyzing the generated video and audio data and sending an emergency notification based on the analysis results, means for providing a visual display device with a notification function to warn the user in real time, and means for implementing an interface function for the user to start video and audio recording.

[0557] "Crime information and crime prevention information" refers to data on crime occurrence and prevention measures provided by police stations and public institutions across the country.

[0558] "Risk areas" refer to areas that have been identified as having a high risk of crime occurring based on the results of an analysis of collected crime and crime prevention information.

[0559] "Location Information" means data that indicates a user's current geographic location, obtained through GPS or other location-determining technology.

[0560] "Notification" refers to the action of displaying or sending a warning or information to a user's device when the user approaches a dangerous area.

[0561] "Video and audio recording" refers to the act of activating the camera and microphone, either at the user's command or automatically, to record video and audio of the surrounding area.

[0562] "Vibration and shock" refers to physical shaking or impact detected by an accelerometer.

[0563] "Emitting light" refers to the act of turning on an LED light or similar as a visual warning when vibration is detected.

[0564] "Emergency call" refers to the action of automatically notifying pre-defined contacts or the police to request help.

[0565] "Analyzing the generated video and audio data" refers to the process of analyzing video and audio data using AI models, etc., to determine dangerous situations.

[0566] "Send emergency notification" refers to the action of notifying set contacts, such as family and friends, of the user's situation based on the analysis results.

[0567] "Visual display device" refers to a device (e.g., smart glasses) worn by a user that displays information in real time.

[0568] The "interface function" refers to a function that provides an operation method for the user to start video and audio recording.

[0569] This invention is a security system to ensure the safety of users when they are out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies them when they approach a dangerous area. It also provides a series of processes that automatically start recording and recording when it detects vibration or impact, and automatically reports and notifies users in the event of an emergency.

[0570] System configuration

[0571] The system consists of the following components:

[0572] 1. Server:

[0573] Collection of crime and crime prevention information: Data provided by police stations and public institutions across the country is periodically obtained via API. The server analyzes this data to identify crime trends and high-risk areas.

[0574] Monitoring user location information: The server periodically receives location information from the user's smart device and sends a warning notification to the user if the device approaches a dangerous area.

[0575] 2. User terminal (smart glasses):

[0576] Video and audio recording function: When the user confirms the alert notification, the smart glasses' built-in camera and microphone will activate and record the surrounding video and audio. The recorded data will be saved in the internal storage or cloud service.

[0577] Vibration and impact detection: The smart glasses' built-in acceleration sensor constantly detects vibrations, and when a strong vibration is detected, the LED light will turn on. At the same time, if a strong impact is detected, an emergency signal will be automatically sent to the server.

[0578] Automatic notification function: When the smart glasses receive an emergency signal, they will automatically make an emergency call according to the emergency protocol pre-set on the server side. The call will include the user's current location information and real-time video recording data.

[0579] Emergency Notification: If the generative AI model analyzes camera footage and audio and determines that the user is in danger, it will send an emergency notification to pre-defined contacts, including location information and a link to the video recording.

[0580] Hardware and Software Details

[0581] Hardware:

[0582] Smart Glasses: Uses a display (e.g., smart glasses) that displays information in real time.

[0583] Acceleration sensor: Built-in sensor for detecting vibration and impact.

[0584] High-resolution camera and microphone: Equipped with a device for recording video and audio of the surroundings.

[0585] LED light: Visual warning when vibration is detected.

[0586] software:

[0587] Server: Uses cloud services such as AWS and Google Cloud Platform to collect and analyze data and execute emergency protocols.

[0588] Generative AI model: Analyzes video and audio data using OpenAI GPT-4 and other models to determine emergency situations.

[0589] Notification app: An application for iOS and Android devices that monitors the user's location, controls recording and video recording, and sends emergency notifications.

[0590] Specific examples

[0591] A user puts on smart glasses while walking down a street at night. The server detects that the user is approaching a dangerous area, and a warning message is displayed on the smart glasses. The user confirms the warning and starts recording. If a suspicious person approaches and grabs the user's shoulder, the smart glasses detect a strong vibration, turn on an LED light, and automatically start recording. If the suspicious person causes further harm, the generating AI analyzes the situation and makes an emergency call via the server. In addition, an emergency notification is sent to family and friends.

[0592] Example prompts for generative AI models

[0593] "I would like to develop the following security application. A system in which users wear smart glasses and are warned of approaching dangerous areas when going out at night. The glasses can record video and audio using the camera and microphone, and will turn on an LED light if a suspicious person comes into contact with the user. If vibration or impact is detected, the glasses will automatically report the user and send an emergency notification."

[0594] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0595] Step 1:

[0596] The server collects crime and crime prevention information from all over the country. The input is data provided by police stations and public institutions. This data is obtained via API and stored in a database on the server. The output is a database of the collected crime and crime prevention information.

[0597] Step 2:

[0598] The server analyzes the collected crime and crime prevention information and identifies risk areas. The input is crime information in the database. The server performs data analysis to identify high-risk areas using statistical models and machine learning algorithms. The output is information on areas identified as high-risk areas.

[0599] Step 3:

[0600] The server receives location information periodically sent from the user's smart glasses. The input is the user's location information. The server monitors this information in real time, and if it determines that the user is approaching a dangerous area, it sends a warning notification to the user's smart glasses. The output is a warning message displayed on the user's smart glasses.

[0601] Step 4:

[0602] The user confirms the warning notification and taps the video / audio button on the smart glasses. The input is the warning notification. This action activates the camera and microphone on the smart glasses and starts recording the surrounding video and audio. The output is the video and audio data.

[0603] Step 5:

[0604] The smart glasses' accelerometer constantly detects vibrations. The input is the accelerometer data. If a strong vibration is detected, the smart glasses will turn on the LED light to scare off suspicious people. The output is the LED light being turned on and the glasses will continue recording.

[0605] Step 6:

[0606] If a strong impact is detected again, the smart glasses will automatically send an emergency signal to the server. The input is the impact data from the acceleration sensor. The output is the emergency signal sent to the server.

[0607] Step 7:

[0608] When the server receives an emergency signal, it automatically makes an emergency call according to a pre-defined emergency protocol. The input is the emergency signal and the user's current location information. The call includes the user's current location information and real-time video recording data. The output is a call to 110.

[0609] Step 8:

[0610] The generative AI model sequentially analyzes camera footage and audio and determines whether the user is in a dangerous situation. The input is video and audio data. The output is the analysis result that determines whether the situation is an emergency.

[0611] Step 9:

[0612] The server sends emergency notifications to pre-defined contacts, such as family members and close friends, based on the analysis results of the generative AI model. The inputs are the analysis results, the user's location information, and a link to the video recording data. The output is an emergency notification message sent to the emergency contacts.

[0613] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0614] This invention is a crime prevention system for ensuring personal safety, primarily when out at night, that collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, this system uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[0615] Program processing

[0616] Collection and notification of crime prevention information

[0617] server

[0618] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. This data is collected and machine learning algorithms are used to calculate crime trends and risk levels for each area. Identified risk areas are stored in a database and updated in real time.

[0619] User location monitoring and notification

[0620] server

[0621] The server receives location information periodically sent from the user's smartphone. The user's smartphone identifies its location using GPS and periodically sends the location data to the server. When the user approaches a dangerous area, a warning notification is sent to the smartphone.

[0622] Emotion recognition by emotion engine

[0623] Terminal

[0624] The emotion engine recognizes emotions such as fear and anxiety by monitoring biometric indicators such as the user's heart rate and galvanic skin response. These biometric indicators are acquired through a wearable device and transmitted to the emotion engine.

[0625] Automatic response through emotion recognition

[0626] Terminal

[0627] If the emotion engine detects that the user is feeling fear or anxiety, the following steps are automatically performed:

[0628] 1. Start recording and audio: When the emotion engine recognizes the user's fear or anxiety, the walking recorder will automatically start and record video and audio of the surrounding area.

[0629] 2. Emergency Call: The emotion engine recognizes the user's fear and anxiety and, if it determines that a crisis is occurring, automatically initiates a call to 110. The call will include the user's current location and recorded video.

[0630] 3. Sending emergency notifications: If the emotion engine determines that the user is experiencing fear or anxiety, it will immediately send an emergency notification to pre-defined contacts (e.g., family members). The notification will include the message "The user is in danger," along with location information and a link to the video data.

[0631] Specific examples

[0632] Night Out Scenario

[0633] When a user is walking on a street at night, the server monitors the user's current location and detects that they are approaching a dangerous area, sending a warning notification to their smartphone. The user receives the notification, but begins to feel anxious or scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the walking recorder detects strong vibrations and immediately lights up an LED light. Furthermore, if the user feels extreme fear or is shocked, the emotion engine immediately calls 110 and sends an emergency notification to their family.

[0634] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[0635] The processing flow will be explained below.

[0636] Step 1: Gather crime prevention information

[0637] Subject: Server

[0638] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, and this data is stored in the server's database.

[0639] Step 2: Data analysis

[0640] Subject: Server

[0641] The server analyzes the collected crime and crime prevention information, and uses machine learning algorithms to calculate crime trends and risk levels for each area. The analysis results are stored in a database.

[0642] Step 3: Location Monitoring

[0643] Subject: Server

[0644] The server receives location information periodically sent from the user's smartphone. The smartphone uses GPS to identify the user's location and sends that data to the server.

[0645] Step 4: Risk Area Notification

[0646] Subject: Server

[0647] When the server detects that the user is approaching a high-risk area based on the user's current location and the analysis results, it sends a danger notification to the user's smartphone. The notification includes a message saying, "You are currently approaching a high-risk area."

[0648] Step 5: Emotion Recognition with the Emotion Engine

[0649] Subject: Terminal

[0650] The emotion engine built into the device receives biometric data such as the user's heart rate and skin galvanic response from the wearable device, analyzes this data, and evaluates the user's emotional state (especially fear and anxiety) in real time.

[0651] Step 6: Start recording and recording with emotion recognition

[0652] Subject: Terminal

[0653] When the emotion engine recognizes the user's fear or anxiety, the device automatically starts recording and audio recording. The Walking Recorder's camera and microphone are activated to capture video and audio.

[0654] Step 7: Emotion-Recognition Automatic Reporting

[0655] Subject: Terminal

[0656] If the emotion engine recognizes the user's fear or anxiety, and detects a strong impact or fall, the device will automatically call 110. The call will include the user's current location and recorded video.

[0657] Step 8: Send emergency notifications

[0658] Subject: Terminal

[0659] If the emotion engine detects that the user is in danger, the device will immediately send an emergency notification to pre-defined contacts, containing the message "The user is in danger," along with location information and a link to the video data.

[0660] Specific examples

[0661] Night Out Scenario

[0662] When a user is walking on a street at night, the server monitors the user's current location and sends a warning notification to their smartphone if it detects that they are approaching a dangerous area. The user receives the notification, and the emotion engine begins monitoring their heart rate and skin galvanic response. If the user begins to feel anxious, the emotion engine recognizes this and the walking recorder automatically begins recording and audio recording. If a suspicious person approaches, the device detects vibrations and impacts and automatically reports the incident. An emergency notification is also sent to family members.

[0663] Example 2

[0664] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0665] Personal safety when out at night is an important issue in modern society. To address this issue, a system that can detect danger in real time and respond quickly is needed. However, current security systems do not take the user's emotional state into account, making it difficult to respond appropriately. There is also a need for a system that can automatically respond when the user feels fear or anxiety.

[0666] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting crime information and crime prevention information from all over the country; means for analyzing the collected information and identifying risk areas; means for monitoring the user's location information and notifying the user when the user approaches the risk area; means for collecting the user's biometric indicators and analyzing their emotional state; means for starting video and audio recording when it is determined that the user is feeling fear or anxiety; means for monitoring the biometric indicators in real time and automatically making an emergency call when fear or anxiety is detected; means for detecting vibrations and impacts; means for emitting light when the vibrations are detected; means for analyzing the generated video and audio data; and means for sending an emergency notification based on the analysis results. This enables danger to be detected in real time and appropriate responses taking into account the user's emotional state.

[0667] "Means for collecting crime information and crime prevention information nationwide" refers to the means for collecting data on the number of crimes and crime prevention information obtained from each region of the country and storing it on a server.

[0668] "Means for analyzing and identifying risk areas" refers to means for analyzing collected crime information and crime prevention information and identifying areas where there is a high risk of crime occurring.

[0669] The "means for monitoring the user's location information and notifying when the user approaches the risk area" is a means for monitoring the user's current location using GPS and sending a warning notification when the user approaches an identified risk area.

[0670] "Means for collecting biometric indicators of a user and analyzing their emotional state" refers to means for analyzing a user's emotional state based on data such as the user's heart rate and skin galvanic response obtained from a wearable device or the like.

[0671] The "means for starting video and audio recording when it is determined that the user is feeling fear or anxiety" refers to a means for automatically activating the smartphone's camera and microphone to record video and audio of the surrounding area when the emotion engine determines that the user's emotional state is fear or anxiety.

[0672] The "means for automatically making an emergency call when fear or anxiety is detected" is a means for automatically making an emergency call when fear or anxiety is detected from the biometric indicators of the user.

[0673] The "means for detecting vibration and impact" is a means for detecting vibration or impact when the user receives vibration or impact from the outside.

[0674] The "means for emitting light when vibration is detected" is a means for notifying the surroundings by immediately emitting light based on the detected vibration.

[0675] The "means for analyzing the generated video and audio data" refers to a means for analyzing the video and audio data that has been recorded and detecting abnormalities or emergencies.

[0676] The "means for sending an emergency notification based on the analysis result" is a means for sending an emergency notification to the set contacts when an emergency is recognized based on the analysis result.

[0677] This invention is a crime prevention system for ensuring personal safety when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[0678] Collection and analysis of crime prevention information

[0679] server

[0680] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. For example, it uses government-provided public safety APIs and private crime data providers. The collected data is stored in a database, typically MySQL or PostgreSQL.

[0681] The server uses machine learning algorithms such as TensorFlow and Scikit-learn to analyze crime trends and risk levels in each area and identify risk areas, which are updated in real time.

[0682] Collection and monitoring of user location information

[0683] Terminal

[0684] The user's smartphone periodically obtains location information using the GPS sensor and the smartphone's standard location services (such as Google Location Services).

[0685] The smartphone periodically sends the acquired location information to the server. The HTTPS protocol is used for transmission, ensuring secure data transmission.

[0686] server

[0687] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area, and maps the user's current location using a map service such as Google Maps API.

[0688] If the user is determined to be approaching a dangerous area, a warning notification is sent to the smartphone. The notification is sent using a notification service such as Firebase Cloud Messaging (FCM).

[0689] Biometric data collection for emotion recognition

[0690] Terminal

[0691] Users wear a wearable device such as an Apple Watch or Fitbit to periodically measure vital signs such as heart rate and galvanic skin response, and this data is sent to a smartphone via Bluetooth.

[0692] The smartphone transmits the received biometric data to the emotion engine.

[0693] Automatic response through emotion recognition

[0694] Terminal

[0695] The emotion engine uses libraries such as OpenCV and DeepFace to analyze biometric data and determine whether the user is feeling fear or anxiety.

[0696] 1. If the user's state is judged to be fearful or anxious, the smartphone's camera and microphone will automatically activate and record video and audio of the surrounding area.

[0697] 2. Additionally, the emotion engine uses the Twilio API to automatically place emergency calls, including current location information and recorded video.

[0698] 3. It also uses SendGrid or Firebase Cloud Messaging (FCM) to send emergency notifications to configured contacts, including a message saying "The user is in danger," along with location information and a link to video data.

[0699] Specific examples of operation

[0700] When a user is walking down a street at night, the server monitors the user's current location and sends a warning notification to the smartphone if it detects that the user is approaching a dangerous area. The user receives the notification, but begins to feel anxious and scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the smartphone's camera will record video and the microphone will record audio. The emotion engine also analyzes the user's condition and, if it determines that the situation is critical, will immediately call emergency services and send an emergency notification to family members.

[0701] Prompt Sentence Examples

[0702] "Please explain the process of a security system that sends a warning notification to the user's smartphone when the user approaches a dangerous area while out at night, and automatically starts video and audio recording if the emotion engine recognizes fear or anxiety."

[0703] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[0704] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0705] Step 1:

[0706] Collection of crime prevention information

[0707] The server periodically obtains nationwide crime and crime prevention information via API. As input, it receives data from APIs provided by police stations and public institutions. The server analyzes this data and stores it in a database. Specifically, it uses public safety APIs provided by the government and private crime data providers. The output is data in a format that stores the number of crimes and crime prevention information for each region in a database.

[0708] Step 2:

[0709] Crime information analysis

[0710] The server analyzes the stored crime information using machine learning algorithms. It uses the crime information stored in the database as input. It calculates crime trends and risk levels for each area using TensorFlow, Scikit-learn, etc. The output is data showing the crime risk level for each area. The analysis results are then stored back in the database, and risk areas are identified.

[0711] Step 3:

[0712] Collection of user location information

[0713] The user's smartphone periodically acquires location information using a GPS sensor. The input is GPS data, which is acquired using the smartphone's location service (e.g., Google Location Services). The acquired location information is sent to the server using the HTTPS protocol. The output is the user's current location data sent to the server.

[0714] Step 4:

[0715] Location monitoring and notifications

[0716] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area. The input is the user's current location data, which is mapped on a map using Google Maps API. If the server determines that the user is approaching a dangerous area, it sends a warning notification to the smartphone. The output is the warning notification sent to the smartphone.

[0717] Step 5:

[0718] Biometric data collection

[0719] A user wears a wearable device and periodically measures their heart rate and electrodermal response. The input is the biometric data measured by the wearable device. For example, a device such as an Apple Watch or Fitbit collects this data and transmits it to a smartphone via Bluetooth. The output is the biometric data transmitted to the smartphone.

[0720] Step 6:

[0721] Emotional state analysis

[0722] The smartphone sends the received biometric data to the emotion engine. The input is the biometric data sent from the wearable device. The emotion engine analyzes this data using libraries such as OpenCV and DeepFace to determine whether the user is feeling fear or anxiety. The output is the user's emotional state data as a result of the analysis.

[0723] Step 7:

[0724] Start of video and audio recording

[0725] When the emotion engine recognizes the user's fear or anxiety, it automatically activates the smartphone's camera and microphone. The input is emotional state data, and it is determined that the user is feeling fear or anxiety. Video and audio recording is started, and surrounding video and audio are recorded. The output is the video and audio data.

[0726] Step 8:

[0727] Handling emergency calls

[0728] If the emotion engine recognizes the user's fear or anxiety and determines that the situation is critical, the server automatically makes an emergency call. The input is recorded video and audio data, as well as current location information. Using the Twilio API, this data is included in the call content. The output is data indicating that an emergency call has been made.

[0729] Step 9:

[0730] Sending emergency notifications

[0731] The server sends an emergency notification based on the analysis results. The input is the emotional state data as the analysis result, the video and audio data recorded, and the current location information. The emergency notification is sent to the specified contacts (e.g., family members) using SendGrid or Firebase Cloud Messaging (FCM). The output is the emergency notification sent to the specified contacts.

[0732] (Application example 2)

[0733] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0734] Conventional security systems only have the function of notifying users when they approach a dangerous area, but do not take into account the user's emotional state or biometric data to respond immediately. This can result in a delay in appropriate response when the user senses danger. In addition, video and audio recording must be started manually, which can lead to situations where the user cannot operate the system in an emergency, making effective security measures ineffective.

[0735] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0736] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, means for monitoring the user's location information and notifying the user when the user approaches the risk area, means for recognizing the user's emotions by monitoring the user's heart rate and galvanic skin response, means for automatically starting video and audio recording when fear or anxiety is recognized, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results. This allows the user's emotional state to be reflected in real time, enabling quick and accurate crime prevention measures.

[0737] "Crime information" refers to the number of crimes occurring across the country and detailed information about crimes, and is data provided by police stations and public institutions.

[0738] "Crime prevention information" is data that includes information on crime prevention measures and warning information about areas where danger is expected, and is information to ensure safety and security.

[0739] "Risk areas" refer to areas where crimes are frequent or where caution is required for crime prevention, and are identified based on collected crime and crime prevention information.

[0740] "Location Information" means data that indicates a user's current geographic location, obtained using GPS or other location-determining technology.

[0741] An "emotion engine" is software or algorithm that analyzes biometric indicators such as a user's heart rate and galvanic skin response to recognize the user's emotional state.

[0742] "Video and audio recording" refers to the function of recording video and audio using a camera or microphone, and is used to generate evidence for crime prevention purposes.

[0743] "Vibration" is a physical movement caused by a device being shaken or struck by a user, and is detected by a sensor.

[0744] "Impact" is a physical action caused by the application of sudden force such as strong vibration or impact, and is detected by a sensor.

[0745] "Emergency Notification" is a feature that instantly notifies appropriate authorities, such as police, of a crisis situation and is performed automatically when certain events occur.

[0746] "Generated video and audio data" refers to video and audio data recorded by the video and audio recording functions.

[0747] "Emergency Notification" is a feature that allows you to quickly notify designated contacts (e.g., family and friends) if you are in a dangerous situation.

[0748] This invention is implemented as a security application called "Safe Night Guardian" that is installed on smartphones. This system ensures the safety of users who go out at night or other unsafe times. It collects and analyzes crime and crime prevention information from all over the country in real time, and links the user's current location and emotional state to enable quick response.

[0749] Hardware used

[0750] 1. Smartphones: Uses location information (GPS), camera and microphone. Examples: Apple iPhone and Samsung Galaxy.

[0751] 2. Wearable devices: monitor heart rate and galvanic skin response. Examples: Fitbit, Apple Watch.

[0752] Software used

[0753] 1. Server side:

[0754] AWS EC2: Cloud server for running programs.

[0755] AWS RDS: Database management for crime and crime prevention information.

[0756] Python: Data processing and machine learning algorithm implementation.

[0757] Scikit-Learn: A machine learning library for analyzing crime trends.

[0758] 2. Client side:

[0759] Android / iOS app: User interface and location data collection.

[0760] TensorFlow: A machine learning platform for building emotion engines.

[0761] Firebase Cloud Messaging: Real-time notifications to users.

[0762] 3. Notification system:

[0763] Twilio API: For emergency calling and notification.

[0764] System operation explanation

[0765] Collection and analysis of crime prevention information

[0766] The server periodically receives crime and crime prevention information from police and public institutions across the country via API. Based on this data, machine learning algorithms (using Python and Scikit-Learn) are used to calculate crime trends and risk levels for each region. This information is stored in AWS RDS and updated in real time.

[0767] User location monitoring and notification

[0768] The smartphone's GPS function is used to obtain the user's location information, which is then periodically sent to the server. The server compares this location information with information about dangerous areas stored in a database, and sends a warning notification to the smartphone if the user approaches a dangerous area (using Firebase Cloud Messaging).

[0769] Emotion recognition by emotion engine

[0770] Biometric data such as heart rate and skin galvanic response is collected from wearable devices (e.g., Fitbit, Apple Watch), and analyzed by an emotion engine using TensorFlow. The emotion engine recognizes the user's state of fear or anxiety in real time and takes action, such as automatically starting video or audio recording, as needed.

[0771] Auto-response function

[0772] If the emotion engine detects that the user is feeling fear or anxiety, it will automatically:

[0773] 1. Automatically start recording and recording: Record the situation using your smartphone's camera and microphone.

[0774] 2. Emergency call: A call to 110 is made using the Twilio API, and the user's current location and recorded video are provided.

[0775] 3. Emergency notification: Notify pre-defined contacts (e.g., family members) that the user is in danger and send them location information and a link to video data.

[0776] Specific examples

[0777] When a user is walking down a street at night, the app monitors their location in the background and displays a warning if they approach a dangerous area. If the user feels anxious, the smartwatch detects an increase in heart rate and galvanic skin response. The emotion engine analyzes this and, if it recognizes a state of fear, automatically starts video and audio recording and simultaneously calls 110. An emergency notification is also sent to family members.

[0778] Example prompts to input to the generative AI model

[0779] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[0780] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0781] Step 1:

[0782] Collection of crime prevention information

[0783] The server regularly obtains crime and crime prevention information from police and public institutions across the country via API.

[0784] Input: Crime and crime prevention data provided by police departments and public agency APIs.

[0785] Output: The acquired data is saved on the server. The server saves the acquired data in a database (e.g. AWS RDS).

[0786] Step 2:

[0787] Crime information analysis

[0788] The server analyzes the collected crime and crime prevention data using machine learning algorithms (e.g., Python and Scikit-Learn) to calculate crime trends and risk levels for each area.

[0789] Input: Crime and crime prevention data stored in a database.

[0790] Output: Analysis report data showing crime trends and risk levels. The server updates this data and stores it as risk area information.

[0791] Step 3:

[0792] Location acquisition and monitoring

[0793] The device (smartphone) periodically obtains the user's current location using its GPS function and sends it to the server.

[0794] Input: User location data obtained from the smartphone's GPS.

[0795] Output: The user's current location information is sent to the server, which monitors the user's location in real time and prepares a warning notification if the user approaches a dangerous area.

[0796] Step 4:

[0797] Notification when approaching a dangerous area

[0798] The server compares the user's location information with the danger area information, and if the user approaches a danger area, it sends a warning notification to the device (smartphone).

[0799] Input: User's current location information and danger area information.

[0800] Output: A warning notification is sent to your smartphone. Use a notification system (e.g., Firebase Cloud Messaging) to display real-time warnings.

[0801] Step 5:

[0802] Biometric data acquisition and emotion recognition

[0803] A wearable device (e.g., Fitbit, Apple Watch) worn by the user captures heart rate and electrodermal response data and sends them to the emotion engine, which uses TensorFlow to analyze this data and recognize the user's emotional state (e.g., fear, anxiety).

[0804] Input: Biometric data such as heart rate and galvanic skin response obtained from wearable devices.

[0805] Output: The user's emotional state (e.g., fear, anxiety) as a result of the emotion engine's analysis. If the emotion engine detects fear or anxiety, it prepares the next step.

[0806] Step 6:

[0807] Automatic start of video and audio recording

[0808] When fear or anxiety is detected from the user's biometric data, the device (smartphone) automatically starts recording and recording audio using its camera and microphone.

[0809] Input: Signal from emotion engine (user's emotional state).

[0810] Output: Start recording. The device will record video and audio and save the data.

[0811] Step 7:

[0812] Making an emergency call

[0813] If the device's emotion engine detects fear or anxiety, it will automatically call 110 using an emergency notification system (e.g., Twilio API) and provide current location information and recorded video.

[0814] Input: Emotion engine signals, video and audio data, and current location information.

[0815] Output: Report, current location and recorded video sent to police. The device will automatically make an emergency call.

[0816] Step 8:

[0817] Sending emergency notifications

[0818] If the emotion engine detects fear or anxiety, the device will send an emergency notification to pre-defined contacts (e.g., family members) informing them that they are in danger, along with their current location and a link to the recorded video.

[0819] Input: Emotion engine signal, current location information, and recorded video link.

[0820] Output: Emergency notification sent to configured contacts. Notifications are sent to family members using Firebase Cloud Messaging.

[0821] Example prompts to input to the generative AI model

[0822] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[0823] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0824] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0825] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0826] [Third embodiment]

[0827] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0828] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0829] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0830] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0831] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0832] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0833] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0834] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0835] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0836] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0837] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0838] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0839] This invention is a security system for ensuring personal safety, primarily when out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it automatically starts recording and recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[0840] Program processing

[0841] Collection and notification of crime prevention information

[0842] server

[0843] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. This includes programs that retrieve data via APIs. The server analyzes this data and runs algorithms to identify crime trends and high-risk areas. The identified risk areas are stored in a database that is continually updated.

[0844] User location monitoring and notification

[0845] server

[0846] The server receives location information periodically sent from the user's smartphone. The server-side program monitors this location information in real time, and if it determines that the user's current location is approaching a risky area, it immediately sends a warning notification to the user's smartphone. The notification includes the message, "You are currently approaching a high-risk area."

[0847] Start of video and audio recording

[0848] User

[0849] The user checks the warning notification on their smartphone and taps the recording button on the smartphone app. This activates the Walking Recorder's high-resolution camera and microphone, recording video and audio of the surrounding area. The recorded data is saved to the device's internal storage or a cloud service.

[0850] Vibration and shock detection

[0851] Terminal

[0852] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will immediately flash at maximum brightness to scare off suspicious individuals. At the same time, if a strong impact is detected again, an emergency signal will be automatically sent to your smartphone.

[0853] Automatic reporting

[0854] Terminal

[0855] When a smartphone receives an emergency signal, it automatically calls 110 according to pre-defined emergency protocols, and the call includes the user's current location and real-time video recording.

[0856] emergency notification

[0857] Terminal

[0858] The AI ​​analyzes the camera footage and audio sequentially, and if it determines that the user is in danger, it sends an emergency notification to pre-defined contacts such as family and close friends. The notification includes a message saying "The user is currently in danger," along with location information and a link to the video data. This enables rapid assistance in an emergency.

[0859] Specific examples

[0860] Night Out Scenario

[0861] When a user is walking on a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone. The user receives the notification and starts recording on the Walking Recorder. After a while, a suspicious person approaches from behind and grabs the user's shoulder. The Walking Recorder detects a strong vibration and immediately lights up its LED light. At the same time, recording continues, and if the suspicious person delivers a stronger shock, the Walking Recorder automatically reports the incident and sends the video data to the police. The generating AI then analyzes the situation and sends an emergency notification to the user's family.

[0862] This system not only allows users to go out safely at night, but also enables them to respond quickly in the event of an unforeseen incident.

[0863] The processing flow will be explained below.

[0864] Step 1: Gather crime prevention information

[0865] Subject: Server

[0866] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, specifically collecting data from government crime databases and public data from local police stations.

[0867] Step 2: Data analysis

[0868] Subject: Server

[0869] The server analyzes the collected crime and crime prevention information, using machine learning algorithms to calculate crime trends and risk levels for each area, allowing risk areas to be identified.

[0870] Step 3: Location Monitoring

[0871] Subject: Server

[0872] The server receives current location information periodically sent from the user's smartphone. The user's smartphone uses GPS to identify its location and periodically sends location data to the server.

[0873] Step 4: Risk Area Notification

[0874] Subject: Server

[0875] If the server determines that the user's current location is approaching a high-risk area, it sends a warning notification to the user's smartphone. This notification includes the message, "You are currently approaching a high-risk area."

[0876] Step 5: Start recording

[0877] Subject: User

[0878] The user checks the warning notification on their smartphone, opens the smartphone app, and taps the video and audio recording button, which activates the walking recorder and starts recording video and audio of the surrounding area.

[0879] Step 6: Vibration and shock sensing

[0880] Subject: Terminal

[0881] The Walking Recorder uses a built-in acceleration sensor to constantly detect vibrations. If a strong vibration is detected, the LED light will immediately flash at maximum brightness. If a stronger impact is subsequently detected, an emergency signal will be sent to your smartphone.

[0882] Step 7: Automatic reporting

[0883] Subject: Terminal

[0884] When a smartphone receives an emergency signal, it automatically initiates the 110 emergency call process, which includes the user's current location and recorded video data, allowing the police to immediately be notified of the situation at the scene.

[0885] Step 8: Emergency Notification

[0886] Subject: Terminal

[0887] The AI ​​analyzes the camera footage and audio in real time, and if it determines that the user is in danger, an emergency notification is sent to pre-defined contacts (such as family members). This notification includes a message saying "The user is currently in danger," along with location information and a link to the video data.

[0888] This series of processes improves the safety of the user and allows for a quick response in dangerous situations.

[0889] Example 1

[0890] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0891] In modern society, crime prevention measures are extremely important when people are out at night or alone. However, current crime prevention systems have difficulty notifying users of crime information and dangerous areas in real time, and they have limited means of taking prompt and appropriate action in emergencies. For these reasons, a system that can ensure even greater safety is needed.

[0892] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0893] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, and means for monitoring the user's location information and notifying the user when the user approaches the risk area. This allows the user to receive real-time notifications when the user approaches a risk area. The server also includes means for starting video and audio recording in response to a user's instruction, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results, thereby enabling the user to respond quickly and appropriately when they encounter an unexpected situation. The system includes a means for monitoring a user's location information in real time and detecting approach to a dangerous area, a means for sending a warning notification as a push notification, a means for detecting vibrations using an acceleration sensor in the security system and controlling an LED light when the vibration exceeds a threshold, a means for storing the generated video data in a cloud service, a means for sending an emergency signal to a smart device using Bluetooth or Wi-Fi Direct, a means for using a telephone API to automatically make an emergency call, and a means for sending an emergency notification using SMS or a notification API. This allows users to act safely even when out at night or in isolated situations, and enables quick response in the event of an emergency.

[0894] "Means of collecting crime information and crime prevention information from all over the country" refers to the function of regularly obtaining crime information and crime prevention information provided by police stations and public institutions from all over the country and storing it in a database.

[0895] "Means for analyzing the collected information and identifying risk areas" refers to the function of analyzing collected crime information and crime prevention information and identifying crime trends and high-risk areas.

[0896] "Means for monitoring the user's location information and notifying when the user approaches the dangerous area" refers to a function that monitors the location information sent from the user's smart device and sends a warning notification when the user approaches the dangerous area.

[0897] The term "means for starting video and audio recording in response to a user's instruction" refers to a function that allows a user to start video and audio recording by operating an application.

[0898] "Means for detecting vibrations and impacts" refers to the function of detecting vibrations and impacts using an acceleration sensor built into the security device.

[0899] "Means for emitting light when vibration is detected" refers to the function of illuminating the security device's LED light at maximum brightness when vibration is detected by the acceleration sensor.

[0900] "Means for automatically making an emergency call when an impact is detected" refers to a function that automatically makes an emergency call when a strong impact is detected by the acceleration sensor.

[0901] "Means for analyzing generated video and audio data" refers to the function of analyzing video and audio data recorded by a user's smart device.

[0902] "Means for sending an emergency notification based on the analysis results" refers to the function in which the generative AI model analyzes video and audio data and sends a notification if it determines that an emergency exists.

[0903] "Means for monitoring the user's location information in real time and detecting when the user approaches a dangerous area" refers to a function for monitoring the user's location information in real time and detecting when the user approaches a dangerous area.

[0904] "Means for sending a warning notification as a push notification" refers to a function that sends a warning using a push notification when a user approaches a dangerous area.

[0905] "Means for detecting vibrations using an acceleration sensor in a security system and controlling an LED light when the vibrations exceed a threshold" refers to a function that detects vibrations using an acceleration sensor and controls an LED light to turn on when the vibrations exceed a certain threshold.

[0906] "Means for storing generated video data in a cloud service" refers to a function for storing video data recorded on a user's device in a cloud service.

[0907] "Means for transmitting emergency signals to a smart device using Bluetooth or Wi-Fi Direct" refers to the function of transmitting emergency signals to a user's smart device using Bluetooth or Wi-Fi Direct.

[0908] "Means of using a telephone API to automatically make an emergency call" refers to a function that automatically makes an emergency call using a telephone API when an emergency signal is received.

[0909] "Means of sending emergency notifications using SMS or notification APIs" refers to the ability to send emergency notifications to pre-defined contacts using SMS or notification APIs.

[0910] This invention is a security system for ensuring personal safety, primarily when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. It also automatically starts recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[0911] Server Operation

[0912] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. The collection process uses an API, including a program that sends HTTP requests to retrieve data in JSON format and stores it in a database. The server analyzes the collected data and runs machine learning algorithms to identify crime trends and high-risk areas. The results are stored in a database, providing constantly up-to-date information.

[0913] The server also receives location information periodically sent from the user's smartphone and monitors it in real time. When the user approaches a risky area, the server immediately generates a warning notification and sends it to the user's smartphone using a notification service such as Firebase Cloud Messaging (FCM). The specific notification message may say, "You are currently approaching a high-risk area."

[0914] User smartphone behavior

[0915] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. This action calls the smartphone's camera API and microphone API, which starts recording the surrounding video and audio. The recorded data is saved in the device's internal storage or on a cloud service.

[0916] Walking Recorder Operation

[0917] The Walking Recorder constantly detects vibrations using a built-in accelerometer. If a strong vibration is detected, the system immediately turns on the LED light at maximum brightness. This is achieved by acquiring vibration data using the sensor API and calling the LED light control API when a threshold is exceeded. If an even stronger impact is detected, the Walking Recorder will send an emergency signal to your smartphone. This is done using Bluetooth or Wi-Fi Direct.

[0918] Handling emergency calls and notifications

[0919] When a smartphone receives an emergency signal, it automatically makes an emergency call using a phone API according to pre-defined emergency protocols. This call includes the user's current location and recorded video. If the generative AI model analyzes the video and audio and determines that the user is in danger, it sends a notification to pre-defined emergency contacts via SMS or a notification API. The notification includes the user's current location and a link to the video data.

[0920] Specific examples

[0921] For example, if a user is walking down a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone saying, "You are approaching a high-risk area." The user confirms the notification and taps the recording button to begin recording. If a suspicious person approaches and grabs the user's shoulder, the Walking Recorder detects the strong vibration and immediately illuminates its LED light. If an even stronger impact is applied, the Walking Recorder sends an emergency signal to the smartphone, which then automatically reports the incident and sends the video data and current location to the police. The generative AI model then analyzes the situation and sends an emergency notification to the user's family.

[0922] Prompt Sentence Examples

[0923] Could you please explain the details of the nighttime security system? Location monitoring, danger zone notification, sudden vibration and shock detection, video and audio recording start, automatic reporting, and emergency notification.

[0924] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0925] Step 1:

[0926] server

[0927] The server collects crime and crime prevention information provided by police stations and public institutions across the country. Specifically, the server periodically sends HTTP requests to the API endpoint and retrieves data in JSON format. This data includes information such as the date, time, location, and type of crime. The collected data is stored in a database and used for subsequent analysis. (Input): Response data from the API. (Output): Crime information stored in the database.

[0928] Step 2:

[0929] server

[0930] The server analyzes the collected crime information using a machine learning algorithm to identify high-risk areas. Specifically, it applies a clustering algorithm (e.g., K-means algorithm) based on past crime data to identify areas where crimes are concentrated. The results of this analysis are stored in a database and used to match users' location information. (Input): Crime information stored in the database. (Output): High-risk areas identified through analysis.

[0931] Step 3:

[0932] server

[0933] The server receives real-time location information periodically sent from the user's smartphone. The received location information is sent to the server as GPS data, and monitoring is performed based on that data. If the user's current location approaches a high-risk area, the server generates a warning notification. (Input): GPS data from the smartphone. (Output): Generated warning notification.

[0934] Step 4:

[0935] server

[0936] The server sends the generated warning notification to the user's smartphone. Specifically, it sends a push notification using a notification service such as Firebase Cloud Messaging (FCM). This notification contains the message "You are currently approaching a high-risk area." (Input): Warning notification generated by the server. (Output): Push notification sent to the user's smartphone.

[0937] Step 5:

[0938] User

[0939] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. The smartphone app calls the camera API and microphone API to begin recording video and audio. This recorded data is saved in the internal storage or on a cloud service. (Input): User's tap operation. (Output): Video and audio data saved on the smartphone.

[0940] Step 6:

[0941] Terminal

[0942] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will turn on at maximum brightness to scare off suspicious people. The sensor API is used to obtain vibration data, and if the vibration exceeds a threshold, the LED control API is called to turn on the light. (Input): Vibration data from the acceleration sensor. (Output): The lit LED light.

[0943] Step 7:

[0944] Terminal

[0945] If the Walking Recorder detects another strong impact, it immediately sends an emergency signal to your smartphone. The emergency signal is transmitted using Bluetooth or Wi-Fi Direct. This signal contains information about the strength of the impact and your location. (Input): Strong impact data from the acceleration sensor. (Output): Emergency signal sent to your smartphone.

[0946] Step 8:

[0947] Smartphone

[0948] When the smartphone receives an emergency signal, it automatically calls 110 using the phone API according to the pre-set emergency protocol. The call includes the user's current location and real-time video recording. (Input): Emergency signal from the walking recorder. (Output): Current location and video data sent along with the automatic call.

[0949] Step 9:

[0950] Generative AI Models

[0951] The generating AI sequentially analyzes camera footage and audio, and if it determines that the user is in danger, it sends an emergency notification to pre-set contacts such as family and close friends. Specifically, it uses a notification API and SMS service to send a message saying "The user is currently in danger," along with location information and a link to the video data. (Input): Camera footage and audio data. (Output): Emergency notification sent to family and pre-set contacts.

[0952] (Application example 1)

[0953] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0954] Ensuring personal safety when out at night is extremely important, but conventional security systems often lack sufficient risk area notification and emergency response capabilities, requiring users to operate the system directly. Furthermore, users lack a means to receive visual information, making it difficult to quickly recognize danger. Furthermore, automation of emergency calls and video and audio recording is insufficient, potentially preventing effective action even in situations requiring a rapid response.

[0955] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0956] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country and monitoring user location information, means for analyzing the collected information to identify risk areas, means for notifying the user when they approach a risk area, and means for starting video and audio recording in response to a user's instruction.In this invention, by providing an easy-to-use device and interface, it is possible to quickly and effectively recognize danger and take appropriate action, by detecting vibrations and impacts using a built-in sensor and emitting light to automatically start video and audio recording, means for analyzing the generated video and audio data and sending an emergency notification based on the analysis results, means for providing a visual display device with a notification function to warn the user in real time, and means for implementing an interface function for the user to start video and audio recording.

[0957] "Crime information and crime prevention information" refers to data on crime occurrence and prevention measures provided by police stations and public institutions across the country.

[0958] "Risk areas" refer to areas that have been identified as having a high risk of crime occurring based on the results of an analysis of collected crime and crime prevention information.

[0959] "Location Information" means data that indicates a user's current geographic location, obtained through GPS or other location-determining technology.

[0960] "Notification" refers to the action of displaying or sending a warning or information to a user's device when the user approaches a dangerous area.

[0961] "Video and audio recording" refers to the act of activating the camera and microphone, either at the user's command or automatically, to record video and audio of the surrounding area.

[0962] "Vibration and shock" refers to physical shaking or impact detected by an accelerometer.

[0963] "Emitting light" refers to the act of turning on an LED light or similar as a visual warning when vibration is detected.

[0964] "Emergency call" refers to the action of automatically notifying pre-defined contacts or the police to request help.

[0965] "Analyzing the generated video and audio data" refers to the process of analyzing video and audio data using AI models, etc., to determine dangerous situations.

[0966] "Send emergency notification" refers to the action of notifying set contacts, such as family and friends, of the user's situation based on the analysis results.

[0967] "Visual display device" refers to a device (e.g., smart glasses) worn by a user that displays information in real time.

[0968] The "interface function" refers to a function that provides an operation method for the user to start video and audio recording.

[0969] This invention is a security system to ensure the safety of users when they are out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies them when they approach a dangerous area. It also provides a series of processes that automatically start recording and recording when it detects vibration or impact, and automatically reports and notifies users in the event of an emergency.

[0970] System configuration

[0971] The system consists of the following components:

[0972] 1. Server:

[0973] Collection of crime and crime prevention information: Data provided by police stations and public institutions across the country is periodically obtained via API. The server analyzes this data to identify crime trends and high-risk areas.

[0974] Monitoring user location information: The server periodically receives location information from the user's smart device and sends a warning notification to the user if the device approaches a dangerous area.

[0975] 2. User terminal (smart glasses):

[0976] Video and audio recording function: When the user confirms the alert notification, the smart glasses' built-in camera and microphone will activate and record the surrounding video and audio. The recorded data will be saved in the internal storage or cloud service.

[0977] Vibration and impact detection: The smart glasses' built-in acceleration sensor constantly detects vibrations, and when a strong vibration is detected, the LED light will turn on. At the same time, if a strong impact is detected, an emergency signal will be automatically sent to the server.

[0978] Automatic notification function: When the smart glasses receive an emergency signal, they will automatically make an emergency call according to the emergency protocol pre-set on the server side. The call will include the user's current location information and real-time video recording data.

[0979] Emergency Notification: If the generative AI model analyzes camera footage and audio and determines that the user is in danger, it will send an emergency notification to pre-defined contacts, including location information and a link to the video recording.

[0980] Hardware and Software Details

[0981] Hardware:

[0982] Smart Glasses: Uses a display (e.g., smart glasses) that displays information in real time.

[0983] Acceleration sensor: Built-in sensor for detecting vibration and impact.

[0984] High-resolution camera and microphone: Equipped with a device for recording video and audio of the surroundings.

[0985] LED light: Visual warning when vibration is detected.

[0986] software:

[0987] Server: Uses cloud services such as AWS and Google Cloud Platform to collect and analyze data and execute emergency protocols.

[0988] Generative AI model: Analyzes video and audio data using OpenAI GPT-4 and other models to determine emergency situations.

[0989] Notification app: An application for iOS and Android devices that monitors the user's location, controls recording and video recording, and sends emergency notifications.

[0990] Specific examples

[0991] A user puts on smart glasses while walking down a street at night. The server detects that the user is approaching a dangerous area, and a warning message is displayed on the smart glasses. The user confirms the warning and starts recording. If a suspicious person approaches and grabs the user's shoulder, the smart glasses detect a strong vibration, turn on an LED light, and automatically start recording. If the suspicious person causes further harm, the generating AI analyzes the situation and makes an emergency call via the server. In addition, an emergency notification is sent to family and friends.

[0992] Example prompts for generative AI models

[0993] "I would like to develop the following security application. A system in which users wear smart glasses and are warned of approaching dangerous areas when going out at night. The glasses can record video and audio using the camera and microphone, and will turn on an LED light if a suspicious person comes into contact with the user. If vibration or impact is detected, the glasses will automatically report the user and send an emergency notification."

[0994] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0995] Step 1:

[0996] The server collects crime and crime prevention information from all over the country. The input is data provided by police stations and public institutions. This data is obtained via API and stored in a database on the server. The output is a database of the collected crime and crime prevention information.

[0997] Step 2:

[0998] The server analyzes the collected crime and crime prevention information and identifies risk areas. The input is crime information in the database. The server performs data analysis to identify high-risk areas using statistical models and machine learning algorithms. The output is information on areas identified as high-risk areas.

[0999] Step 3:

[1000] The server receives location information periodically sent from the user's smart glasses. The input is the user's location information. The server monitors this information in real time, and if it determines that the user is approaching a dangerous area, it sends a warning notification to the user's smart glasses. The output is a warning message displayed on the user's smart glasses.

[1001] Step 4:

[1002] The user confirms the warning notification and taps the video / audio button on the smart glasses. The input is the warning notification. This action activates the camera and microphone on the smart glasses and starts recording the surrounding video and audio. The output is the video and audio data.

[1003] Step 5:

[1004] The smart glasses' accelerometer constantly detects vibrations. The input is the accelerometer data. If a strong vibration is detected, the smart glasses will turn on the LED light to scare off suspicious people. The output is the LED light being turned on and the glasses will continue recording.

[1005] Step 6:

[1006] If a strong impact is detected again, the smart glasses will automatically send an emergency signal to the server. The input is the impact data from the acceleration sensor. The output is the emergency signal sent to the server.

[1007] Step 7:

[1008] When the server receives an emergency signal, it automatically makes an emergency call according to a pre-defined emergency protocol. The input is the emergency signal and the user's current location information. The call includes the user's current location information and real-time video recording data. The output is a call to 110.

[1009] Step 8:

[1010] The generative AI model sequentially analyzes camera footage and audio and determines whether the user is in a dangerous situation. The input is video and audio data. The output is the analysis result that determines whether the situation is an emergency.

[1011] Step 9:

[1012] The server sends emergency notifications to pre-defined contacts, such as family members and close friends, based on the analysis results of the generative AI model. The inputs are the analysis results, the user's location information, and a link to the video recording data. The output is an emergency notification message sent to the emergency contacts.

[1013] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1014] This invention is a crime prevention system for ensuring personal safety, primarily when out at night, that collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, this system uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[1015] Program processing

[1016] Collection and notification of crime prevention information

[1017] server

[1018] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. This data is collected and machine learning algorithms are used to calculate crime trends and risk levels for each area. Identified risk areas are stored in a database and updated in real time.

[1019] User location monitoring and notification

[1020] server

[1021] The server receives location information periodically sent from the user's smartphone. The user's smartphone identifies its location using GPS and periodically sends the location data to the server. When the user approaches a dangerous area, a warning notification is sent to the smartphone.

[1022] Emotion recognition by emotion engine

[1023] Terminal

[1024] The emotion engine recognizes emotions such as fear and anxiety by monitoring biometric indicators such as the user's heart rate and galvanic skin response. These biometric indicators are acquired through a wearable device and transmitted to the emotion engine.

[1025] Automatic response through emotion recognition

[1026] Terminal

[1027] If the emotion engine detects that the user is feeling fear or anxiety, the following steps are automatically performed:

[1028] 1. Start recording and audio: When the emotion engine recognizes the user's fear or anxiety, the walking recorder will automatically start and record video and audio of the surrounding area.

[1029] 2. Emergency Call: The emotion engine recognizes the user's fear and anxiety and, if it determines that a crisis is occurring, automatically initiates a call to 110. The call will include the user's current location and recorded video.

[1030] 3. Sending emergency notifications: If the emotion engine determines that the user is experiencing fear or anxiety, it will immediately send an emergency notification to pre-defined contacts (e.g., family members). The notification will include the message "The user is in danger," along with location information and a link to the video data.

[1031] Specific examples

[1032] Night Out Scenario

[1033] When a user is walking on a street at night, the server monitors the user's current location and detects that they are approaching a dangerous area, sending a warning notification to their smartphone. The user receives the notification, but begins to feel anxious or scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the walking recorder detects strong vibrations and immediately lights up an LED light. Furthermore, if the user feels extreme fear or is shocked, the emotion engine immediately calls 110 and sends an emergency notification to their family.

[1034] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[1035] The processing flow will be explained below.

[1036] Step 1: Gather crime prevention information

[1037] Subject: Server

[1038] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, and this data is stored in the server's database.

[1039] Step 2: Data analysis

[1040] Subject: Server

[1041] The server analyzes the collected crime and crime prevention information, and uses machine learning algorithms to calculate crime trends and risk levels for each area. The analysis results are stored in a database.

[1042] Step 3: Location Monitoring

[1043] Subject: Server

[1044] The server receives location information periodically sent from the user's smartphone. The smartphone uses GPS to identify the user's location and sends that data to the server.

[1045] Step 4: Risk Area Notification

[1046] Subject: Server

[1047] When the server detects that the user is approaching a high-risk area based on the user's current location and the analysis results, it sends a danger notification to the user's smartphone. The notification includes a message saying, "You are currently approaching a high-risk area."

[1048] Step 5: Emotion Recognition with the Emotion Engine

[1049] Subject: Terminal

[1050] The emotion engine built into the device receives biometric data such as the user's heart rate and skin galvanic response from the wearable device, analyzes this data, and evaluates the user's emotional state (especially fear and anxiety) in real time.

[1051] Step 6: Start recording and recording with emotion recognition

[1052] Subject: Terminal

[1053] When the emotion engine recognizes the user's fear or anxiety, the device automatically starts recording and audio recording. The Walking Recorder's camera and microphone are activated to capture video and audio.

[1054] Step 7: Emotion-Recognition Automatic Reporting

[1055] Subject: Terminal

[1056] If the emotion engine recognizes the user's fear or anxiety, and detects a strong impact or fall, the device will automatically call 110. The call will include the user's current location and recorded video.

[1057] Step 8: Send emergency notifications

[1058] Subject: Terminal

[1059] If the emotion engine detects that the user is in danger, the device will immediately send an emergency notification to pre-defined contacts, containing the message "The user is in danger," along with location information and a link to the video data.

[1060] Specific examples

[1061] Night Out Scenario

[1062] When a user is walking on a street at night, the server monitors the user's current location and sends a warning notification to their smartphone if it detects that they are approaching a dangerous area. The user receives the notification, and the emotion engine begins monitoring their heart rate and skin galvanic response. If the user begins to feel anxious, the emotion engine recognizes this and the walking recorder automatically begins recording and audio recording. If a suspicious person approaches, the device detects vibrations and impacts and automatically reports the incident. An emergency notification is also sent to family members.

[1063] Example 2

[1064] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1065] Personal safety when out at night is an important issue in modern society. To address this issue, a system that can detect danger in real time and respond quickly is needed. However, current security systems do not take the user's emotional state into account, making it difficult to respond appropriately. There is also a need for a system that can automatically respond when the user feels fear or anxiety.

[1066] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting crime information and crime prevention information from all over the country; means for analyzing the collected information and identifying risk areas; means for monitoring the user's location information and notifying the user when the user approaches the risk area; means for collecting the user's biometric indicators and analyzing their emotional state; means for starting video and audio recording when it is determined that the user is feeling fear or anxiety; means for monitoring the biometric indicators in real time and automatically making an emergency call when fear or anxiety is detected; means for detecting vibrations and impacts; means for emitting light when the vibrations are detected; means for analyzing the generated video and audio data; and means for sending an emergency notification based on the analysis results. This enables danger to be detected in real time and appropriate responses taking into account the user's emotional state.

[1067] "Means for collecting crime information and crime prevention information nationwide" refers to the means for collecting data on the number of crimes and crime prevention information obtained from each region of the country and storing it on a server.

[1068] "Means for analyzing and identifying risk areas" refers to means for analyzing collected crime information and crime prevention information and identifying areas where there is a high risk of crime occurring.

[1069] The "means for monitoring the user's location information and notifying when the user approaches the risk area" is a means for monitoring the user's current location using GPS and sending a warning notification when the user approaches an identified risk area.

[1070] "Means for collecting biometric indicators of a user and analyzing their emotional state" refers to means for analyzing a user's emotional state based on data such as the user's heart rate and skin galvanic response obtained from a wearable device or the like.

[1071] The "means for starting video and audio recording when it is determined that the user is feeling fear or anxiety" refers to a means for automatically activating the smartphone's camera and microphone to record video and audio of the surrounding area when the emotion engine determines that the user's emotional state is fear or anxiety.

[1072] The "means for automatically making an emergency call when fear or anxiety is detected" is a means for automatically making an emergency call when fear or anxiety is detected from the biometric indicators of the user.

[1073] The "means for detecting vibration and impact" is a means for detecting vibration or impact when the user receives vibration or impact from the outside.

[1074] The "means for emitting light when vibration is detected" is a means for notifying the surroundings by immediately emitting light based on the detected vibration.

[1075] The "means for analyzing the generated video and audio data" refers to a means for analyzing the video and audio data that has been recorded and detecting abnormalities or emergencies.

[1076] The "means for sending an emergency notification based on the analysis result" is a means for sending an emergency notification to the set contacts when an emergency is recognized based on the analysis result.

[1077] This invention is a crime prevention system for ensuring personal safety when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[1078] Collection and analysis of crime prevention information

[1079] server

[1080] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. For example, it uses government-provided public safety APIs and private crime data providers. The collected data is stored in a database, typically MySQL or PostgreSQL.

[1081] The server uses machine learning algorithms such as TensorFlow and Scikit-learn to analyze crime trends and risk levels in each area and identify risk areas, which are updated in real time.

[1082] Collection and monitoring of user location information

[1083] Terminal

[1084] The user's smartphone periodically obtains location information using the GPS sensor and the smartphone's standard location services (such as Google Location Services).

[1085] The smartphone periodically sends the acquired location information to the server. The HTTPS protocol is used for transmission, ensuring secure data transmission.

[1086] server

[1087] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area, and maps the user's current location using a map service such as Google Maps API.

[1088] If the user is determined to be approaching a dangerous area, a warning notification is sent to the smartphone. The notification is sent using a notification service such as Firebase Cloud Messaging (FCM).

[1089] Biometric data collection for emotion recognition

[1090] Terminal

[1091] Users wear a wearable device such as an Apple Watch or Fitbit to periodically measure vital signs such as heart rate and galvanic skin response, and this data is sent to a smartphone via Bluetooth.

[1092] The smartphone transmits the received biometric data to the emotion engine.

[1093] Automatic response through emotion recognition

[1094] Terminal

[1095] The emotion engine uses libraries such as OpenCV and DeepFace to analyze biometric data and determine whether the user is feeling fear or anxiety.

[1096] 1. If the user's state is judged to be fearful or anxious, the smartphone's camera and microphone will automatically activate and record video and audio of the surrounding area.

[1097] 2. Additionally, the emotion engine uses the Twilio API to automatically place emergency calls, including current location information and recorded video.

[1098] 3. It also uses SendGrid or Firebase Cloud Messaging (FCM) to send emergency notifications to configured contacts, including a message saying "The user is in danger," along with location information and a link to video data.

[1099] Specific examples of operation

[1100] When a user is walking down a street at night, the server monitors the user's current location and sends a warning notification to the smartphone if it detects that the user is approaching a dangerous area. The user receives the notification, but begins to feel anxious and scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the smartphone's camera will record video and the microphone will record audio. The emotion engine also analyzes the user's condition and, if it determines that the situation is critical, will immediately call emergency services and send an emergency notification to family members.

[1101] Prompt Sentence Examples

[1102] "Please explain the process of a security system that sends a warning notification to the user's smartphone when the user approaches a dangerous area while out at night, and automatically starts video and audio recording if the emotion engine recognizes fear or anxiety."

[1103] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[1104] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1105] Step 1:

[1106] Collection of crime prevention information

[1107] The server periodically obtains nationwide crime and crime prevention information via API. As input, it receives data from APIs provided by police stations and public institutions. The server analyzes this data and stores it in a database. Specifically, it uses public safety APIs provided by the government and private crime data providers. The output is data in a format that stores the number of crimes and crime prevention information for each region in a database.

[1108] Step 2:

[1109] Crime information analysis

[1110] The server analyzes the stored crime information using machine learning algorithms. It uses the crime information stored in the database as input. It calculates crime trends and risk levels for each area using TensorFlow, Scikit-learn, etc. The output is data showing the crime risk level for each area. The analysis results are then stored back in the database, and risk areas are identified.

[1111] Step 3:

[1112] Collection of user location information

[1113] The user's smartphone periodically acquires location information using a GPS sensor. The input is GPS data, which is acquired using the smartphone's location service (e.g., Google Location Services). The acquired location information is sent to the server using the HTTPS protocol. The output is the user's current location data sent to the server.

[1114] Step 4:

[1115] Location monitoring and notifications

[1116] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area. The input is the user's current location data, which is mapped on a map using Google Maps API. If the server determines that the user is approaching a dangerous area, it sends a warning notification to the smartphone. The output is the warning notification sent to the smartphone.

[1117] Step 5:

[1118] Biometric data collection

[1119] A user wears a wearable device and periodically measures their heart rate and electrodermal response. The input is the biometric data measured by the wearable device. For example, a device such as an Apple Watch or Fitbit collects this data and transmits it to a smartphone via Bluetooth. The output is the biometric data transmitted to the smartphone.

[1120] Step 6:

[1121] Emotional state analysis

[1122] The smartphone sends the received biometric data to the emotion engine. The input is the biometric data sent from the wearable device. The emotion engine analyzes this data using libraries such as OpenCV and DeepFace to determine whether the user is feeling fear or anxiety. The output is the user's emotional state data as a result of the analysis.

[1123] Step 7:

[1124] Start of video and audio recording

[1125] When the emotion engine recognizes the user's fear or anxiety, it automatically activates the smartphone's camera and microphone. The input is emotional state data, and it is determined that the user is feeling fear or anxiety. Video and audio recording is started, and surrounding video and audio are recorded. The output is the video and audio data.

[1126] Step 8:

[1127] Handling emergency calls

[1128] If the emotion engine recognizes the user's fear or anxiety and determines that the situation is critical, the server automatically makes an emergency call. The input is recorded video and audio data, as well as current location information. Using the Twilio API, this data is included in the call content. The output is data indicating that an emergency call has been made.

[1129] Step 9:

[1130] Sending emergency notifications

[1131] The server sends an emergency notification based on the analysis results. The input is the emotional state data as the analysis result, the video and audio data recorded, and the current location information. The emergency notification is sent to the specified contacts (e.g., family members) using SendGrid or Firebase Cloud Messaging (FCM). The output is the emergency notification sent to the specified contacts.

[1132] (Application example 2)

[1133] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1134] Conventional security systems only have the function of notifying users when they approach a dangerous area, but do not take into account the user's emotional state or biometric data to respond immediately. This can result in a delay in appropriate response when the user senses danger. In addition, video and audio recording must be started manually, which can lead to situations where the user cannot operate the system in an emergency, making effective security measures ineffective.

[1135] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1136] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, means for monitoring the user's location information and notifying the user when the user approaches the risk area, means for recognizing the user's emotions by monitoring the user's heart rate and galvanic skin response, means for automatically starting video and audio recording when fear or anxiety is recognized, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results. This allows the user's emotional state to be reflected in real time, enabling quick and accurate crime prevention measures.

[1137] "Crime information" refers to the number of crimes occurring across the country and detailed information about crimes, and is data provided by police stations and public institutions.

[1138] "Crime prevention information" is data that includes information on crime prevention measures and warning information about areas where danger is expected, and is information to ensure safety and security.

[1139] "Risk areas" refer to areas where crimes are frequent or where caution is required for crime prevention, and are identified based on collected crime and crime prevention information.

[1140] "Location Information" means data that indicates a user's current geographic location, obtained using GPS or other location-determining technology.

[1141] An "emotion engine" is software or algorithm that analyzes biometric indicators such as a user's heart rate and galvanic skin response to recognize the user's emotional state.

[1142] "Video and audio recording" refers to the function of recording video and audio using a camera or microphone, and is used to generate evidence for crime prevention purposes.

[1143] "Vibration" is a physical movement caused by a device being shaken or struck by a user, and is detected by a sensor.

[1144] "Impact" is a physical action caused by the application of sudden force such as strong vibration or impact, and is detected by a sensor.

[1145] "Emergency Notification" is a feature that instantly notifies appropriate authorities, such as police, of a crisis situation and is performed automatically when certain events occur.

[1146] "Generated video and audio data" refers to video and audio data recorded by the video and audio recording functions.

[1147] "Emergency Notification" is a feature that allows you to quickly notify designated contacts (e.g., family and friends) if you are in a dangerous situation.

[1148] This invention is implemented as a security application called "Safe Night Guardian" that is installed on smartphones. This system ensures the safety of users who go out at night or other unsafe times. It collects and analyzes crime and crime prevention information from all over the country in real time, and links the user's current location and emotional state to enable quick response.

[1149] Hardware used

[1150] 1. Smartphones: Uses location information (GPS), camera and microphone. Examples: Apple iPhone and Samsung Galaxy.

[1151] 2. Wearable devices: monitor heart rate and galvanic skin response. Examples: Fitbit, Apple Watch.

[1152] Software used

[1153] 1. Server side:

[1154] AWS EC2: Cloud server for running programs.

[1155] AWS RDS: Database management for crime and crime prevention information.

[1156] Python: Data processing and machine learning algorithm implementation.

[1157] Scikit-Learn: A machine learning library for analyzing crime trends.

[1158] 2. Client side:

[1159] Android / iOS app: User interface and location data collection.

[1160] TensorFlow: A machine learning platform for building emotion engines.

[1161] Firebase Cloud Messaging: Real-time notifications to users.

[1162] 3. Notification system:

[1163] Twilio API: For emergency calling and notification.

[1164] System operation explanation

[1165] Collection and analysis of crime prevention information

[1166] The server periodically receives crime and crime prevention information from police and public institutions across the country via API. Based on this data, machine learning algorithms (using Python and Scikit-Learn) are used to calculate crime trends and risk levels for each region. This information is stored in AWS RDS and updated in real time.

[1167] User location monitoring and notification

[1168] The smartphone's GPS function is used to obtain the user's location information, which is then periodically sent to the server. The server compares this location information with information about dangerous areas stored in a database, and sends a warning notification to the smartphone if the user approaches a dangerous area (using Firebase Cloud Messaging).

[1169] Emotion recognition by emotion engine

[1170] Biometric data such as heart rate and skin galvanic response is collected from wearable devices (e.g., Fitbit, Apple Watch), and analyzed by an emotion engine using TensorFlow. The emotion engine recognizes the user's state of fear or anxiety in real time and takes action, such as automatically starting video or audio recording, as needed.

[1171] Auto-response function

[1172] If the emotion engine detects that the user is feeling fear or anxiety, it will automatically:

[1173] 1. Automatically start recording and recording: Record the situation using your smartphone's camera and microphone.

[1174] 2. Emergency call: A call to 110 is made using the Twilio API, and the user's current location and recorded video are provided.

[1175] 3. Emergency notification: Notify pre-defined contacts (e.g., family members) that the user is in danger and send them location information and a link to video data.

[1176] Specific examples

[1177] When a user is walking down a street at night, the app monitors their location in the background and displays a warning if they approach a dangerous area. If the user feels anxious, the smartwatch detects an increase in heart rate and galvanic skin response. The emotion engine analyzes this and, if it recognizes a state of fear, automatically starts video and audio recording and simultaneously calls 110. An emergency notification is also sent to family members.

[1178] Example prompts to input to the generative AI model

[1179] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[1180] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1181] Step 1:

[1182] Collection of crime prevention information

[1183] The server regularly obtains crime and crime prevention information from police and public institutions across the country via API.

[1184] Input: Crime and crime prevention data provided by police departments and public agency APIs.

[1185] Output: The acquired data is saved on the server. The server saves the acquired data in a database (e.g. AWS RDS).

[1186] Step 2:

[1187] Crime information analysis

[1188] The server analyzes the collected crime and crime prevention data using machine learning algorithms (e.g., Python and Scikit-Learn) to calculate crime trends and risk levels for each area.

[1189] Input: Crime and crime prevention data stored in a database.

[1190] Output: Analysis report data showing crime trends and risk levels. The server updates this data and stores it as risk area information.

[1191] Step 3:

[1192] Location acquisition and monitoring

[1193] The device (smartphone) periodically obtains the user's current location using its GPS function and sends it to the server.

[1194] Input: User location data obtained from the smartphone's GPS.

[1195] Output: The user's current location information is sent to the server, which monitors the user's location in real time and prepares a warning notification if the user approaches a dangerous area.

[1196] Step 4:

[1197] Notification when approaching a dangerous area

[1198] The server compares the user's location information with the danger area information, and if the user approaches a danger area, it sends a warning notification to the device (smartphone).

[1199] Input: User's current location information and danger area information.

[1200] Output: A warning notification is sent to your smartphone. Use a notification system (e.g., Firebase Cloud Messaging) to display real-time warnings.

[1201] Step 5:

[1202] Biometric data acquisition and emotion recognition

[1203] A wearable device (e.g., Fitbit, Apple Watch) worn by the user captures heart rate and electrodermal response data and sends them to the emotion engine, which uses TensorFlow to analyze this data and recognize the user's emotional state (e.g., fear, anxiety).

[1204] Input: Biometric data such as heart rate and galvanic skin response obtained from wearable devices.

[1205] Output: The user's emotional state (e.g., fear, anxiety) as a result of the emotion engine's analysis. If the emotion engine detects fear or anxiety, it prepares the next step.

[1206] Step 6:

[1207] Automatic start of video and audio recording

[1208] When fear or anxiety is detected from the user's biometric data, the device (smartphone) automatically starts recording and recording audio using its camera and microphone.

[1209] Input: Signal from emotion engine (user's emotional state).

[1210] Output: Start recording. The device will record video and audio and save the data.

[1211] Step 7:

[1212] Making an emergency call

[1213] If the device's emotion engine detects fear or anxiety, it will automatically call 110 using an emergency notification system (e.g., Twilio API) and provide current location information and recorded video.

[1214] Input: Emotion engine signals, video and audio data, and current location information.

[1215] Output: Report, current location and recorded video sent to police. The device will automatically make an emergency call.

[1216] Step 8:

[1217] Sending emergency notifications

[1218] If the emotion engine detects fear or anxiety, the device will send an emergency notification to pre-defined contacts (e.g., family members) informing them that they are in danger, along with their current location and a link to the recorded video.

[1219] Input: Emotion engine signal, current location information, and recorded video link.

[1220] Output: Emergency notification sent to configured contacts. Notifications are sent to family members using Firebase Cloud Messaging.

[1221] Example prompts to input to the generative AI model

[1222] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[1223] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1224] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1225] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1226] [Fourth embodiment]

[1227] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1228] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1229] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1230] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1231] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1232] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1233] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1234] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1235] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1236] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1237] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1238] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1239] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1240] This invention is a security system for ensuring personal safety, primarily when out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it automatically starts recording and recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[1241] Program processing

[1242] Collection and notification of crime prevention information

[1243] server

[1244] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. This includes programs that retrieve data via APIs. The server analyzes this data and runs algorithms to identify crime trends and high-risk areas. The identified risk areas are stored in a database that is continually updated.

[1245] User location monitoring and notification

[1246] server

[1247] The server receives location information periodically sent from the user's smartphone. The server-side program monitors this location information in real time, and if it determines that the user's current location is approaching a risky area, it immediately sends a warning notification to the user's smartphone. The notification includes the message, "You are currently approaching a high-risk area."

[1248] Start of video and audio recording

[1249] User

[1250] The user checks the warning notification on their smartphone and taps the recording button on the smartphone app. This activates the Walking Recorder's high-resolution camera and microphone, recording video and audio of the surrounding area. The recorded data is saved to the device's internal storage or a cloud service.

[1251] Vibration and shock detection

[1252] Terminal

[1253] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will immediately flash at maximum brightness to scare off suspicious individuals. At the same time, if a strong impact is detected again, an emergency signal will be automatically sent to your smartphone.

[1254] Automatic reporting

[1255] Terminal

[1256] When a smartphone receives an emergency signal, it automatically calls 110 according to pre-defined emergency protocols, and the call includes the user's current location and real-time video recording.

[1257] emergency notification

[1258] Terminal

[1259] The AI ​​analyzes the camera footage and audio sequentially, and if it determines that the user is in danger, it sends an emergency notification to pre-defined contacts such as family and close friends. The notification includes a message saying "The user is currently in danger," along with location information and a link to the video data. This enables rapid assistance in an emergency.

[1260] Specific examples

[1261] Night Out Scenario

[1262] When a user is walking on a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone. The user receives the notification and starts recording on the Walking Recorder. After a while, a suspicious person approaches from behind and grabs the user's shoulder. The Walking Recorder detects a strong vibration and immediately lights up its LED light. At the same time, recording continues, and if the suspicious person delivers a stronger shock, the Walking Recorder automatically reports the incident and sends the video data to the police. The generating AI then analyzes the situation and sends an emergency notification to the user's family.

[1263] This system not only allows users to go out safely at night, but also enables them to respond quickly in the event of an unforeseen incident.

[1264] The processing flow will be explained below.

[1265] Step 1: Gather crime prevention information

[1266] Subject: Server

[1267] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, specifically collecting data from government crime databases and public data from local police stations.

[1268] Step 2: Data analysis

[1269] Subject: Server

[1270] The server analyzes the collected crime and crime prevention information, using machine learning algorithms to calculate crime trends and risk levels for each area, allowing risk areas to be identified.

[1271] Step 3: Location Monitoring

[1272] Subject: Server

[1273] The server receives current location information periodically sent from the user's smartphone. The user's smartphone uses GPS to identify its location and periodically sends location data to the server.

[1274] Step 4: Risk Area Notification

[1275] Subject: Server

[1276] If the server determines that the user's current location is approaching a high-risk area, it sends a warning notification to the user's smartphone. This notification includes the message, "You are currently approaching a high-risk area."

[1277] Step 5: Start recording

[1278] Subject: User

[1279] The user checks the warning notification on their smartphone, opens the smartphone app, and taps the video and audio recording button, which activates the walking recorder and starts recording video and audio of the surrounding area.

[1280] Step 6: Vibration and shock sensing

[1281] Subject: Terminal

[1282] The Walking Recorder uses a built-in acceleration sensor to constantly detect vibrations. If a strong vibration is detected, the LED light will immediately flash at maximum brightness. If a stronger impact is subsequently detected, an emergency signal will be sent to your smartphone.

[1283] Step 7: Automatic reporting

[1284] Subject: Terminal

[1285] When a smartphone receives an emergency signal, it automatically initiates the 110 emergency call process, which includes the user's current location and recorded video data, allowing the police to immediately be notified of the situation at the scene.

[1286] Step 8: Emergency Notification

[1287] Subject: Terminal

[1288] The AI ​​analyzes the camera footage and audio in real time, and if it determines that the user is in danger, an emergency notification is sent to pre-defined contacts (such as family members). This notification includes a message saying "The user is currently in danger," along with location information and a link to the video data.

[1289] This series of processes improves the safety of the user and allows for a quick response in dangerous situations.

[1290] Example 1

[1291] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1292] In modern society, crime prevention measures are extremely important when people are out at night or alone. However, current crime prevention systems have difficulty notifying users of crime information and dangerous areas in real time, and they have limited means of taking prompt and appropriate action in emergencies. For these reasons, a system that can ensure even greater safety is needed.

[1293] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1294] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, and means for monitoring the user's location information and notifying the user when the user approaches the risk area. This allows the user to receive real-time notifications when the user approaches a risk area. The server also includes means for starting video and audio recording in response to a user's instruction, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results, thereby enabling the user to respond quickly and appropriately when they encounter an unexpected situation. The system includes a means for monitoring a user's location information in real time and detecting approach to a dangerous area, a means for sending a warning notification as a push notification, a means for detecting vibrations using an acceleration sensor in the security system and controlling an LED light when the vibration exceeds a threshold, a means for storing the generated video data in a cloud service, a means for sending an emergency signal to a smart device using Bluetooth or Wi-Fi Direct, a means for using a telephone API to automatically make an emergency call, and a means for sending an emergency notification using SMS or a notification API. This allows users to act safely even when out at night or in isolated situations, and enables quick response in the event of an emergency.

[1295] "Means of collecting crime information and crime prevention information from all over the country" refers to the function of regularly obtaining crime information and crime prevention information provided by police stations and public institutions from all over the country and storing it in a database.

[1296] "Means for analyzing the collected information and identifying risk areas" refers to the function of analyzing collected crime information and crime prevention information and identifying crime trends and high-risk areas.

[1297] "Means for monitoring the user's location information and notifying when the user approaches the dangerous area" refers to a function that monitors the location information sent from the user's smart device and sends a warning notification when the user approaches the dangerous area.

[1298] The term "means for starting video and audio recording in response to a user's instruction" refers to a function that allows a user to start video and audio recording by operating an application.

[1299] "Means for detecting vibrations and impacts" refers to the function of detecting vibrations and impacts using an acceleration sensor built into the security device.

[1300] "Means for emitting light when vibration is detected" refers to the function of illuminating the security device's LED light at maximum brightness when vibration is detected by the acceleration sensor.

[1301] "Means for automatically making an emergency call when an impact is detected" refers to a function that automatically makes an emergency call when a strong impact is detected by the acceleration sensor.

[1302] "Means for analyzing generated video and audio data" refers to the function of analyzing video and audio data recorded by a user's smart device.

[1303] "Means for sending an emergency notification based on the analysis results" refers to the function in which the generative AI model analyzes video and audio data and sends a notification if it determines that an emergency exists.

[1304] "Means for monitoring the user's location information in real time and detecting when the user approaches a dangerous area" refers to a function for monitoring the user's location information in real time and detecting when the user approaches a dangerous area.

[1305] "Means for sending a warning notification as a push notification" refers to a function that sends a warning using a push notification when a user approaches a dangerous area.

[1306] "Means for detecting vibrations using an acceleration sensor in a security system and controlling an LED light when the vibrations exceed a threshold" refers to a function that detects vibrations using an acceleration sensor and controls an LED light to turn on when the vibrations exceed a certain threshold.

[1307] "Means for storing generated video data in a cloud service" refers to a function for storing video data recorded on a user's device in a cloud service.

[1308] "Means for transmitting emergency signals to a smart device using Bluetooth or Wi-Fi Direct" refers to the function of transmitting emergency signals to a user's smart device using Bluetooth or Wi-Fi Direct.

[1309] "Means of using a telephone API to automatically make an emergency call" refers to a function that automatically makes an emergency call using a telephone API when an emergency signal is received.

[1310] "Means of sending emergency notifications using SMS or notification APIs" refers to the ability to send emergency notifications to pre-defined contacts using SMS or notification APIs.

[1311] This invention is a security system for ensuring personal safety, primarily when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. It also automatically starts recording when it detects vibration or impact, and automatically reports and notifies the user in an emergency.

[1312] Server Operation

[1313] The server regularly collects crime and crime prevention information provided by police stations and public institutions across the country. The collection process uses an API, including a program that sends HTTP requests to retrieve data in JSON format and stores it in a database. The server analyzes the collected data and runs machine learning algorithms to identify crime trends and high-risk areas. The results are stored in a database, providing constantly up-to-date information.

[1314] The server also receives location information periodically sent from the user's smartphone and monitors it in real time. When the user approaches a risky area, the server immediately generates a warning notification and sends it to the user's smartphone using a notification service such as Firebase Cloud Messaging (FCM). The specific notification message may say, "You are currently approaching a high-risk area."

[1315] User smartphone behavior

[1316] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. This action calls the smartphone's camera API and microphone API, which starts recording the surrounding video and audio. The recorded data is saved in the device's internal storage or on a cloud service.

[1317] Walking Recorder Operation

[1318] The Walking Recorder constantly detects vibrations using a built-in accelerometer. If a strong vibration is detected, the system immediately turns on the LED light at maximum brightness. This is achieved by acquiring vibration data using the sensor API and calling the LED light control API when a threshold is exceeded. If an even stronger impact is detected, the Walking Recorder will send an emergency signal to your smartphone. This is done using Bluetooth or Wi-Fi Direct.

[1319] Handling emergency calls and notifications

[1320] When a smartphone receives an emergency signal, it automatically makes an emergency call using a phone API according to pre-defined emergency protocols. This call includes the user's current location and recorded video. If the generative AI model analyzes the video and audio and determines that the user is in danger, it sends a notification to pre-defined emergency contacts via SMS or a notification API. The notification includes the user's current location and a link to the video data.

[1321] Specific examples

[1322] For example, if a user is walking down a street at night, the server detects that they are approaching a dangerous area and sends a warning notification to their smartphone saying, "You are approaching a high-risk area." The user confirms the notification and taps the recording button to begin recording. If a suspicious person approaches and grabs the user's shoulder, the Walking Recorder detects the strong vibration and immediately illuminates its LED light. If an even stronger impact is applied, the Walking Recorder sends an emergency signal to the smartphone, which then automatically reports the incident and sends the video data and current location to the police. The generative AI model then analyzes the situation and sends an emergency notification to the user's family.

[1323] Prompt Sentence Examples

[1324] Could you please explain the details of the nighttime security system? Location monitoring, danger zone notification, sudden vibration and shock detection, video and audio recording start, automatic reporting, and emergency notification.

[1325] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1326] Step 1:

[1327] server

[1328] The server collects crime and crime prevention information provided by police stations and public institutions across the country. Specifically, the server periodically sends HTTP requests to the API endpoint and retrieves data in JSON format. This data includes information such as the date, time, location, and type of crime. The collected data is stored in a database and used for subsequent analysis. (Input): Response data from the API. (Output): Crime information stored in the database.

[1329] Step 2:

[1330] server

[1331] The server analyzes the collected crime information using a machine learning algorithm to identify high-risk areas. Specifically, it applies a clustering algorithm (e.g., K-means algorithm) based on past crime data to identify areas where crimes are concentrated. The results of this analysis are stored in a database and used to match users' location information. (Input): Crime information stored in the database. (Output): High-risk areas identified through analysis.

[1332] Step 3:

[1333] server

[1334] The server receives real-time location information periodically sent from the user's smartphone. The received location information is sent to the server as GPS data, and monitoring is performed based on that data. If the user's current location approaches a high-risk area, the server generates a warning notification. (Input): GPS data from the smartphone. (Output): Generated warning notification.

[1335] Step 4:

[1336] server

[1337] The server sends the generated warning notification to the user's smartphone. Specifically, it sends a push notification using a notification service such as Firebase Cloud Messaging (FCM). This notification contains the message "You are currently approaching a high-risk area." (Input): Warning notification generated by the server. (Output): Push notification sent to the user's smartphone.

[1338] Step 5:

[1339] User

[1340] The user checks the warning notification displayed on their smartphone and taps the recording button in the smartphone app. The smartphone app calls the camera API and microphone API to begin recording video and audio. This recorded data is saved in the internal storage or on a cloud service. (Input): User's tap operation. (Output): Video and audio data saved on the smartphone.

[1341] Step 6:

[1342] Terminal

[1343] The Walking Recorder constantly detects vibrations using a built-in acceleration sensor. If a strong vibration is detected, the LED light will turn on at maximum brightness to scare off suspicious people. The sensor API is used to obtain vibration data, and if the vibration exceeds a threshold, the LED control API is called to turn on the light. (Input): Vibration data from the acceleration sensor. (Output): The lit LED light.

[1344] Step 7:

[1345] Terminal

[1346] If the Walking Recorder detects another strong impact, it immediately sends an emergency signal to your smartphone. The emergency signal is transmitted using Bluetooth or Wi-Fi Direct. This signal contains information about the strength of the impact and your location. (Input): Strong impact data from the acceleration sensor. (Output): Emergency signal sent to your smartphone.

[1347] Step 8:

[1348] Smartphone

[1349] When the smartphone receives an emergency signal, it automatically calls 110 using the phone API according to the pre-set emergency protocol. The call includes the user's current location and real-time video recording. (Input): Emergency signal from the walking recorder. (Output): Current location and video data sent along with the automatic call.

[1350] Step 9:

[1351] Generative AI Models

[1352] The generating AI sequentially analyzes camera footage and audio, and if it determines that the user is in danger, it sends an emergency notification to pre-set contacts such as family and close friends. Specifically, it uses a notification API and SMS service to send a message saying "The user is currently in danger," along with location information and a link to the video data. (Input): Camera footage and audio data. (Output): Emergency notification sent to family and pre-set contacts.

[1353] (Application example 1)

[1354] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1355] Ensuring personal safety when out at night is extremely important, but conventional security systems often lack sufficient risk area notification and emergency response capabilities, requiring users to operate the system directly. Furthermore, users lack a means to receive visual information, making it difficult to quickly recognize danger. Furthermore, automation of emergency calls and video and audio recording is insufficient, potentially preventing effective action even in situations requiring a rapid response.

[1356] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1357] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country and monitoring user location information, means for analyzing the collected information to identify risk areas, means for notifying the user when they approach a risk area, and means for starting video and audio recording in response to a user's instruction.In this invention, by providing an easy-to-use device and interface, it is possible to quickly and effectively recognize danger and take appropriate action, by detecting vibrations and impacts using a built-in sensor and emitting light to automatically start video and audio recording, means for analyzing the generated video and audio data and sending an emergency notification based on the analysis results, means for providing a visual display device with a notification function to warn the user in real time, and means for implementing an interface function for the user to start video and audio recording.

[1358] "Crime information and crime prevention information" refers to data on crime occurrence and prevention measures provided by police stations and public institutions across the country.

[1359] "Risk areas" refer to areas that have been identified as having a high risk of crime occurring based on the results of an analysis of collected crime and crime prevention information.

[1360] "Location Information" means data that indicates a user's current geographic location, obtained through GPS or other location-determining technology.

[1361] "Notification" refers to the action of displaying or sending a warning or information to a user's device when the user approaches a dangerous area.

[1362] "Video and audio recording" refers to the act of activating the camera and microphone, either at the user's command or automatically, to record video and audio of the surrounding area.

[1363] "Vibration and shock" refers to physical shaking or impact detected by an accelerometer.

[1364] "Emitting light" refers to the act of turning on an LED light or similar as a visual warning when vibration is detected.

[1365] "Emergency call" refers to the action of automatically notifying pre-defined contacts or the police to request help.

[1366] "Analyzing the generated video and audio data" refers to the process of analyzing video and audio data using AI models, etc., to determine dangerous situations.

[1367] "Send emergency notification" refers to the action of notifying set contacts, such as family and friends, of the user's situation based on the analysis results.

[1368] "Visual display device" refers to a device (e.g., smart glasses) worn by a user that displays information in real time.

[1369] The "interface function" refers to a function that provides an operation method for the user to start video and audio recording.

[1370] This invention is a security system to ensure the safety of users when they are out at night. It collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies them when they approach a dangerous area. It also provides a series of processes that automatically start recording and recording when it detects vibration or impact, and automatically reports and notifies users in the event of an emergency.

[1371] System configuration

[1372] The system consists of the following components:

[1373] 1. Server:

[1374] Collection of crime and crime prevention information: Data provided by police stations and public institutions across the country is periodically obtained via API. The server analyzes this data to identify crime trends and high-risk areas.

[1375] Monitoring user location information: The server periodically receives location information from the user's smart device and sends a warning notification to the user if the device approaches a dangerous area.

[1376] 2. User terminal (smart glasses):

[1377] Video and audio recording function: When the user confirms the alert notification, the smart glasses' built-in camera and microphone will activate and record the surrounding video and audio. The recorded data will be saved in the internal storage or cloud service.

[1378] Vibration and impact detection: The smart glasses' built-in acceleration sensor constantly detects vibrations, and when a strong vibration is detected, the LED light will turn on. At the same time, if a strong impact is detected, an emergency signal will be automatically sent to the server.

[1379] Automatic notification function: When the smart glasses receive an emergency signal, they will automatically make an emergency call according to the emergency protocol pre-set on the server side. The call will include the user's current location information and real-time video recording data.

[1380] Emergency Notification: If the generative AI model analyzes camera footage and audio and determines that the user is in danger, it will send an emergency notification to pre-defined contacts, including location information and a link to the video recording.

[1381] Hardware and Software Details

[1382] Hardware:

[1383] Smart Glasses: Uses a display (e.g., smart glasses) that displays information in real time.

[1384] Acceleration sensor: Built-in sensor for detecting vibration and impact.

[1385] High-resolution camera and microphone: Equipped with a device for recording video and audio of the surroundings.

[1386] LED light: Visual warning when vibration is detected.

[1387] software:

[1388] Server: Uses cloud services such as AWS and Google Cloud Platform to collect and analyze data and execute emergency protocols.

[1389] Generative AI model: Analyzes video and audio data using OpenAI GPT-4 and other models to determine emergency situations.

[1390] Notification app: An application for iOS and Android devices that monitors the user's location, controls recording and video recording, and sends emergency notifications.

[1391] Specific examples

[1392] A user puts on smart glasses while walking down a street at night. The server detects that the user is approaching a dangerous area, and a warning message is displayed on the smart glasses. The user confirms the warning and starts recording. If a suspicious person approaches and grabs the user's shoulder, the smart glasses detect a strong vibration, turn on an LED light, and automatically start recording. If the suspicious person causes further harm, the generating AI analyzes the situation and makes an emergency call via the server. In addition, an emergency notification is sent to family and friends.

[1393] Example prompts for generative AI models

[1394] "I would like to develop the following security application. A system in which users wear smart glasses and are warned of approaching dangerous areas when going out at night. The glasses can record video and audio using the camera and microphone, and will turn on an LED light if a suspicious person comes into contact with the user. If vibration or impact is detected, the glasses will automatically report the user and send an emergency notification."

[1395] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1396] Step 1:

[1397] The server collects crime and crime prevention information from all over the country. The input is data provided by police stations and public institutions. This data is obtained via API and stored in a database on the server. The output is a database of the collected crime and crime prevention information.

[1398] Step 2:

[1399] The server analyzes the collected crime and crime prevention information and identifies risk areas. The input is crime information in the database. The server performs data analysis to identify high-risk areas using statistical models and machine learning algorithms. The output is information on areas identified as high-risk areas.

[1400] Step 3:

[1401] The server receives location information periodically sent from the user's smart glasses. The input is the user's location information. The server monitors this information in real time, and if it determines that the user is approaching a dangerous area, it sends a warning notification to the user's smart glasses. The output is a warning message displayed on the user's smart glasses.

[1402] Step 4:

[1403] The user confirms the warning notification and taps the video / audio button on the smart glasses. The input is the warning notification. This action activates the camera and microphone on the smart glasses and starts recording the surrounding video and audio. The output is the video and audio data.

[1404] Step 5:

[1405] The smart glasses' accelerometer constantly detects vibrations. The input is the accelerometer data. If a strong vibration is detected, the smart glasses will turn on the LED light to scare off suspicious people. The output is the LED light being turned on and the glasses will continue recording.

[1406] Step 6:

[1407] If a strong impact is detected again, the smart glasses will automatically send an emergency signal to the server. The input is the impact data from the acceleration sensor. The output is the emergency signal sent to the server.

[1408] Step 7:

[1409] When the server receives an emergency signal, it automatically makes an emergency call according to a pre-defined emergency protocol. The input is the emergency signal and the user's current location information. The call includes the user's current location information and real-time video recording data. The output is a call to 110.

[1410] Step 8:

[1411] The generative AI model sequentially analyzes camera footage and audio and determines whether the user is in a dangerous situation. The input is video and audio data. The output is the analysis result that determines whether the situation is an emergency.

[1412] Step 9:

[1413] The server sends emergency notifications to pre-defined contacts, such as family members and close friends, based on the analysis results of the generative AI model. The inputs are the analysis results, the user's location information, and a link to the video recording data. The output is an emergency notification message sent to the emergency contacts.

[1414] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1415] This invention is a crime prevention system for ensuring personal safety, primarily when out at night, that collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, this system uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[1416] Program processing

[1417] Collection and notification of crime prevention information

[1418] server

[1419] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. This data is collected and machine learning algorithms are used to calculate crime trends and risk levels for each area. Identified risk areas are stored in a database and updated in real time.

[1420] User location monitoring and notification

[1421] server

[1422] The server receives location information periodically sent from the user's smartphone. The user's smartphone identifies its location using GPS and periodically sends the location data to the server. When the user approaches a dangerous area, a warning notification is sent to the smartphone.

[1423] Emotion recognition by emotion engine

[1424] Terminal

[1425] The emotion engine recognizes emotions such as fear and anxiety by monitoring biometric indicators such as the user's heart rate and galvanic skin response. These biometric indicators are acquired through a wearable device and transmitted to the emotion engine.

[1426] Automatic response through emotion recognition

[1427] Terminal

[1428] If the emotion engine detects that the user is feeling fear or anxiety, the following steps are automatically performed:

[1429] 1. Start recording and audio: When the emotion engine recognizes the user's fear or anxiety, the walking recorder will automatically start and record video and audio of the surrounding area.

[1430] 2. Emergency Call: The emotion engine recognizes the user's fear and anxiety and, if it determines that a crisis is occurring, automatically initiates a call to 110. The call will include the user's current location and recorded video.

[1431] 3. Sending emergency notifications: If the emotion engine determines that the user is experiencing fear or anxiety, it will immediately send an emergency notification to pre-defined contacts (e.g., family members). The notification will include the message "The user is in danger," along with location information and a link to the video data.

[1432] Specific examples

[1433] Night Out Scenario

[1434] When a user is walking on a street at night, the server monitors the user's current location and detects that they are approaching a dangerous area, sending a warning notification to their smartphone. The user receives the notification, but begins to feel anxious or scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the walking recorder detects strong vibrations and immediately lights up an LED light. Furthermore, if the user feels extreme fear or is shocked, the emotion engine immediately calls 110 and sends an emergency notification to their family.

[1435] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[1436] The processing flow will be explained below.

[1437] Step 1: Gather crime prevention information

[1438] Subject: Server

[1439] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API, and this data is stored in the server's database.

[1440] Step 2: Data analysis

[1441] Subject: Server

[1442] The server analyzes the collected crime and crime prevention information, and uses machine learning algorithms to calculate crime trends and risk levels for each area. The analysis results are stored in a database.

[1443] Step 3: Location Monitoring

[1444] Subject: Server

[1445] The server receives location information periodically sent from the user's smartphone. The smartphone uses GPS to identify the user's location and sends that data to the server.

[1446] Step 4: Risk Area Notification

[1447] Subject: Server

[1448] When the server detects that the user is approaching a high-risk area based on the user's current location and the analysis results, it sends a danger notification to the user's smartphone. The notification includes a message saying, "You are currently approaching a high-risk area."

[1449] Step 5: Emotion Recognition with the Emotion Engine

[1450] Subject: Terminal

[1451] The emotion engine built into the device receives biometric data such as the user's heart rate and skin galvanic response from the wearable device, analyzes this data, and evaluates the user's emotional state (especially fear and anxiety) in real time.

[1452] Step 6: Start recording and recording with emotion recognition

[1453] Subject: Terminal

[1454] When the emotion engine recognizes the user's fear or anxiety, the device automatically starts recording and audio recording. The Walking Recorder's camera and microphone are activated to capture video and audio.

[1455] Step 7: Emotion-Recognition Automatic Reporting

[1456] Subject: Terminal

[1457] If the emotion engine recognizes the user's fear or anxiety, and detects a strong impact or fall, the device will automatically call 110. The call will include the user's current location and recorded video.

[1458] Step 8: Send emergency notifications

[1459] Subject: Terminal

[1460] If the emotion engine detects that the user is in danger, the device will immediately send an emergency notification to pre-defined contacts, containing the message "The user is in danger," along with location information and a link to the video data.

[1461] Specific examples

[1462] Night Out Scenario

[1463] When a user is walking on a street at night, the server monitors the user's current location and sends a warning notification to their smartphone if it detects that they are approaching a dangerous area. The user receives the notification, and the emotion engine begins monitoring their heart rate and skin galvanic response. If the user begins to feel anxious, the emotion engine recognizes this and the walking recorder automatically begins recording and audio recording. If a suspicious person approaches, the device detects vibrations and impacts and automatically reports the incident. An emergency notification is also sent to family members.

[1464] Example 2

[1465] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1466] Personal safety when out at night is an important issue in modern society. To address this issue, a system that can detect danger in real time and respond quickly is needed. However, current security systems do not take the user's emotional state into account, making it difficult to respond appropriately. There is also a need for a system that can automatically respond when the user feels fear or anxiety.

[1467] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting crime information and crime prevention information from all over the country; means for analyzing the collected information and identifying risk areas; means for monitoring the user's location information and notifying the user when the user approaches the risk area; means for collecting the user's biometric indicators and analyzing their emotional state; means for starting video and audio recording when it is determined that the user is feeling fear or anxiety; means for monitoring the biometric indicators in real time and automatically making an emergency call when fear or anxiety is detected; means for detecting vibrations and impacts; means for emitting light when the vibrations are detected; means for analyzing the generated video and audio data; and means for sending an emergency notification based on the analysis results. This enables danger to be detected in real time and appropriate responses taking into account the user's emotional state.

[1468] "Means for collecting crime information and crime prevention information nationwide" refers to the means for collecting data on the number of crimes and crime prevention information obtained from each region of the country and storing it on a server.

[1469] "Means for analyzing and identifying risk areas" refers to means for analyzing collected crime information and crime prevention information and identifying areas where there is a high risk of crime occurring.

[1470] The "means for monitoring the user's location information and notifying when the user approaches the risk area" is a means for monitoring the user's current location using GPS and sending a warning notification when the user approaches an identified risk area.

[1471] "Means for collecting biometric indicators of a user and analyzing their emotional state" refers to means for analyzing a user's emotional state based on data such as the user's heart rate and skin galvanic response obtained from a wearable device or the like.

[1472] The "means for starting video and audio recording when it is determined that the user is feeling fear or anxiety" refers to a means for automatically activating the smartphone's camera and microphone to record video and audio of the surrounding area when the emotion engine determines that the user's emotional state is fear or anxiety.

[1473] The "means for automatically making an emergency call when fear or anxiety is detected" is a means for automatically making an emergency call when fear or anxiety is detected from the biometric indicators of the user.

[1474] The "means for detecting vibration and impact" is a means for detecting vibration or impact when the user receives vibration or impact from the outside.

[1475] The "means for emitting light when vibration is detected" is a means for notifying the surroundings by immediately emitting light based on the detected vibration.

[1476] The "means for analyzing the generated video and audio data" refers to a means for analyzing the video and audio data that has been recorded and detecting abnormalities or emergencies.

[1477] The "means for sending an emergency notification based on the analysis result" is a means for sending an emergency notification to the set contacts when an emergency is recognized based on the analysis result.

[1478] This invention is a crime prevention system for ensuring personal safety when out at night. This system collects crime and crime prevention information from all over the country in real time, monitors the user's location, and notifies the user when they approach a dangerous area. Furthermore, it uses an emotion engine to recognize the user's emotions and takes emergency action if the user feels fear or anxiety.

[1479] Collection and analysis of crime prevention information

[1480] server

[1481] The server periodically obtains crime and crime prevention information provided by police stations and public institutions across the country via API. For example, it uses government-provided public safety APIs and private crime data providers. The collected data is stored in a database, typically MySQL or PostgreSQL.

[1482] The server uses machine learning algorithms such as TensorFlow and Scikit-learn to analyze crime trends and risk levels in each area and identify risk areas, which are updated in real time.

[1483] Collection and monitoring of user location information

[1484] Terminal

[1485] The user's smartphone periodically obtains location information using the GPS sensor and the smartphone's standard location services (such as Google Location Services).

[1486] The smartphone periodically sends the acquired location information to the server. The HTTPS protocol is used for transmission, ensuring secure data transmission.

[1487] server

[1488] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area, and maps the user's current location using a map service such as Google Maps API.

[1489] If the user is determined to be approaching a dangerous area, a warning notification is sent to the smartphone. The notification is sent using a notification service such as Firebase Cloud Messaging (FCM).

[1490] Biometric data collection for emotion recognition

[1491] Terminal

[1492] Users wear a wearable device such as an Apple Watch or Fitbit to periodically measure vital signs such as heart rate and galvanic skin response, and this data is sent to a smartphone via Bluetooth.

[1493] The smartphone transmits the received biometric data to the emotion engine.

[1494] Automatic response through emotion recognition

[1495] Terminal

[1496] The emotion engine uses libraries such as OpenCV and DeepFace to analyze biometric data and determine whether the user is feeling fear or anxiety.

[1497] 1. If the user's state is judged to be fearful or anxious, the smartphone's camera and microphone will automatically activate and record video and audio of the surrounding area.

[1498] 2. Additionally, the emotion engine uses the Twilio API to automatically place emergency calls, including current location information and recorded video.

[1499] 3. It also uses SendGrid or Firebase Cloud Messaging (FCM) to send emergency notifications to configured contacts, including a message saying "The user is in danger," along with location information and a link to video data.

[1500] Specific examples of operation

[1501] When a user is walking down a street at night, the server monitors the user's current location and sends a warning notification to the smartphone if it detects that the user is approaching a dangerous area. The user receives the notification, but begins to feel anxious and scared. The emotion engine recognizes this and automatically starts recording and recording. If a suspicious person approaches, the smartphone's camera will record video and the microphone will record audio. The emotion engine also analyzes the user's condition and, if it determines that the situation is critical, will immediately call emergency services and send an emergency notification to family members.

[1502] Prompt Sentence Examples

[1503] "Please explain the process of a security system that sends a warning notification to the user's smartphone when the user approaches a dangerous area while out at night, and automatically starts video and audio recording if the emotion engine recognizes fear or anxiety."

[1504] In this way, the present invention can take the user's emotional state into consideration, allowing for more accurate and prompt crime prevention measures.

[1505] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1506] Step 1:

[1507] Collection of crime prevention information

[1508] The server periodically obtains nationwide crime and crime prevention information via API. As input, it receives data from APIs provided by police stations and public institutions. The server analyzes this data and stores it in a database. Specifically, it uses public safety APIs provided by the government and private crime data providers. The output is data in a format that stores the number of crimes and crime prevention information for each region in a database.

[1509] Step 2:

[1510] Crime information analysis

[1511] The server analyzes the stored crime information using machine learning algorithms. It uses the crime information stored in the database as input. It calculates crime trends and risk levels for each area using TensorFlow, Scikit-learn, etc. The output is data showing the crime risk level for each area. The analysis results are then stored back in the database, and risk areas are identified.

[1512] Step 3:

[1513] Collection of user location information

[1514] The user's smartphone periodically acquires location information using a GPS sensor. The input is GPS data, which is acquired using the smartphone's location service (e.g., Google Location Services). The acquired location information is sent to the server using the HTTPS protocol. The output is the user's current location data sent to the server.

[1515] Step 4:

[1516] Location monitoring and notifications

[1517] The server analyzes the received location information and determines in real time whether the user is approaching a dangerous area. The input is the user's current location data, which is mapped on a map using Google Maps API. If the server determines that the user is approaching a dangerous area, it sends a warning notification to the smartphone. The output is the warning notification sent to the smartphone.

[1518] Step 5:

[1519] Biometric data collection

[1520] A user wears a wearable device and periodically measures their heart rate and electrodermal response. The input is the biometric data measured by the wearable device. For example, a device such as an Apple Watch or Fitbit collects this data and transmits it to a smartphone via Bluetooth. The output is the biometric data transmitted to the smartphone.

[1521] Step 6:

[1522] Emotional state analysis

[1523] The smartphone sends the received biometric data to the emotion engine. The input is the biometric data sent from the wearable device. The emotion engine analyzes this data using libraries such as OpenCV and DeepFace to determine whether the user is feeling fear or anxiety. The output is the user's emotional state data as a result of the analysis.

[1524] Step 7:

[1525] Start of video and audio recording

[1526] When the emotion engine recognizes the user's fear or anxiety, it automatically activates the smartphone's camera and microphone. The input is emotional state data, and it is determined that the user is feeling fear or anxiety. Video and audio recording is started, and surrounding video and audio are recorded. The output is the video and audio data.

[1527] Step 8:

[1528] Handling emergency calls

[1529] If the emotion engine recognizes the user's fear or anxiety and determines that the situation is critical, the server automatically makes an emergency call. The input is recorded video and audio data, as well as current location information. Using the Twilio API, this data is included in the call content. The output is data indicating that an emergency call has been made.

[1530] Step 9:

[1531] Sending emergency notifications

[1532] The server sends an emergency notification based on the analysis results. The input is the emotional state data as the analysis result, the video and audio data recorded, and the current location information. The emergency notification is sent to the specified contacts (e.g., family members) using SendGrid or Firebase Cloud Messaging (FCM). The output is the emergency notification sent to the specified contacts.

[1533] (Application example 2)

[1534] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1535] Conventional security systems only have the function of notifying users when they approach a dangerous area, but do not take into account the user's emotional state or biometric data to respond immediately. This can result in a delay in appropriate response when the user senses danger. In addition, video and audio recording must be started manually, which can lead to situations where the user cannot operate the system in an emergency, making effective security measures ineffective.

[1536] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1537] In this invention, the server includes means for collecting crime information and crime prevention information from all over the country, means for analyzing the collected information and identifying risk areas, means for monitoring the user's location information and notifying the user when the user approaches the risk area, means for recognizing the user's emotions by monitoring the user's heart rate and galvanic skin response, means for automatically starting video and audio recording when fear or anxiety is recognized, means for detecting vibrations and impacts, means for emitting light when the vibrations are detected, means for automatically making an emergency call when the impact is detected, means for analyzing the generated video and audio data, and means for sending an emergency notification based on the analysis results. This allows the user's emotional state to be reflected in real time, enabling quick and accurate crime prevention measures.

[1538] "Crime information" refers to the number of crimes occurring across the country and detailed information about crimes, and is data provided by police stations and public institutions.

[1539] "Crime prevention information" is data that includes information on crime prevention measures and warning information about areas where danger is expected, and is information to ensure safety and security.

[1540] "Risk areas" refer to areas where crimes are frequent or where caution is required for crime prevention, and are identified based on collected crime and crime prevention information.

[1541] "Location Information" means data that indicates a user's current geographic location, obtained using GPS or other location-determining technology.

[1542] An "emotion engine" is software or algorithm that analyzes biometric indicators such as a user's heart rate and galvanic skin response to recognize the user's emotional state.

[1543] "Video and audio recording" refers to the function of recording video and audio using a camera or microphone, and is used to generate evidence for crime prevention purposes.

[1544] "Vibration" is a physical movement caused by a device being shaken or struck by a user, and is detected by a sensor.

[1545] "Impact" is a physical action caused by the application of sudden force such as strong vibration or impact, and is detected by a sensor.

[1546] "Emergency Notification" is a feature that instantly notifies appropriate authorities, such as police, of a crisis situation and is performed automatically when certain events occur.

[1547] "Generated video and audio data" refers to video and audio data recorded by the video and audio recording functions.

[1548] "Emergency Notification" is a feature that allows you to quickly notify designated contacts (e.g., family and friends) if you are in a dangerous situation.

[1549] This invention is implemented as a security application called "Safe Night Guardian" that is installed on smartphones. This system ensures the safety of users who go out at night or other unsafe times. It collects and analyzes crime and crime prevention information from all over the country in real time, and links the user's current location and emotional state to enable quick response.

[1550] Hardware used

[1551] 1. Smartphones: Uses location information (GPS), camera and microphone. Examples: Apple iPhone and Samsung Galaxy.

[1552] 2. Wearable devices: monitor heart rate and galvanic skin response. Examples: Fitbit, Apple Watch.

[1553] Software used

[1554] 1. Server side:

[1555] AWS EC2: Cloud server for running programs.

[1556] AWS RDS: Database management for crime and crime prevention information.

[1557] Python: Data processing and machine learning algorithm implementation.

[1558] Scikit-Learn: A machine learning library for analyzing crime trends.

[1559] 2. Client side:

[1560] Android / iOS app: User interface and location data collection.

[1561] TensorFlow: A machine learning platform for building emotion engines.

[1562] Firebase Cloud Messaging: Real-time notifications to users.

[1563] 3. Notification system:

[1564] Twilio API: For emergency calling and notification.

[1565] System operation explanation

[1566] Collection and analysis of crime prevention information

[1567] The server periodically receives crime and crime prevention information from police and public institutions across the country via API. Based on this data, machine learning algorithms (using Python and Scikit-Learn) are used to calculate crime trends and risk levels for each region. This information is stored in AWS RDS and updated in real time.

[1568] User location monitoring and notification

[1569] The smartphone's GPS function is used to obtain the user's location information, which is then periodically sent to the server. The server compares this location information with information about dangerous areas stored in a database, and sends a warning notification to the smartphone if the user approaches a dangerous area (using Firebase Cloud Messaging).

[1570] Emotion recognition by emotion engine

[1571] Biometric data such as heart rate and skin galvanic response is collected from wearable devices (e.g., Fitbit, Apple Watch), and analyzed by an emotion engine using TensorFlow. The emotion engine recognizes the user's state of fear or anxiety in real time and takes action, such as automatically starting video or audio recording, as needed.

[1572] Auto-response function

[1573] If the emotion engine detects that the user is feeling fear or anxiety, it will automatically:

[1574] 1. Automatically start recording and recording: Record the situation using your smartphone's camera and microphone.

[1575] 2. Emergency call: A call to 110 is made using the Twilio API, and the user's current location and recorded video are provided.

[1576] 3. Emergency notification: Notify pre-defined contacts (e.g., family members) that the user is in danger and send them location information and a link to video data.

[1577] Specific examples

[1578] When a user is walking down a street at night, the app monitors their location in the background and displays a warning if they approach a dangerous area. If the user feels anxious, the smartwatch detects an increase in heart rate and galvanic skin response. The emotion engine analyzes this and, if it recognizes a state of fear, automatically starts video and audio recording and simultaneously calls 110. An emergency notification is also sent to family members.

[1579] Example prompts to input to the generative AI model

[1580] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[1581] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1582] Step 1:

[1583] Collection of crime prevention information

[1584] The server regularly obtains crime and crime prevention information from police and public institutions across the country via API.

[1585] Input: Crime and crime prevention data provided by police departments and public agency APIs.

[1586] Output: The acquired data is saved on the server. The server saves the acquired data in a database (e.g. AWS RDS).

[1587] Step 2:

[1588] Crime information analysis

[1589] The server analyzes the collected crime and crime prevention data using machine learning algorithms (e.g., Python and Scikit-Learn) to calculate crime trends and risk levels for each area.

[1590] Input: Crime and crime prevention data stored in a database.

[1591] Output: Analysis report data showing crime trends and risk levels. The server updates this data and stores it as risk area information.

[1592] Step 3:

[1593] Location acquisition and monitoring

[1594] The device (smartphone) periodically obtains the user's current location using its GPS function and sends it to the server.

[1595] Input: User location data obtained from the smartphone's GPS.

[1596] Output: The user's current location information is sent to the server, which monitors the user's location in real time and prepares a warning notification if the user approaches a dangerous area.

[1597] Step 4:

[1598] Notification when approaching a dangerous area

[1599] The server compares the user's location information with the danger area information, and if the user approaches a danger area, it sends a warning notification to the device (smartphone).

[1600] Input: User's current location information and danger area information.

[1601] Output: A warning notification is sent to your smartphone. Use a notification system (e.g., Firebase Cloud Messaging) to display real-time warnings.

[1602] Step 5:

[1603] Biometric data acquisition and emotion recognition

[1604] A wearable device (e.g., Fitbit, Apple Watch) worn by the user captures heart rate and electrodermal response data and sends them to the emotion engine, which uses TensorFlow to analyze this data and recognize the user's emotional state (e.g., fear, anxiety).

[1605] Input: Biometric data such as heart rate and galvanic skin response obtained from wearable devices.

[1606] Output: The user's emotional state (e.g., fear, anxiety) as a result of the emotion engine's analysis. If the emotion engine detects fear or anxiety, it prepares the next step.

[1607] Step 6:

[1608] Automatic start of video and audio recording

[1609] When fear or anxiety is detected from the user's biometric data, the device (smartphone) automatically starts recording and recording audio using its camera and microphone.

[1610] Input: Signal from emotion engine (user's emotional state).

[1611] Output: Start recording. The device will record video and audio and save the data.

[1612] Step 7:

[1613] Making an emergency call

[1614] If the device's emotion engine detects fear or anxiety, it will automatically call 110 using an emergency notification system (e.g., Twilio API) and provide current location information and recorded video.

[1615] Input: Emotion engine signals, video and audio data, and current location information.

[1616] Output: Report, current location and recorded video sent to police. The device will automatically make an emergency call.

[1617] Step 8:

[1618] Sending emergency notifications

[1619] If the emotion engine detects fear or anxiety, the device will send an emergency notification to pre-defined contacts (e.g., family members) informing them that they are in danger, along with their current location and a link to the recorded video.

[1620] Input: Emotion engine signal, current location information, and recorded video link.

[1621] Output: Emergency notification sent to configured contacts. Notifications are sent to family members using Firebase Cloud Messaging.

[1622] Example prompts to input to the generative AI model

[1623] Please give us a concrete scenario of how the SafeNight Guardian app can help individuals stay safe at night. Please also explain in detail the automated response process using biometric data and emotion recognition from wearable devices.

[1624] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1625] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1626] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1627] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1628] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1629] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1630] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the...

Claims

1. A means of collecting crime information and crime prevention information from all over the country; means for analyzing the collected information and identifying risk areas; means for monitoring the user's location information and notifying the user when the user approaches the dangerous area; means for starting video and audio recording in response to a user's instruction; means for sensing vibration and shock; means for emitting light when the vibration is sensed; means for automatically making an emergency call when the impact is detected; means for analyzing the generated video and audio data; means for sending an emergency notification based on the analysis results; A system including:

2. 2. The system of claim 1, wherein the emergency call includes location information and video data.

3. 2. The system of claim 1, wherein the means for sending the emergency notification sends the emergency notification to a family member or a predefined contact.

Images