System

Abstract

To provide a system for improving the safety of an area by quickly and efficiently detecting and coping with abnormal behavior.SOLUTION: The system includes means for patrolling crime prone areas with drones, means for mounting sensors and generating AI to detect unusual behaviors and situations, means for alerting and advising upon detection of unusual situations, and means for reporting to police in emergencies.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, areas with high crime and fraud rates have required methods to detect abnormal behavior early and take appropriate countermeasures. However, there are limitations to mobilizing personnel for patrols and surveillance, and surveillance activities are particularly difficult in dangerous areas or over wide areas. Furthermore, existing security systems are not adequately able to detect abnormal behavior in real time and provide rapid countermeasures. For this reason, it is necessary to provide innovative methods for quickly and efficiently detecting and responding to abnormal behavior and improving local safety. [Means for solving the problem]

[0005] This invention provides a system that uses drones to patrol high-crime areas, detecting abnormal behavior and enabling rapid response. Specifically, the drone is equipped with sensors and generative AI, providing a means for detecting abnormal behavior and situations in real time. Furthermore, if an abnormality is detected, the drone automatically issues warnings and advice, and if necessary, notifies the police. Furthermore, by providing a means for collecting, analyzing, and reporting patrol data, the system ensures local safety more efficiently and effectively than conventional systems.

[0006] A "drone" is an unmanned aerial vehicle that flies remotely or autonomously to carry out various missions and tasks.

[0007] A "high-crime area" refers to a specific area where crimes statistically occur frequently.

[0008] "Patrol" refers to the activity of moving around a specific area on a regular or planned basis to monitor and observe.

[0009] "Sensor" refers to a device or component that detects physical or environmental data and outputs it as an electrical signal.

[0010] "Generative AI" refers to artificial intelligence that uses machine learning and deep learning techniques to analyze, predict, and classify data.

[0011] "Abnormal behavior" refers to actions or behavior that are out of the ordinary and may indicate criminal or fraudulent activity.

[0012] An "abnormal situation" is a situation or environment that is different from the norm and that may lead to crime or fraud.

[0013] "Warning" refers to the act of issuing a warning message or alarm to urge people to stop certain behavior.

[0014] "Advice" refers to providing appropriate responses or guidelines for a particular situation or behavior.

[0015] "Reporting to the police" refers to the act of informing police authorities of abnormal behavior or an emergency situation and requesting a response.

[0016] A "patrol schedule" refers to a plan that defines the areas and times that a drone should patrol.

[0017] "Data collection" refers to the acquisition and storage of observed information using sensors.

[0018] "Analysis" refers to the act of analyzing collected data to derive meaningful information and patterns.

[0019] "Action instructions" refer to commands from the server to the drone to perform specific actions.

[0020] "Real-time data" refers to data that is acquired and processed immediately, without delay.

[0021] "Location information" refers to data indicating the current location of a drone, and refers to information obtained from systems such as GPS.

[0022] "Data accumulation" refers to storing collected data and managing it for future analysis or reference.

[0023] "Analysis" refers to the process of examining collected data to find meaning and draw specific conclusions or predictions.

[0024] "Report" refers to the document that organizes and documents the collected and analyzed data and the results, and provides them to the relevant parties. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system mainly consists of four elements: a drone, a server, a terminal, and a user, all of which work in close coordination with each other.

[0047] System configuration

[0048] 1. Drones

[0049] The drone is equipped with various sensors such as a camera, audio sensor, and environmental sensor, as well as generative AI.

[0050] The drones automatically patrol designated crime-prone areas based on instructions from the server.

[0051] If the battery level gets low, it can automatically return and recharge.

[0052] 2. Server

[0053] The server is responsible for the overall chain of command and data analysis.

[0054] Generate a schedule for the patrol area and send it to the drone.

[0055] It receives data sent from the drone in real time and instructs appropriate actions based on the analysis results.

[0056] It also has a function to report to the police if abnormal behavior or situations are detected.

[0057] 3. Terminal

[0058] The device is equipped with generative AI that quickly analyzes data sent from the drone in real time.

[0059] Detects abnormal behavior and situations and reports the results to the server.

[0060] 4. Users

[0061] Users include system administrators, police operators, and local residents.

[0062] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[0063] Police operators receive emergency calls and respond quickly to the scene.

[0064] Program processing overview

[0065] Preparing and conducting patrols

[0066] The server generates a patrol schedule for high-crime areas based on data from the patrol area and sends it to the drone.

[0067] The drone automatically patrols a set route and collects data using various sensors.

[0068] The device analyzes the data in real time and detects abnormal behavior or situations.

[0069] Detecting and responding to abnormal behavior

[0070] If the device detects an abnormality, it immediately reports the situation to the server.

[0071] The server receives the report and analyzes the type and urgency of the abnormal behavior.

[0072] The server issues warnings and advice to the drone.

[0073] The drone approaches anyone behaving abnormally and uses audio messages and warning lights to warn them.

[0074] Emergency response

[0075] If the server determines that the emergency is high, it will automatically notify the police.

[0076] The server sends the drone's current location and real-time video data to the police.

[0077] The user (police operator) receives the report and arranges for a response at the scene.

[0078] Data Collection and Reporting

[0079] After the drone has completed its designated patrol area, it sends all collected data to the server.

[0080] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[0081] The server generates a report summarizing the analysis results and provides it to the user.

[0082] The user receives the report and takes any necessary measures or adjustments.

[0083] Specific examples

[0084] Example 1: Detecting suspicious activity in a parking lot at night

[0085] The server sends the drone a schedule for patrolling the parking lot at night.

[0086] As the drone patrols the parking lot, it captures several people trading something around a car.

[0087] The device analyzes the video data and determines that the transaction is suspicious.

[0088] The server analyzes the situation and issues instructions to the drone to warn it off.

[0089] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[0090] If it is determined that police are needed, the server will notify them and they will rush to the scene.

[0091] Example 2: Detecting suspected fraudulent calls

[0092] The server sends the drone a patrol schedule for areas prone to fraud.

[0093] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0094] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0095] The server analyzes the situation and provides advice and instructions to the drone.

[0096] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[0097] If necessary, the server will notify the police, who will respond to the scene.

[0098] This invention is designed to automate security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, thereby improving overall community safety and enhancing residents' sense of security.

[0099] The processing flow will be explained below.

[0100] Step 1:

[0101] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[0102] Step 2:

[0103] The server sends the generated patrol schedule to the drone.

[0104] Step 3:

[0105] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[0106] Step 4:

[0107] The server confirms that the drone is ready and sends a signal to activate the drone.

[0108] Step 5:

[0109] The drone will begin flying automatically, following the set patrol route.

[0110] Step 6:

[0111] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[0112] Step 7:

[0113] The data collected by the drone is sent to the device in real time.

[0114] Step 8:

[0115] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[0116] Step 9:

[0117] If the device detects an abnormality, it transfers the data and analysis results to the server.

[0118] Step 10:

[0119] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[0120] Step 11:

[0121] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[0122] Step 12:

[0123] The drone will play messages and turn on warning lights to deter abnormal behavior.

[0124] Step 13:

[0125] The server will reassess the situation and notify the police if necessary.

[0126] Step 14:

[0127] The server sends the drone's current location and real-time video data to the police.

[0128] Step 15:

[0129] The user (police operator) confirms the report and arranges for on-site response.

[0130] Step 16:

[0131] After the drone completes its patrol, it sends all collected data to a server.

[0132] Step 17:

[0133] The server stores the collected data and analyzes long-term crime trends and problem areas.

[0134] Step 18:

[0135] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[0136] Step 19:

[0137] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[0138] Example 1

[0139] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0140] Conventional security patrol systems were unable to efficiently patrol high-crime areas, making it difficult to detect and respond to abnormal behavior or situations in real time. Another issue was delays in reporting emergencies to the police, making it difficult to respond quickly.

[0141] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0142] In this invention, the server includes a means for generating a patrol schedule for high-crime areas based on patrol area data and transmitting it to the drone, a means for the drone to detect abnormal behavior or situations in real time using various sensors and generation AI and transmit that data to the terminal, and a means for issuing warnings or advice when an abnormal situation is detected. This enables highly efficient detection and response to abnormal behavior in real time.

[0143] "Patrol area" refers to a specific area or region in a high-crime neighborhood that will be patrolled by a drone.

[0144] A "drone" is an unmanned aerial vehicle that operates by remote control or automatic pilot and is equipped with a camera and various sensors.

[0145] A "server" is a computer that is responsible for the chain of command and management of the entire system, including collecting data, analyzing it, and sending instructions.

[0146] A "terminal" is a computer that receives and analyzes data sent from the drone.

[0147] "Generative AI" is part of a system that uses artificial intelligence technology to analyze data and make decisions, and refers to an algorithm that utilizes a generative model.

[0148] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is likely to be criminal or dangerous.

[0149] A "warning" is a warning or advice given to discourage certain behavior.

[0150] An "emergency" is a situation in which a serious danger or crime is in progress and requires a rapid response.

[0151] "Reporting" refers to the act of informing the police or other relevant authorities of abnormal behavior or an emergency.

[0152] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system is mainly composed of four elements: a server, a terminal, a drone, and a user, and each element works in close coordination with the others.

[0153] Hardware and software used

[0154] Server: A computer with high-performance data analysis capabilities and the processing power to operate generative AI models. Specifically, it is a server equipped with a large-capacity hard disk and a high-speed processor.

[0155] Device: A computer that receives and analyzes data in real time and runs the generative AI model. This device can be a laptop or a high-performance desktop computer.

[0156] Drone: An unmanned aerial vehicle equipped with cameras, audio sensors, and environmental sensors, with autopilot capabilities, a battery management system, and the ability to return automatically when it needs to be recharged.

[0157] Generative AI model: An artificial intelligence algorithm for detecting abnormal behavior or situations. For example, TensorFlow is used for visual analysis, and PyTorch is used for audio analysis.

[0158] Program processing

[0159] 1. The server collects data on the patrol area and generates a patrol schedule using a generative AI model. The generated schedule includes prompts such as "Patrol the parking lot at night and detect suspicious behavior."

[0160] 2. The server sends the generated patrol schedule to the drone and issues patrol instructions to the drone.

[0161] 3. The drone patrols the set area on autopilot, collecting video data from the camera, audio data from the audio sensor, and temperature and humidity data from the environmental sensors.

[0162] 4. The data collected by the drone is sent to the terminal in real time via the drone's communication module using Wi-Fi or 4G / 5G networks.

[0163] 5. The device quickly analyzes the data it receives using a generative AI model, inputs a prompt such as "Analyze the collected data and detect any abnormalities," and reports any abnormal behavior or abnormal situations to the server.

[0164] 6. The server receives the report and evaluates the type of abnormal behavior (e.g., voyeurism, violent behavior) and the urgency.

[0165] 7. The server issues warnings and advice to the drone, such as "play a warning message" or "turn on a warning light."

[0166] 8. The drone will play a specified voice message (e.g., "Misconduct in this area is prohibited. Please leave immediately.") and turn on a warning light.

[0167] 9. If the server determines that the abnormal behavior is of high urgency, it will automatically notify the police. The notification process will send the drone's current location, real-time video data, and detailed information about the abnormal behavior.

[0168] 10. After the drone has completed its designated patrol area, it will send all collected data to the server.

[0169] 11. The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[0170] 12. The server generates a report summarizing the analysis results and provides it to the user, who then takes necessary measures and makes adjustments based on the report.

[0171] Specific examples of programs

[0172] 1. Detecting suspicious activity in parking lots at night:

[0173] The server sends the drone a schedule for patrolling the parking lot at night.

[0174] As the drone patrols the parking lot, it captures several people conducting transactions around a car.

[0175] The device analyzes the video data and determines that the transaction is suspicious.

[0176] The server analyzes the situation and issues instructions to the drone to warn it off.

[0177] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[0178] If necessary, the server will notify the police, who will rush to the scene.

[0179] 2. Detecting Suspected Fraudulent Calls:

[0180] The server sends the drone a patrol schedule for areas prone to fraud.

[0181] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0182] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0183] The server analyzes the situation and provides advice and instructions to the drone.

[0184] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[0185] If necessary, the server will notify the police, who will respond to the scene.

[0186] In this way, this invention enables efficient security patrols in high-crime areas, enabling highly accurate detection of abnormal behavior and rapid response, thereby improving the safety of the entire area and increasing the sense of security for residents.

[0187] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0188] Step 1:

[0189] The server collects data for the patrol area. Specifically, the server imports past data on crime hotspots and analyzes crime rates by time of day and day of the week. The input is crime occurrence data, and the output is a crime risk map for each area as a result of the analysis.

[0190] Step 2:

[0191] The server uses the generative AI model to generate a schedule for the patrol area. The server inputs a prompt statement, "Please generate a patrol schedule for a specific area," into the AI ​​model and obtains the patrol schedule generated by the AI. The inputs are a crime risk map and the prompt statement, and the output is the patrol schedule.

[0192] Step 3:

[0193] The server sends the generated patrol schedule to the drone. Specifically, it transfers a data packet containing the generated schedule to the drone via Wi-Fi or 4G / 5G networks. The input is the patrol schedule, and the output is a confirmation that the transmission to the drone has been completed.

[0194] Step 4:

[0195] The server issues a "start patrol" command to the drone. The drone starts patrolling the designated area in autopilot mode. The input is the start patrol command, and the output is the drone starting to patrol.

[0196] Step 5:

[0197] The drone collects data in real time using various sensors (camera, audio sensor, environmental sensor). The input is environmental data within the patrol area, and the output is the collected sensor data.

[0198] Step 6:

[0199] The data collected by the drone is sent to the terminal in real time. Specifically, the data is sent to the terminal via Wi-Fi or 4G / 5G networks using the drone's communication module. The input is various sensor data, and the output is confirmation that the data has been sent to the terminal.

[0200] Step 7:

[0201] The data received by the device is quickly analyzed using a generative AI model. The device inputs the prompt "Analyze the collected data and detect abnormalities" into the AI ​​model to detect abnormal behavior or situations. The input is various sensor data and the prompt, and the output is the detection result of abnormal behavior or situations.

[0202] Step 8:

[0203] The device reports abnormal behavior or abnormal conditions to the server. Specifically, it sends a data packet containing the abnormality detection result to the server. The input is the abnormality detection result, and the output is a confirmation that the report has been sent to the server.

[0204] Step 9:

[0205] The server analyzes the received abnormal behavior reports and evaluates their urgency. The input is the abnormal behavior report data, and the output is the urgency evaluation result of the abnormal behavior.

[0206] Step 10:

[0207] The server issues warnings and advice to the drone. For example, it sends instructions such as "play a warning message" or "turn on a warning light." The input is the warning instruction, and the output is confirmation that the instruction has been sent to the drone.

[0208] Step 11:

[0209] The drone plays a specified audio message and turns on a warning light. For example, it might play the message "Misconduct in this location is prohibited. Please leave immediately." The input is the warning instruction, and the output is the actual warning action.

[0210] Step 12:

[0211] If the server determines that the abnormal behavior is of high urgency, it automatically notifies the police. The server sends the drone's current location, real-time video data, and detailed information about the abnormal behavior to the police. The input is the trigger for the emergency call, and the output is confirmation that the call to the police has been completed.

[0212] Step 13:

[0213] After the drone finishes its patrol area, it sends all collected data to the server. The input is the data collected after the patrol, and the output is a confirmation that the data has been sent to the server.

[0214] Step 14:

[0215] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior. The input is the collected data, and the output is the analysis results.

[0216] Step 15:

[0217] The server generates a report summarizing the analysis results and provides it to the user. The user then takes necessary measures and makes adjustments based on the report. The input is the analysis results, and the output is the report.

[0218] (Application example 1)

[0219] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0220] There is a need for efficient patrols, detection of abnormal behavior, and rapid response in high-crime areas. However, current systems rely heavily on manual patrols and monitoring, which are costly and time-consuming and make it difficult to respond in real time. Furthermore, delayed response in emergencies can have fatal consequences. To solve this problem, a more efficient and rapid method is needed.

[0221] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0222] In this invention, the server includes means for patrolling high-crime areas by drones, means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, means for issuing warnings and advice when an abnormal situation is detected, means for notifying the police in an emergency, means for transmitting data collected from the drone to a smartphone in real time, means for delivering the real-time data to a user terminal in streaming format, and means for sending a push notification to the user terminal when abnormal behavior is detected. This enables efficient patrols, rapid detection of abnormal behavior, real-time information provision to users, and rapid response in emergencies.

[0223] A "drone" is an unmanned aerial vehicle that flies remotely or autonomously and is equipped with cameras and sensors to collect data.

[0224] A "high-crime area" is a specific area where crimes occur frequently based on past data and statistics.

[0225] "Abnormal behavior" refers to behavior that differs from normal behavior and is suspected of being criminal or fraudulent.

[0226] An "abnormal situation" is a situation that differs from the normal environment or circumstances and suggests that abnormal or illegal activity is occurring.

[0227] A "sensor" is a device that collects environmental information and converts it into an analog or digital signal.

[0228] "Generative AI" is an artificial intelligence technology that identifies and judges abnormal behavior and situations based on large amounts of data.

[0229] "Warning" refers to the act of giving a warning or advice to a target person.

[0230] "Advice" refers to the act of making suggestions or instructions to encourage appropriate behavior by a person.

[0231] An "emergency" is an extraordinary situation that requires immediate action.

[0232] "Police" refers to a public institution whose primary mission is to prevent crime and maintain public order.

[0233] "Reporting" refers to the act of reporting an incident or abnormal situation to the relevant authorities.

[0234] "Real-time" refers to a state in which data collection and processing occur immediately, without delay.

[0235] A "smartphone" is a mobile phone with computing power and internet connectivity.

[0236] "Streaming format" refers to a technology that transmits data continuously and receives and plays it back in real time.

[0237] A "user terminal" is a digital device that displays data and performs operations.

[0238] "Push notifications" refers to a technology that automatically sends messages from a web server to a client device.

[0239] Overall system configuration

[0240] This invention is a system consisting of a drone, a server, a terminal, and a user. Each element works together to efficiently patrol high-crime areas, detect abnormal behavior, issue warnings, and notify the police in emergencies.

[0241] 1. Drones

[0242] Drones are autonomous unmanned aerial vehicles equipped with cameras, audio sensors, and environmental sensors. They patrol crime-prone areas under instructions from a server and transmit data collected by various sensors to the server in real time. Furthermore, drones are equipped with generative AI, allowing them to analyze some of the data themselves.

[0243] The drone is equipped with a battery management system that allows it to automatically return to a charging station and recharge when the battery level is low.

[0244] 2. Server

[0245] The server is responsible for the chain of command for the entire system and data analysis. It sends patrol area schedules to the drones and receives and analyzes the data sent in real time. The server is equipped with advanced generative AI to detect and analyze abnormal behavior. If an abnormality is detected, the server sends a warning instruction to the drone and, if necessary, sends a push notification to the user's smartphone. It also has a function to automatically notify the police in an emergency. The server can provide real-time data to the user in streaming format. It also accumulates and analyzes the collected data and generates periodic reports.

[0246] 3. Terminal

[0247] The device is equipped with a generative AI that analyzes data sent from the drone in real time. If it detects abnormal behavior or an abnormal situation, it immediately reports the results to the server. The device also provides an interface for users to configure the system, check patrol schedules, and monitor abnormal behavior. It also has the function of sending a push notification to the user's device if abnormal behavior is detected.

[0248] 4. Users

[0249] Users include system administrators, police operators, and local residents. Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior. In an emergency, users can manually notify the police from their smartphones.

[0250] Natural language description of the process

[0251] The drone patrols designated high-crime areas and collects data using various sensors. The collected data is sent to a server in real time via 4G / 5G. On the server, a generating AI analyzes the data and detects abnormal behavior or situations. If an abnormality is detected, the server sends a warning command to the drone and a push notification to the user's smartphone. In the event of an emergency, the server automatically notifies the police. Users can view the real-time data in streaming format on their smartphone and can also make additional reports as needed.

[0252] Examples of specific examples and prompts

[0253] Specific examples

[0254] While a drone is patrolling a parking lot at night, the camera captures several people engaging in suspicious transactions. This video data is sent to a server in real time, and the generating AI determines that the activity is suspicious. The server immediately sends a warning command to the drone, which plays a message saying, "Fraudulent activity is prohibited. Please leave the area immediately." The server also sends a push notification to the user's smartphone, allowing them to review the video in real time. If necessary, the server automatically notifies the police.

[0255] Prompt Sentence Examples

[0256] "While a drone is patrolling a parking lot at night, several people appear to be conducting suspicious transactions. Based on this data, detect suspicious activity, send a warning command to the drone, and send a real-time notification to the user's smartphone. Also, describe the process for automatically notifying the police if necessary."

[0257] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0258] Step 1:

[0259] The server generates a patrol schedule for high-crime areas. It uses area data and time data as input and generates the schedule based on this. The generated schedule data is sent as output to the drone. This sets the drone to patrol the specified area at the specified time.

[0260] Step 2:

[0261] The drone begins patrolling high-crime areas based on the received schedule. It uses schedule data from the server as input and environmental data and video data obtained by sensors and cameras as output, sending them to the server in real time.

[0262] Step 3:

[0263] The server receives data sent from the drone in real time and analyzes it using generative AI. Video data, audio data, and environmental data are used as input, and data processing is performed to analyze and detect abnormal behavior and abnormal situations. The output is the detection result of abnormal behavior and abnormal situations.

[0264] Step 4:

[0265] If the server detects abnormal behavior or an abnormal situation, it sends a warning or advice to the drone. It uses the detection result data as input and sends warning instruction data to the drone as output. Specific actions include having the drone play an audio message such as, "Unauthorized activity is prohibited in this location. Please leave the area immediately."

[0266] Step 5:

[0267] If the server detects abnormal behavior or an abnormal situation, it sends a push notification to the user's smartphone. It uses the detection result data as input and sends a notification message to the user's device as output. Specifically, it sends a notification message to the user saying, "Emergency: Suspicious person has entered the premises!"

[0268] Step 6:

[0269] If the server detects an emergency, it automatically notifies the police. It uses the detection result data and location data as input, and sends a notification message to the police system as output. Specifically, it provides the police with the drone's current location and real-time video data.

[0270] Step 7:

[0271] After the drone completes its patrol, it sends all collected data to the server. It uses the environmental data, video data, and audio data collected during the patrol as input, and sends all data to the server as output.

[0272] Step 8:

[0273] The server stores the collected data and uses generation AI to analyze it. The collected data is used as input, and data processing is performed to extract the frequency and trends of abnormal behavior. The analysis results are obtained as output. Specific operations include generating statistical data on the location and frequency of abnormal behavior.

[0274] Step 9:

[0275] The server generates a report summarizing the analysis results and provides it to the user. It uses the analysis result data as input, processes the data, and formats it into a report format. It then sends the report data to the user's terminal as output.

[0276] Step 10:

[0277] The user receives the report and takes necessary measures and adjustments. The report data is used as input, and the output is used to adjust local safety measures and patrol schedules. Specific actions include setting up new patrol areas and strengthening crime prevention measures.

[0278] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0279] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. This system consists of five main components: a drone, a server, a terminal, a user, and an emotion engine. The detailed functions and linked operations of each component are described below.

[0280] System configuration

[0281] 1. Drones

[0282] The drone is equipped with a camera, audio sensors, environmental sensors, and generative AI.

[0283] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[0284] It detects abnormal behavior and situations and sends the data to the server in real time.

[0285] 2. Server

[0286] The server is responsible for controlling the entire system and analyzing data.

[0287] Generate a schedule for the patrol area and send it to the drone.

[0288] It receives data transmitted from the drone and analyzes it using generative AI to detect abnormal behavior and determine appropriate action.

[0289] In case of an emergency, it will notify the police and provide the drone's location and real-time data.

[0290] 3. Terminal

[0291] The device is equipped with generative AI that analyzes data sent from the drone in real time.

[0292] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[0293] 4. Users

[0294] Users include system administrators, police operators, local residents, etc.

[0295] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[0296] Police operators receive emergency calls and respond quickly to the scene.

[0297] 5. Emotion Engine

[0298] The emotion engine analyzes the user's video and audio data and has the ability to recognize emotions in real time.

[0299] The emotion engine adjusts the response of the drone and the entire system based on the emotions it recognizes.

[0300] Program processing overview

[0301] Preparing and conducting patrols

[0302] The server generates a patrol schedule based on data on the patrol area and sends it to the drone.

[0303] The drone automatically patrols a set route and collects data using various sensors.

[0304] The device analyzes the data in real time and detects abnormal behavior or situations.

[0305] Detecting and responding to abnormal behavior

[0306] If the device detects an abnormality, it transfers the data and analysis results to the server.

[0307] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[0308] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[0309] The emotion engine analyzes the user's emotions and adjusts the drone's response.

[0310] Emergency response

[0311] If the server determines that the emergency is high, it will automatically notify the police.

[0312] The server sends the drone's current location and real-time video data to the police.

[0313] The user (police operator) confirms the report and arranges for on-site response.

[0314] Data Collection and Reporting

[0315] After the drone has completed its designated patrol area, it sends all collected data to the server.

[0316] The server stores the collected data and analyzes long-term crime trends and problem areas.

[0317] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[0318] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[0319] Specific examples

[0320] Example 1: Detecting suspicious activity in a parking lot at night

[0321] The server sends the drone a schedule for patrolling the parking lot at night.

[0322] As the drone patrols the parking lot, it captures several people trading something around a car.

[0323] The device analyzes the video data and determines that the transaction is suspicious.

[0324] The server analyzes the situation and issues instructions to the drone to warn it off.

[0325] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[0326] At the same time, the emotion engine analyzes the suspicious person's reaction and determines whether additional action (e.g., immediate police reporting) is necessary.

[0327] Example 2: Detecting suspected fraudulent calls

[0328] The server sends the drone a patrol schedule for areas prone to fraud.

[0329] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0330] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0331] The server analyzes the situation and provides advice and instructions to the drone.

[0332] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[0333] An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[0334] If necessary, the server will notify the police, who will respond to the scene.

[0335] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[0336] The processing flow will be explained below.

[0337] Step 1:

[0338] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[0339] Step 2:

[0340] The server sends the generated patrol schedule to the drone.

[0341] Step 3:

[0342] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[0343] Step 4:

[0344] The server confirms that the drone is ready and sends a signal to activate the drone.

[0345] Step 5:

[0346] The drone will begin flying automatically, following the set patrol route.

[0347] Step 6:

[0348] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[0349] Step 7:

[0350] The data collected by the drone is sent to the device in real time.

[0351] Step 8:

[0352] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[0353] Step 9:

[0354] If the device detects an abnormality, it transfers the data and analysis results to the server.

[0355] Step 10:

[0356] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[0357] Step 11:

[0358] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[0359] Step 12:

[0360] The emotion engine analyzes the video and audio data of suspicious individuals and recognizes their emotions in real time.

[0361] Step 13:

[0362] Based on the emotions recognized by the emotion engine, the server further adjusts the drone's response.

[0363] Step 14:

[0364] Depending on the emotion the drone recognizes, for example, if tension is detected, it will play an appropriate additional message, such as "This behavior has been recorded. Please leave the area immediately."

[0365] Step 15:

[0366] The server will reassess the situation and notify the police if necessary.

[0367] Step 16:

[0368] The server sends the drone's current location and real-time video data to the police.

[0369] Step 17:

[0370] The user (police operator) confirms the report and arranges for on-site response.

[0371] Step 18:

[0372] After the drone completes its patrol, it sends all collected data to a server.

[0373] Step 19:

[0374] The server stores the collected data and analyzes long-term crime trends and problem areas.

[0375] Step 20:

[0376] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[0377] Step 21:

[0378] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[0379] Example 2

[0380] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] Conventional security patrol systems do not adequately detect abnormal behavior in real time or respond immediately in high-crime areas. Furthermore, rather than simply detecting abnormalities, appropriate responses that take into account the user's emotions are required, but this is difficult to achieve with conventional systems. This has led to problems such as a decline in crime prevention capabilities and a lack of security for local residents.

[0382] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0383] In this invention, the server includes a means for patrolling high-crime areas by drone, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, and an emotion recognition engine means for analyzing user emotions in real time and adjusting the response of the entire system. This enables rapid detection of abnormal behavior, immediate response, and response that takes user emotions into consideration.

[0384] A "drone" is an unmanned aircraft that can be remotely controlled and is equipped with cameras and various sensors to patrol crime-prone areas.

[0385] "Generative AI" is an artificial intelligence system that uses machine learning algorithms to recognize patterns from input data and automatically detect abnormal behavior or situations.

[0386] A "sensor" is a device that detects physical changes and outputs them as data, and in this system this includes cameras, audio sensors, and environmental sensors.

[0387] A "patrol schedule" is a plan that defines the drone's flight routes and time periods in high-crime areas.

[0388] "Warning" is an action that warns or advises the person involved or those around them when abnormal behavior or an abnormal situation is detected.

[0389] "Advice" is an action that instructs the appropriate way to respond to a detected abnormal situation.

[0390] An "emergency call" is the act of immediately notifying the police or relevant agencies when abnormal behavior or an abnormal situation occurs, and encouraging a prompt response.

[0391] An "emotion recognition engine" is a system that analyzes a user's video and audio data and recognizes their emotional state in real time.

[0392] "User" refers to anyone who uses this patrol system, such as a system administrator, police operator, or local resident.

[0393] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. The system is composed of the following main components: a drone, a server, a terminal, a user, and an emotion recognition engine.

[0394] System configuration

[0395] 1. Drones

[0396] Drones are equipped with cameras, audio sensors, environmental sensors, and generative AI. A typical drone is a general-purpose unmanned aerial vehicle (e.g., a general-purpose unmanned flying device) equipped with a camera and various sensors.

[0397] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[0398] Various sensors detect abnormal behavior and situations and send the data to a server in real time.

[0399] 2. Server

[0400] The server is responsible for controlling the entire system and analyzing data. A high-performance server machine (e.g., high-performance computer equipment) is used.

[0401] Generate a schedule for the patrol area and send it to the drone.

[0402] The data transmitted from the drone is analyzed and generative AI is used to detect abnormal behavior and determine appropriate actions.

[0403] In case of an emergency, the system will notify the police and provide the drone's location and real-time data.

[0404] 3. Terminal

[0405] The terminal is equipped with generative AI and analyzes data transmitted in real time from the drone, using high-performance AI computing platforms such as NVIDIA Jetson.

[0406] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[0407] 4. Users

[0408] Users include system administrators, police operators, local residents, and other users of the system.

[0409] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[0410] Police operators receive emergency calls and respond quickly to the scene.

[0411] 5. Emotion Recognition Engine

[0412] The emotion recognition engine analyzes the user's video and audio data and has the ability to recognize emotions in real time. Emotion recognition software such as Affectiva SDK is used.

[0413] The drone and the entire system can then adjust their response based on the emotions they recognize.

[0414] System Operation

[0415] The server generates an optimal patrol schedule based on data on the patrol area and sends it to the drone. The drone automatically patrols a set route and collects data using various sensors. The collected data is sent to the terminal in real time and analyzed by the generating AI. If the terminal detects abnormal behavior, it reports the data and analysis results to the server. The server determines the type and urgency of the abnormal behavior and instructs the drone to take appropriate action. In the event of an emergency, the server automatically notifies the police, and police operators respond promptly to the scene.

[0416] Specific examples

[0417] Example 1: Detecting suspicious activity in a parking lot at night

[0418] The server sends the drone a schedule for patrolling the parking lot at night.

[0419] As the drone patrols the parking lot, it captures several people trading something around a car.

[0420] The device analyzes the video data and determines that the transaction is suspicious.

[0421] The server analyzes the situation and issues instructions to the drone to warn it off.

[0422] As the drone approaches, it plays a message saying, "Unauthorized activity is prohibited. Please leave the area immediately." The emotion engine analyzes the suspicious person's reaction and considers additional measures if necessary.

[0423] Example 2: Detecting suspected fraudulent calls

[0424] The server sends the drone a patrol schedule for areas prone to fraud.

[0425] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0426] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0427] The server analyzes the situation and provides advice and instructions to the drone.

[0428] The drone approaches and plays a message: "This may be a scam. Please end this call and contact the police." An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[0429] Prompt Sentence Examples

[0430] "What should you do if a drone detects unauthorized activity in a parking lot at night?"

[0431] "Please tell me how the device and server work together when the drone detects a conversation that is suspected to be fraudulent."

[0432] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[0433] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0434] Step 1:

[0435] The server generates an optimal tour schedule based on the data of the tour area.

[0436] Input: Geographic data of high-crime areas and historical crime records

[0437] Data processing: Generative AI models are used to analyze geographic data and crime records to optimize patrol routes and times.

[0438] Output: Tour schedule

[0439] Specific operation: The server retrieves crime data from the database for the past year, analyzes it using a generative AI model (e.g., a general-purpose AI platform), generates an optimal patrol schedule, and transmits it to the drone.

[0440] Step 2:

[0441] The drone receives the patrol schedule and automatically patrols the set route.

[0442] Input: Tour schedule sent from the server

[0443] Data calculation: Uses GPS data and internal map data to navigate routes

[0444] Output: Route execution status and collected data

[0445] How it works: The drone travels to a designated starting point, flies a pre-defined route based on GPS and internal map data, and collects data from its surroundings using cameras and audio sensors.

[0446] Step 3:

[0447] The device receives data transmitted from the drone in real time and analyzes it using a generative AI model.

[0448] Input: Video data, audio data, and environmental data from drones

[0449] Data calculation: Analyzes video data and audio data to identify abnormal behavior

[0450] Output: Detection results and analysis data

[0451] Specific operation: The device receives a data stream from the drone in real time. The video data is analyzed using a TensorFlow model to identify specific behavioral patterns (e.g., a person running away). The audio data is analyzed using a voice recognition system (e.g., a general-purpose voice analysis system) to detect abnormal conversations.

[0452] Step 4:

[0453] If the device detects abnormal behavior, it sends the data and analysis results to the server.

[0454] Input: Analysis results and anomaly detection data

[0455] Data calculation: Determining the type and urgency of abnormal behavior

[0456] Output: Notifications and detailed data to send to the server

[0457] Specific operation: When the device identifies abnormal behavior (e.g., suspicious transaction activity), it sends a notification to the server with detailed data attached.

[0458] Step 5:

[0459] The server determines the type and urgency of abnormal behavior and sends appropriate action instructions to the drone.

[0460] Input: Analysis results and anomaly detection data from the device

[0461] Data calculations: assessing urgency and determining appropriate actions

[0462] Output: Instructions to the drone

[0463] Specific operation: The server receives the analysis data and determines the urgency of the abnormal behavior as "high." It then sends a "warning" command to the drone, playing a message saying, "That behavior is illegal. Please stop immediately."

[0464] Step 6:

[0465] An emotion recognition engine analyzes the user's emotions and adjusts the system's overall response.

[0466] Input: Video and audio data from a drone or device

[0467] Data Computing: Recognizing Emotional States Using Emotion Analysis Algorithms

[0468] Output: System-wide response adjustment instructions

[0469] Specific operation: The emotion recognition engine analyzes video and audio data to determine the reaction of suspicious individuals, and adjusts the drone's and the system's overall response as needed.

[0470] Step 7:

[0471] If the server determines that an emergency situation exists, it will automatically notify the police.

[0472] Input: Emergency data from drones and devices

[0473] Data calculation: generating and sending message content

[0474] Output: Police report details and real-time data

[0475] Specific operation: The server determines the abnormal behavior as "highly urgent" and initiates a police notification system. The notification includes the drone's current location, detailed emergency data, and a real-time video link.

[0476] Step 8:

[0477] After the drone completes its patrol area, it sends all collected data to the server.

[0478] Input: Drone-collected data

[0479] Data calculation: storing in a database and analyzing

[0480] Output: Long-term data analysis results and reports

[0481] Specific operation: When the drone completes its patrol, it transmits the collected data to a server via Wi-Fi or 4G / 5G. The server stores the data in a database and performs trend analysis using AI. A report is generated and sent to the system administrator.

[0482] (Application example 2)

[0483] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0484] Improving safety in crime-prone areas requires efficient, real-time patrols and the detection of abnormal behavior. However, conventional systems are slow to detect and respond to abnormal behavior, making it difficult to quickly detect and deal with suspicious individuals and criminals. They also lack the ability to respond flexibly to on-site situations or take measures that take user emotions into consideration. This makes it difficult to ensure the safety and security of local residents.

[0485] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0486] In this invention, the server includes a means for patrolling high-crime areas using drones, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, an emotion recognition means for analyzing the user's emotions, and a means for adjusting responses based on the emotion recognition results. This enables abnormal behavior to be detected in real time, enabling prompt and appropriate responses. Furthermore, flexible measures that take the user's emotions into account can increase the safety of the entire area and the sense of security of residents.

[0487] A "drone" is an unmanned aerial vehicle equipped with sensors and generative AI to patrol high-crime areas and detect abnormal behavior or situations.

[0488] A "sensor" is a device installed on a drone to detect changes in the environment or abnormal behavior, and is a device that collects data such as audio, video, and temperature.

[0489] "Generative AI" is artificial intelligence that analyzes collected data and detects abnormal behavior and situations in real time.

[0490] "Emotion recognition means" is a technology that analyzes the user's video and audio data and recognizes emotions in real time.

[0491] "Response adjustment means based on emotion recognition results" is a technology that adjusts the response of the drone and the entire system based on the emotional data analyzed by the emotion recognition means.

[0492] A "patrol schedule" is a server-generated plan of routes and time settings for drones to efficiently patrol high-crime areas.

[0493] "Real-time analysis means" is a technology that analyzes data collected from drones in real time and quickly detects abnormal behavior.

[0494] "Warning and advisory measures" are technologies that allow drones to issue warning messages to targets when abnormal behavior is detected.

[0495] "Emergency reporting means" is a technology that allows drones or systems to automatically report to the police when abnormal behavior is deemed to be an emergency.

[0496] As an embodiment of the present invention, the following system is constructed. Details of the system and the processing contents of each means will be explained using specific examples.

[0497] Overall system configuration

[0498] The system of the present invention is mainly composed of the following elements.

[0499] 1. Drones: They patrol high-crime areas and are equipped with cameras, audio sensors, environmental sensors, and generative AI.

[0500] 2. Server: Controls the entire system and is responsible for data analysis, generating drone patrol schedules, and emergency notification functions.

[0501] 3. Terminal: Analyzes data from the drone in real time, detects abnormal behavior, and issues instructions for response.

[0502] 4. Users: Includes system administrators, police operators, local residents, etc.

[0503] 5. Emotion recognition: Analyze the user's emotions and adjust the response.

[0504] Drone patrols and data collection

[0505] The drones patrol high-crime areas according to a pre-set patrol schedule, collecting data using various sensors (cameras, audio sensors, environmental sensors) and analyzing it in real time using generative AI.

[0506] Example: While a drone is patrolling a parking lot at night, the camera captures several people transacting something around a car, which is detected as anomalous behavior.

[0507] Data analysis and processing by the server

[0508] The server receives the data sent from the drone and uses the AI ​​to analyze any abnormal behavior. If a serious abnormality is detected, an emergency call is made.

[0509] Hardware / software used: high-performance computers (e.g., servers with NVIDIA GPUs), generative AI models (e.g., TensorFlow).

[0510] Example: A server analyzes video data captured by a drone and determines that a transaction is suspicious. It then sends a warning to the drone saying, "Fraudulent activity is prohibited here."

[0511] Emotion recognition and response adjustment

[0512] The emotion recognition means analyzes the user's voice and video data to recognize their emotions, and adjusts the drone's response based on the emotion recognition results.

[0513] Example: When a drone issues a warning, if the suspicious person responds aggressively, it will keep its distance, but if the person responds non-aggressively, it will continue to pay more attention.

[0514] Example prompt: "I'm patrolling a high-crime area and I've noticed some unusual behavior. What are my next steps?"

[0515] User operation

[0516] Users (system administrators and police operators) operate the server or terminals to configure the system, check patrol schedules, monitor abnormal behavior, and respond quickly to emergency calls.

[0517] This system can monitor the safety of high-crime areas in real time, detect and respond to abnormal behavior immediately, and take user emotions into consideration, enabling more appropriate security management.

[0518] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0519] Step 1:

[0520] The server generates the drone's patrol schedule. The inputs are geographic information of high-crime areas and past crime data. Using this data, the optimal patrol route and time is determined and sent to the drone. The output is the patrol schedule sent to the drone. Specifically, the server's AI algorithm retrieves past crime data from the database and compares it with map information to determine the patrol route.

[0521] Step 2:

[0522] The drone patrols high-crime areas based on a patrol schedule received from a server. The input is the patrol schedule from the server, and the output is collected real-time video, audio, and environmental data. Specifically, the drone automatically flies a set route, and the onboard cameras and sensors continuously record the surrounding situation.

[0523] Step 3:

[0524] The device receives real-time data transmitted from the drone and analyzes it using a generative AI model. The input is video, audio, and environmental data from the drone, and the output is the detection result of abnormal behavior. Specifically, the device's generative AI model analyzes the received data and applies an algorithm to detect abnormal behavior or situations.

[0525] Step 4:

[0526] The server receives the anomaly analysis results sent from the device and determines the type of anomaly and its level of urgency. The input is the anomaly detection result from the device, and the output is instructions for the drone to take action and, if necessary, a report to the police. Specifically, the server instructs the drone to take appropriate action (such as issuing a warning or sending a warning message) based on the analysis results, and if the level of urgency is high, it automatically reports the anomaly to the police.

[0527] Step 5:

[0528] The drone issues warnings and advice on-site based on instructions from the server. The input is an action instruction from the server, and the output is the playback of a voice message or a specific action (for example, a change in flight pattern). Specific actions include the drone playing a specified message over a speaker and warning suspicious individuals as necessary.

[0529] Step 6:

[0530] The emotion recognition means analyzes the subject's reactions and adjusts the drone's response. The input is video and audio data from the drone's camera and microphone, and the output is the emotion recognition results and instructions for the drone to act. Specifically, the AI ​​model analyzes the video and audio data to recognize the subject's emotions. If an aggressive reaction is observed, the drone will maintain its distance, and if the reaction is non-aggressive, it will issue further warnings.

[0531] Step 7:

[0532] The server accumulates the collected data, analyzes long-term crime trends, and generates reports. The input is all data transmitted by the drones, and the output is analysis results and reports. Specifically, the server stores the collected data in a database, periodically analyzes the data, and generates reports that identify crime trends and problem areas.

[0533] Step 8:

[0534] The user (system administrator or police operator) receives the report and modifies the patrol plan or takes additional measures as necessary. The input is the report from the server, and the output is the modified patrol plan or additional measures. In concrete terms, the user checks the contents of the report and sets up new patrol routes or additional crime prevention measures.

[0535] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0536] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0537] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0538] [Second embodiment]

[0539] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0540] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0541] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0542] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0543] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0544] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0545] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0546] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0547] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0548] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0549] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0550] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0551] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system mainly consists of four elements: a drone, a server, a terminal, and a user, all of which work in close coordination with each other.

[0552] System configuration

[0553] 1. Drones

[0554] The drone is equipped with various sensors such as a camera, audio sensor, and environmental sensor, as well as generative AI.

[0555] The drones automatically patrol designated crime-prone areas based on instructions from the server.

[0556] If the battery level gets low, it can automatically return and recharge.

[0557] 2. Server

[0558] The server is responsible for the overall chain of command and data analysis.

[0559] Generate a schedule for the patrol area and send it to the drone.

[0560] It receives data sent from the drone in real time and instructs appropriate actions based on the analysis results.

[0561] It also has a function to report to the police if abnormal behavior or situations are detected.

[0562] 3. Terminal

[0563] The device is equipped with generative AI that quickly analyzes data sent from the drone in real time.

[0564] Detects abnormal behavior and situations and reports the results to the server.

[0565] 4. Users

[0566] Users include system administrators, police operators, and local residents.

[0567] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[0568] Police operators receive emergency calls and respond quickly to the scene.

[0569] Program processing overview

[0570] Preparing and conducting patrols

[0571] The server generates a patrol schedule for high-crime areas based on data from the patrol area and sends it to the drone.

[0572] The drone automatically patrols a set route and collects data using various sensors.

[0573] The device analyzes the data in real time and detects abnormal behavior or situations.

[0574] Detecting and responding to abnormal behavior

[0575] If the device detects an abnormality, it immediately reports the situation to the server.

[0576] The server receives the report and analyzes the type and urgency of the abnormal behavior.

[0577] The server issues warnings and advice to the drone.

[0578] The drone approaches anyone behaving abnormally and uses audio messages and warning lights to warn them.

[0579] Emergency response

[0580] If the server determines that the emergency is high, it will automatically notify the police.

[0581] The server sends the drone's current location and real-time video data to the police.

[0582] The user (police operator) receives the report and arranges for a response at the scene.

[0583] Data Collection and Reporting

[0584] After the drone has completed its designated patrol area, it sends all collected data to the server.

[0585] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[0586] The server generates a report summarizing the analysis results and provides it to the user.

[0587] The user receives the report and takes any necessary measures or adjustments.

[0588] Specific examples

[0589] Example 1: Detecting suspicious activity in a parking lot at night

[0590] The server sends the drone a schedule for patrolling the parking lot at night.

[0591] As the drone patrols the parking lot, it captures several people trading something around a car.

[0592] The device analyzes the video data and determines that the transaction is suspicious.

[0593] The server analyzes the situation and issues instructions to the drone to warn it off.

[0594] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[0595] If it is determined that police are needed, the server will notify them and they will rush to the scene.

[0596] Example 2: Detecting suspected fraudulent calls

[0597] The server sends the drone a patrol schedule for areas prone to fraud.

[0598] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0599] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0600] The server analyzes the situation and provides advice and instructions to the drone.

[0601] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[0602] If necessary, the server will notify the police, who will respond to the scene.

[0603] This invention is designed to automate security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, thereby improving overall community safety and enhancing residents' sense of security.

[0604] The processing flow will be explained below.

[0605] Step 1:

[0606] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[0607] Step 2:

[0608] The server sends the generated patrol schedule to the drone.

[0609] Step 3:

[0610] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[0611] Step 4:

[0612] The server confirms that the drone is ready and sends a signal to activate the drone.

[0613] Step 5:

[0614] The drone will begin flying automatically, following the set patrol route.

[0615] Step 6:

[0616] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[0617] Step 7:

[0618] The data collected by the drone is sent to the device in real time.

[0619] Step 8:

[0620] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[0621] Step 9:

[0622] If the device detects an abnormality, it transfers the data and analysis results to the server.

[0623] Step 10:

[0624] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[0625] Step 11:

[0626] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[0627] Step 12:

[0628] The drone will play messages and turn on warning lights to deter abnormal behavior.

[0629] Step 13:

[0630] The server will reassess the situation and notify the police if necessary.

[0631] Step 14:

[0632] The server sends the drone's current location and real-time video data to the police.

[0633] Step 15:

[0634] The user (police operator) confirms the report and arranges for on-site response.

[0635] Step 16:

[0636] After the drone completes its patrol, it sends all collected data to a server.

[0637] Step 17:

[0638] The server stores the collected data and analyzes long-term crime trends and problem areas.

[0639] Step 18:

[0640] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[0641] Step 19:

[0642] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[0643] Example 1

[0644] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0645] Conventional security patrol systems were unable to efficiently patrol high-crime areas, making it difficult to detect and respond to abnormal behavior or situations in real time. Another issue was delays in reporting emergencies to the police, making it difficult to respond quickly.

[0646] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0647] In this invention, the server includes a means for generating a patrol schedule for high-crime areas based on patrol area data and transmitting it to the drone, a means for the drone to detect abnormal behavior or situations in real time using various sensors and generation AI and transmit that data to the terminal, and a means for issuing warnings or advice when an abnormal situation is detected. This enables highly efficient detection and response to abnormal behavior in real time.

[0648] "Patrol area" refers to a specific area or region in a high-crime neighborhood that will be patrolled by a drone.

[0649] A "drone" is an unmanned aerial vehicle that operates by remote control or automatic pilot and is equipped with a camera and various sensors.

[0650] A "server" is a computer that is responsible for the chain of command and management of the entire system, including collecting data, analyzing it, and sending instructions.

[0651] A "terminal" is a computer that receives and analyzes data sent from the drone.

[0652] "Generative AI" is part of a system that uses artificial intelligence technology to analyze data and make decisions, and refers to an algorithm that utilizes a generative model.

[0653] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is likely to be criminal or dangerous.

[0654] A "warning" is a warning or advice given to discourage certain behavior.

[0655] An "emergency" is a situation in which a serious danger or crime is in progress and requires a rapid response.

[0656] "Reporting" refers to the act of informing the police or other relevant authorities of abnormal behavior or an emergency.

[0657] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system is mainly composed of four elements: a server, a terminal, a drone, and a user, and each element works in close coordination with the others.

[0658] Hardware and software used

[0659] Server: A computer with high-performance data analysis capabilities and the processing power to operate generative AI models. Specifically, it is a server equipped with a large-capacity hard disk and a high-speed processor.

[0660] Device: A computer that receives and analyzes data in real time and runs the generative AI model. This device can be a laptop or a high-performance desktop computer.

[0661] Drone: An unmanned aerial vehicle equipped with cameras, audio sensors, and environmental sensors, with autopilot capabilities, a battery management system, and the ability to return automatically when it needs to be recharged.

[0662] Generative AI model: An artificial intelligence algorithm for detecting abnormal behavior or situations. For example, TensorFlow is used for visual analysis, and PyTorch is used for audio analysis.

[0663] Program processing

[0664] 1. The server collects data on the patrol area and generates a patrol schedule using a generative AI model. The generated schedule includes prompts such as "Patrol the parking lot at night and detect suspicious behavior."

[0665] 2. The server sends the generated patrol schedule to the drone and issues patrol instructions to the drone.

[0666] 3. The drone patrols the set area on autopilot, collecting video data from the camera, audio data from the audio sensor, and temperature and humidity data from the environmental sensors.

[0667] 4. The data collected by the drone is sent to the terminal in real time via the drone's communication module using Wi-Fi or 4G / 5G networks.

[0668] 5. The device quickly analyzes the data it receives using a generative AI model, inputs a prompt such as "Analyze the collected data and detect any abnormalities," and reports any abnormal behavior or abnormal situations to the server.

[0669] 6. The server receives the report and evaluates the type of abnormal behavior (e.g., voyeurism, violent behavior) and the urgency.

[0670] 7. The server issues warnings and advice to the drone, such as "play a warning message" or "turn on a warning light."

[0671] 8. The drone will play a specified voice message (e.g., "Misconduct in this area is prohibited. Please leave immediately.") and turn on a warning light.

[0672] 9. If the server determines that the abnormal behavior is of high urgency, it will automatically notify the police. The notification process will send the drone's current location, real-time video data, and detailed information about the abnormal behavior.

[0673] 10. After the drone has completed its designated patrol area, it will send all collected data to the server.

[0674] 11. The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[0675] 12. The server generates a report summarizing the analysis results and provides it to the user, who then takes necessary measures and makes adjustments based on the report.

[0676] Specific examples of programs

[0677] 1. Detecting suspicious activity in parking lots at night:

[0678] The server sends the drone a schedule for patrolling the parking lot at night.

[0679] As the drone patrols the parking lot, it captures several people conducting transactions around a car.

[0680] The device analyzes the video data and determines that the transaction is suspicious.

[0681] The server analyzes the situation and issues instructions to the drone to warn it off.

[0682] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[0683] If necessary, the server will notify the police, who will rush to the scene.

[0684] 2. Detecting Suspected Fraudulent Calls:

[0685] The server sends the drone a patrol schedule for areas prone to fraud.

[0686] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0687] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0688] The server analyzes the situation and provides advice and instructions to the drone.

[0689] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[0690] If necessary, the server will notify the police, who will respond to the scene.

[0691] In this way, this invention enables efficient security patrols in high-crime areas, enabling highly accurate detection of abnormal behavior and rapid response, thereby improving the safety of the entire area and increasing the sense of security for residents.

[0692] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0693] Step 1:

[0694] The server collects data for the patrol area. Specifically, the server imports past data on crime hotspots and analyzes crime rates by time of day and day of the week. The input is crime occurrence data, and the output is a crime risk map for each area as a result of the analysis.

[0695] Step 2:

[0696] The server uses the generative AI model to generate a schedule for the patrol area. The server inputs a prompt statement, "Please generate a patrol schedule for a specific area," into the AI ​​model and obtains the patrol schedule generated by the AI. The inputs are a crime risk map and the prompt statement, and the output is the patrol schedule.

[0697] Step 3:

[0698] The server sends the generated patrol schedule to the drone. Specifically, it transfers a data packet containing the generated schedule to the drone via Wi-Fi or 4G / 5G networks. The input is the patrol schedule, and the output is a confirmation that the transmission to the drone has been completed.

[0699] Step 4:

[0700] The server issues a "start patrol" command to the drone. The drone starts patrolling the designated area in autopilot mode. The input is the start patrol command, and the output is the drone starting to patrol.

[0701] Step 5:

[0702] The drone collects data in real time using various sensors (camera, audio sensor, environmental sensor). The input is environmental data within the patrol area, and the output is the collected sensor data.

[0703] Step 6:

[0704] The data collected by the drone is sent to the terminal in real time. Specifically, the data is sent to the terminal via Wi-Fi or 4G / 5G networks using the drone's communication module. The input is various sensor data, and the output is confirmation that the data has been sent to the terminal.

[0705] Step 7:

[0706] The data received by the device is quickly analyzed using a generative AI model. The device inputs the prompt "Analyze the collected data and detect abnormalities" into the AI ​​model to detect abnormal behavior or situations. The input is various sensor data and the prompt, and the output is the detection result of abnormal behavior or situations.

[0707] Step 8:

[0708] The device reports abnormal behavior or abnormal conditions to the server. Specifically, it sends a data packet containing the abnormality detection result to the server. The input is the abnormality detection result, and the output is a confirmation that the report has been sent to the server.

[0709] Step 9:

[0710] The server analyzes the received abnormal behavior reports and evaluates their urgency. The input is the abnormal behavior report data, and the output is the urgency evaluation result of the abnormal behavior.

[0711] Step 10:

[0712] The server issues warnings and advice to the drone. For example, it sends instructions such as "play a warning message" or "turn on a warning light." The input is the warning instruction, and the output is confirmation that the instruction has been sent to the drone.

[0713] Step 11:

[0714] The drone plays a specified audio message and turns on a warning light. For example, it might play the message "Misconduct in this location is prohibited. Please leave immediately." The input is the warning instruction, and the output is the actual warning action.

[0715] Step 12:

[0716] If the server determines that the abnormal behavior is of high urgency, it automatically notifies the police. The server sends the drone's current location, real-time video data, and detailed information about the abnormal behavior to the police. The input is the trigger for the emergency call, and the output is confirmation that the call to the police has been completed.

[0717] Step 13:

[0718] After the drone finishes its patrol area, it sends all collected data to the server. The input is the data collected after the patrol, and the output is a confirmation that the data has been sent to the server.

[0719] Step 14:

[0720] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior. The input is the collected data, and the output is the analysis results.

[0721] Step 15:

[0722] The server generates a report summarizing the analysis results and provides it to the user. The user then takes necessary measures and makes adjustments based on the report. The input is the analysis results, and the output is the report.

[0723] (Application example 1)

[0724] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0725] There is a need for efficient patrols, detection of abnormal behavior, and rapid response in high-crime areas. However, current systems rely heavily on manual patrols and monitoring, which are costly and time-consuming and make it difficult to respond in real time. Furthermore, delayed response in emergencies can have fatal consequences. To solve this problem, a more efficient and rapid method is needed.

[0726] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0727] In this invention, the server includes means for patrolling high-crime areas by drones, means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, means for issuing warnings and advice when an abnormal situation is detected, means for notifying the police in an emergency, means for transmitting data collected from the drone to a smartphone in real time, means for delivering the real-time data to a user terminal in streaming format, and means for sending a push notification to the user terminal when abnormal behavior is detected. This enables efficient patrols, rapid detection of abnormal behavior, real-time information provision to users, and rapid response in emergencies.

[0728] A "drone" is an unmanned aerial vehicle that flies remotely or autonomously and is equipped with cameras and sensors to collect data.

[0729] A "high-crime area" is a specific area where crimes occur frequently based on past data and statistics.

[0730] "Abnormal behavior" refers to behavior that differs from normal behavior and is suspected of being criminal or fraudulent.

[0731] An "abnormal situation" is a situation that differs from the normal environment or circumstances and suggests that abnormal or illegal activity is occurring.

[0732] A "sensor" is a device that collects environmental information and converts it into an analog or digital signal.

[0733] "Generative AI" is an artificial intelligence technology that identifies and judges abnormal behavior and situations based on large amounts of data.

[0734] "Warning" refers to the act of giving a warning or advice to a target person.

[0735] "Advice" refers to the act of making suggestions or instructions to encourage appropriate behavior by a person.

[0736] An "emergency" is an extraordinary situation that requires immediate action.

[0737] "Police" refers to a public institution whose primary mission is to prevent crime and maintain public order.

[0738] "Reporting" refers to the act of reporting an incident or abnormal situation to the relevant authorities.

[0739] "Real-time" refers to a state in which data collection and processing occur immediately, without delay.

[0740] A "smartphone" is a mobile phone with computing power and internet connectivity.

[0741] "Streaming format" refers to a technology that transmits data continuously and receives and plays it back in real time.

[0742] A "user terminal" is a digital device that displays data and performs operations.

[0743] "Push notifications" refers to a technology that automatically sends messages from a web server to a client device.

[0744] Overall system configuration

[0745] This invention is a system consisting of a drone, a server, a terminal, and a user. Each element works together to efficiently patrol high-crime areas, detect abnormal behavior, issue warnings, and notify the police in emergencies.

[0746] 1. Drones

[0747] Drones are autonomous unmanned aerial vehicles equipped with cameras, audio sensors, and environmental sensors. They patrol crime-prone areas under instructions from a server and transmit data collected by various sensors to the server in real time. Furthermore, drones are equipped with generative AI, allowing them to analyze some of the data themselves.

[0748] The drone is equipped with a battery management system that allows it to automatically return to a charging station and recharge when the battery level is low.

[0749] 2. Server

[0750] The server is responsible for the chain of command for the entire system and data analysis. It sends patrol area schedules to the drones and receives and analyzes the data sent in real time. The server is equipped with advanced generative AI to detect and analyze abnormal behavior. If an abnormality is detected, the server sends a warning instruction to the drone and, if necessary, sends a push notification to the user's smartphone. It also has a function to automatically notify the police in an emergency. The server can provide real-time data to the user in streaming format. It also accumulates and analyzes the collected data and generates periodic reports.

[0751] 3. Terminal

[0752] The device is equipped with a generative AI that analyzes data sent from the drone in real time. If it detects abnormal behavior or an abnormal situation, it immediately reports the results to the server. The device also provides an interface for users to configure the system, check patrol schedules, and monitor abnormal behavior. It also has the function of sending a push notification to the user's device if abnormal behavior is detected.

[0753] 4. Users

[0754] Users include system administrators, police operators, and local residents. Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior. In an emergency, users can manually notify the police from their smartphones.

[0755] Natural language description of the process

[0756] The drone patrols designated high-crime areas and collects data using various sensors. The collected data is sent to a server in real time via 4G / 5G. On the server, a generating AI analyzes the data and detects abnormal behavior or situations. If an abnormality is detected, the server sends a warning command to the drone and a push notification to the user's smartphone. In the event of an emergency, the server automatically notifies the police. Users can view the real-time data in streaming format on their smartphone and can also make additional reports as needed.

[0757] Examples of specific examples and prompts

[0758] Specific examples

[0759] While a drone is patrolling a parking lot at night, the camera captures several people engaging in suspicious transactions. This video data is sent to a server in real time, and the generating AI determines that the activity is suspicious. The server immediately sends a warning command to the drone, which plays a message saying, "Fraudulent activity is prohibited. Please leave the area immediately." The server also sends a push notification to the user's smartphone, allowing them to review the video in real time. If necessary, the server automatically notifies the police.

[0760] Prompt Sentence Examples

[0761] "While a drone is patrolling a parking lot at night, several people appear to be conducting suspicious transactions. Based on this data, detect suspicious activity, send a warning command to the drone, and send a real-time notification to the user's smartphone. Also, describe the process for automatically notifying the police if necessary."

[0762] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0763] Step 1:

[0764] The server generates a patrol schedule for high-crime areas. It uses area data and time data as input and generates the schedule based on this. The generated schedule data is sent as output to the drone. This sets the drone to patrol the specified area at the specified time.

[0765] Step 2:

[0766] The drone begins patrolling high-crime areas based on the received schedule. It uses schedule data from the server as input and environmental data and video data obtained by sensors and cameras as output, sending them to the server in real time.

[0767] Step 3:

[0768] The server receives data sent from the drone in real time and analyzes it using generative AI. Video data, audio data, and environmental data are used as input, and data processing is performed to analyze and detect abnormal behavior and abnormal situations. The output is the detection result of abnormal behavior and abnormal situations.

[0769] Step 4:

[0770] If the server detects abnormal behavior or an abnormal situation, it sends a warning or advice to the drone. It uses the detection result data as input and sends warning instruction data to the drone as output. Specific actions include having the drone play an audio message such as, "Unauthorized activity is prohibited in this location. Please leave the area immediately."

[0771] Step 5:

[0772] If the server detects abnormal behavior or an abnormal situation, it sends a push notification to the user's smartphone. It uses the detection result data as input and sends a notification message to the user's device as output. Specifically, it sends a notification message to the user saying, "Emergency: Suspicious person has entered the premises!"

[0773] Step 6:

[0774] If the server detects an emergency, it automatically notifies the police. It uses the detection result data and location data as input, and sends a notification message to the police system as output. Specifically, it provides the police with the drone's current location and real-time video data.

[0775] Step 7:

[0776] After the drone completes its patrol, it sends all collected data to the server. It uses the environmental data, video data, and audio data collected during the patrol as input, and sends all data to the server as output.

[0777] Step 8:

[0778] The server stores the collected data and uses generation AI to analyze it. The collected data is used as input, and data processing is performed to extract the frequency and trends of abnormal behavior. The analysis results are obtained as output. Specific operations include generating statistical data on the location and frequency of abnormal behavior.

[0779] Step 9:

[0780] The server generates a report summarizing the analysis results and provides it to the user. It uses the analysis result data as input, processes the data, and formats it into a report format. It then sends the report data to the user's terminal as output.

[0781] Step 10:

[0782] The user receives the report and takes necessary measures and adjustments. The report data is used as input, and the output is used to adjust local safety measures and patrol schedules. Specific actions include setting up new patrol areas and strengthening crime prevention measures.

[0783] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0784] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. This system consists of five main components: a drone, a server, a terminal, a user, and an emotion engine. The detailed functions and linked operations of each component are described below.

[0785] System configuration

[0786] 1. Drones

[0787] The drone is equipped with a camera, audio sensors, environmental sensors, and generative AI.

[0788] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[0789] It detects abnormal behavior and situations and sends the data to the server in real time.

[0790] 2. Server

[0791] The server is responsible for controlling the entire system and analyzing data.

[0792] Generate a schedule for the patrol area and send it to the drone.

[0793] It receives data transmitted from the drone and analyzes it using generative AI to detect abnormal behavior and determine appropriate action.

[0794] In case of an emergency, it will notify the police and provide the drone's location and real-time data.

[0795] 3. Terminal

[0796] The device is equipped with generative AI that analyzes data sent from the drone in real time.

[0797] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[0798] 4. Users

[0799] Users include system administrators, police operators, local residents, etc.

[0800] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[0801] Police operators receive emergency calls and respond quickly to the scene.

[0802] 5. Emotion Engine

[0803] The emotion engine analyzes the user's video and audio data and has the ability to recognize emotions in real time.

[0804] The emotion engine adjusts the response of the drone and the entire system based on the emotions it recognizes.

[0805] Program processing overview

[0806] Preparing and conducting patrols

[0807] The server generates a patrol schedule based on data on the patrol area and sends it to the drone.

[0808] The drone automatically patrols a set route and collects data using various sensors.

[0809] The device analyzes the data in real time and detects abnormal behavior or situations.

[0810] Detecting and responding to abnormal behavior

[0811] If the device detects an abnormality, it transfers the data and analysis results to the server.

[0812] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[0813] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[0814] The emotion engine analyzes the user's emotions and adjusts the drone's response.

[0815] Emergency response

[0816] If the server determines that the emergency is high, it will automatically notify the police.

[0817] The server sends the drone's current location and real-time video data to the police.

[0818] The user (police operator) confirms the report and arranges for on-site response.

[0819] Data Collection and Reporting

[0820] After the drone has completed its designated patrol area, it sends all collected data to the server.

[0821] The server stores the collected data and analyzes long-term crime trends and problem areas.

[0822] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[0823] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[0824] Specific examples

[0825] Example 1: Detecting suspicious activity in a parking lot at night

[0826] The server sends the drone a schedule for patrolling the parking lot at night.

[0827] As the drone patrols the parking lot, it captures several people trading something around a car.

[0828] The device analyzes the video data and determines that the transaction is suspicious.

[0829] The server analyzes the situation and issues instructions to the drone to warn it off.

[0830] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[0831] At the same time, the emotion engine analyzes the suspicious person's reaction and determines whether additional action (e.g., immediate police reporting) is necessary.

[0832] Example 2: Detecting suspected fraudulent calls

[0833] The server sends the drone a patrol schedule for areas prone to fraud.

[0834] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0835] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0836] The server analyzes the situation and provides advice and instructions to the drone.

[0837] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[0838] An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[0839] If necessary, the server will notify the police, who will respond to the scene.

[0840] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[0841] The processing flow will be explained below.

[0842] Step 1:

[0843] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[0844] Step 2:

[0845] The server sends the generated patrol schedule to the drone.

[0846] Step 3:

[0847] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[0848] Step 4:

[0849] The server confirms that the drone is ready and sends a signal to activate the drone.

[0850] Step 5:

[0851] The drone will begin flying automatically, following the set patrol route.

[0852] Step 6:

[0853] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[0854] Step 7:

[0855] The data collected by the drone is sent to the device in real time.

[0856] Step 8:

[0857] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[0858] Step 9:

[0859] If the device detects an abnormality, it transfers the data and analysis results to the server.

[0860] Step 10:

[0861] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[0862] Step 11:

[0863] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[0864] Step 12:

[0865] The emotion engine analyzes the video and audio data of suspicious individuals and recognizes their emotions in real time.

[0866] Step 13:

[0867] Based on the emotions recognized by the emotion engine, the server further adjusts the drone's response.

[0868] Step 14:

[0869] Depending on the emotion the drone recognizes, for example, if tension is detected, it will play an appropriate additional message, such as "This behavior has been recorded. Please leave the area immediately."

[0870] Step 15:

[0871] The server will reassess the situation and notify the police if necessary.

[0872] Step 16:

[0873] The server sends the drone's current location and real-time video data to the police.

[0874] Step 17:

[0875] The user (police operator) confirms the report and arranges for on-site response.

[0876] Step 18:

[0877] After the drone completes its patrol, it sends all collected data to a server.

[0878] Step 19:

[0879] The server stores the collected data and analyzes long-term crime trends and problem areas.

[0880] Step 20:

[0881] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[0882] Step 21:

[0883] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[0884] Example 2

[0885] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0886] Conventional security patrol systems do not adequately detect abnormal behavior in real time or respond immediately in high-crime areas. Furthermore, rather than simply detecting abnormalities, appropriate responses that take into account the user's emotions are required, but this is difficult to achieve with conventional systems. This has led to problems such as a decline in crime prevention capabilities and a lack of security for local residents.

[0887] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0888] In this invention, the server includes a means for patrolling high-crime areas by drone, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, and an emotion recognition engine means for analyzing user emotions in real time and adjusting the response of the entire system. This enables rapid detection of abnormal behavior, immediate response, and response that takes user emotions into consideration.

[0889] A "drone" is an unmanned aircraft that can be remotely controlled and is equipped with cameras and various sensors to patrol crime-prone areas.

[0890] "Generative AI" is an artificial intelligence system that uses machine learning algorithms to recognize patterns from input data and automatically detect abnormal behavior or situations.

[0891] A "sensor" is a device that detects physical changes and outputs them as data, and in this system this includes cameras, audio sensors, and environmental sensors.

[0892] A "patrol schedule" is a plan that defines the drone's flight routes and time periods in high-crime areas.

[0893] "Warning" is an action that warns or advises the person involved or those around them when abnormal behavior or an abnormal situation is detected.

[0894] "Advice" is an action that instructs the appropriate way to respond to a detected abnormal situation.

[0895] An "emergency call" is the act of immediately notifying the police or relevant agencies when abnormal behavior or an abnormal situation occurs, and encouraging a prompt response.

[0896] An "emotion recognition engine" is a system that analyzes a user's video and audio data and recognizes their emotional state in real time.

[0897] "User" refers to anyone who uses this patrol system, such as a system administrator, police operator, or local resident.

[0898] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. The system is composed of the following main components: a drone, a server, a terminal, a user, and an emotion recognition engine.

[0899] System configuration

[0900] 1. Drones

[0901] Drones are equipped with cameras, audio sensors, environmental sensors, and generative AI. A typical drone is a general-purpose unmanned aerial vehicle (e.g., a general-purpose unmanned flying device) equipped with a camera and various sensors.

[0902] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[0903] Various sensors detect abnormal behavior and situations and send the data to a server in real time.

[0904] 2. Server

[0905] The server is responsible for controlling the entire system and analyzing data. A high-performance server machine (e.g., high-performance computer equipment) is used.

[0906] Generate a schedule for the patrol area and send it to the drone.

[0907] The data transmitted from the drone is analyzed and generative AI is used to detect abnormal behavior and determine appropriate actions.

[0908] In case of an emergency, the system will notify the police and provide the drone's location and real-time data.

[0909] 3. Terminal

[0910] The terminal is equipped with generative AI and analyzes data transmitted in real time from the drone, using high-performance AI computing platforms such as NVIDIA Jetson.

[0911] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[0912] 4. Users

[0913] Users include system administrators, police operators, local residents, and other users of the system.

[0914] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[0915] Police operators receive emergency calls and respond quickly to the scene.

[0916] 5. Emotion Recognition Engine

[0917] The emotion recognition engine analyzes the user's video and audio data and has the ability to recognize emotions in real time. Emotion recognition software such as Affectiva SDK is used.

[0918] The drone and the entire system can then adjust their response based on the emotions they recognize.

[0919] System Operation

[0920] The server generates an optimal patrol schedule based on data on the patrol area and sends it to the drone. The drone automatically patrols a set route and collects data using various sensors. The collected data is sent to the terminal in real time and analyzed by the generating AI. If the terminal detects abnormal behavior, it reports the data and analysis results to the server. The server determines the type and urgency of the abnormal behavior and instructs the drone to take appropriate action. In the event of an emergency, the server automatically notifies the police, and police operators respond promptly to the scene.

[0921] Specific examples

[0922] Example 1: Detecting suspicious activity in a parking lot at night

[0923] The server sends the drone a schedule for patrolling the parking lot at night.

[0924] As the drone patrols the parking lot, it captures several people trading something around a car.

[0925] The device analyzes the video data and determines that the transaction is suspicious.

[0926] The server analyzes the situation and issues instructions to the drone to warn it off.

[0927] As the drone approaches, it plays a message saying, "Unauthorized activity is prohibited. Please leave the area immediately." The emotion engine analyzes the suspicious person's reaction and considers additional measures if necessary.

[0928] Example 2: Detecting suspected fraudulent calls

[0929] The server sends the drone a patrol schedule for areas prone to fraud.

[0930] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[0931] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[0932] The server analyzes the situation and provides advice and instructions to the drone.

[0933] The drone approaches and plays a message: "This may be a scam. Please end this call and contact the police." An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[0934] Prompt Sentence Examples

[0935] "What should you do if a drone detects unauthorized activity in a parking lot at night?"

[0936] "Please tell me how the device and server work together when the drone detects a conversation that is suspected to be fraudulent."

[0937] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[0938] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0939] Step 1:

[0940] The server generates an optimal tour schedule based on the data of the tour area.

[0941] Input: Geographic data of high-crime areas and historical crime records

[0942] Data processing: Generative AI models are used to analyze geographic data and crime records to optimize patrol routes and times.

[0943] Output: Tour schedule

[0944] Specific operation: The server retrieves crime data from the database for the past year, analyzes it using a generative AI model (e.g., a general-purpose AI platform), generates an optimal patrol schedule, and transmits it to the drone.

[0945] Step 2:

[0946] The drone receives the patrol schedule and automatically patrols the set route.

[0947] Input: Tour schedule sent from the server

[0948] Data calculation: Uses GPS data and internal map data to navigate routes

[0949] Output: Route execution status and collected data

[0950] How it works: The drone travels to a designated starting point, flies a pre-defined route based on GPS and internal map data, and collects data from its surroundings using cameras and audio sensors.

[0951] Step 3:

[0952] The device receives data transmitted from the drone in real time and analyzes it using a generative AI model.

[0953] Input: Video data, audio data, and environmental data from drones

[0954] Data calculation: Analyzes video data and audio data to identify abnormal behavior

[0955] Output: Detection results and analysis data

[0956] Specific operation: The device receives a data stream from the drone in real time. The video data is analyzed using a TensorFlow model to identify specific behavioral patterns (e.g., a person running away). The audio data is analyzed using a voice recognition system (e.g., a general-purpose voice analysis system) to detect abnormal conversations.

[0957] Step 4:

[0958] If the device detects abnormal behavior, it sends the data and analysis results to the server.

[0959] Input: Analysis results and anomaly detection data

[0960] Data calculation: Determining the type and urgency of abnormal behavior

[0961] Output: Notifications and detailed data to send to the server

[0962] Specific operation: When the device identifies abnormal behavior (e.g., suspicious transaction activity), it sends a notification to the server with detailed data attached.

[0963] Step 5:

[0964] The server determines the type and urgency of abnormal behavior and sends appropriate action instructions to the drone.

[0965] Input: Analysis results and anomaly detection data from the device

[0966] Data calculations: assessing urgency and determining appropriate actions

[0967] Output: Instructions to the drone

[0968] Specific operation: The server receives the analysis data and determines the urgency of the abnormal behavior as "high." It then sends a "warning" command to the drone, playing a message saying, "That behavior is illegal. Please stop immediately."

[0969] Step 6:

[0970] An emotion recognition engine analyzes the user's emotions and adjusts the system's overall response.

[0971] Input: Video and audio data from a drone or device

[0972] Data Computing: Recognizing Emotional States Using Emotion Analysis Algorithms

[0973] Output: System-wide response adjustment instructions

[0974] Specific operation: The emotion recognition engine analyzes video and audio data to determine the reaction of suspicious individuals, and adjusts the drone's and the system's overall response as needed.

[0975] Step 7:

[0976] If the server determines that an emergency situation exists, it will automatically notify the police.

[0977] Input: Emergency data from drones and devices

[0978] Data calculation: generating and sending message content

[0979] Output: Police report details and real-time data

[0980] Specific operation: The server determines the abnormal behavior as "highly urgent" and initiates a police notification system. The notification includes the drone's current location, detailed emergency data, and a real-time video link.

[0981] Step 8:

[0982] After the drone completes its patrol area, it sends all collected data to the server.

[0983] Input: Drone-collected data

[0984] Data calculation: storing in a database and analyzing

[0985] Output: Long-term data analysis results and reports

[0986] Specific operation: When the drone completes its patrol, it transmits the collected data to a server via Wi-Fi or 4G / 5G. The server stores the data in a database and performs trend analysis using AI. A report is generated and sent to the system administrator.

[0987] (Application example 2)

[0988] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0989] Improving safety in crime-prone areas requires efficient, real-time patrols and the detection of abnormal behavior. However, conventional systems are slow to detect and respond to abnormal behavior, making it difficult to quickly detect and deal with suspicious individuals and criminals. They also lack the ability to respond flexibly to on-site situations or take measures that take user emotions into consideration. This makes it difficult to ensure the safety and security of local residents.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0991] In this invention, the server includes a means for patrolling high-crime areas using drones, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, an emotion recognition means for analyzing the user's emotions, and a means for adjusting responses based on the emotion recognition results. This enables abnormal behavior to be detected in real time, enabling prompt and appropriate responses. Furthermore, flexible measures that take the user's emotions into account can increase the safety of the entire area and the sense of security of residents.

[0992] A "drone" is an unmanned aerial vehicle equipped with sensors and generative AI to patrol high-crime areas and detect abnormal behavior or situations.

[0993] A "sensor" is a device installed on a drone to detect changes in the environment or abnormal behavior, and is a device that collects data such as audio, video, and temperature.

[0994] "Generative AI" is artificial intelligence that analyzes collected data and detects abnormal behavior and situations in real time.

[0995] "Emotion recognition means" is a technology that analyzes the user's video and audio data and recognizes emotions in real time.

[0996] "Response adjustment means based on emotion recognition results" is a technology that adjusts the response of the drone and the entire system based on the emotional data analyzed by the emotion recognition means.

[0997] A "patrol schedule" is a server-generated plan of routes and time settings for drones to efficiently patrol high-crime areas.

[0998] "Real-time analysis means" is a technology that analyzes data collected from drones in real time and quickly detects abnormal behavior.

[0999] "Warning and advisory measures" are technologies that allow drones to issue warning messages to targets when abnormal behavior is detected.

[1000] "Emergency reporting means" is a technology that allows drones or systems to automatically report to the police when abnormal behavior is deemed to be an emergency.

[1001] As an embodiment of the present invention, the following system is constructed. Details of the system and the processing contents of each means will be explained using specific examples.

[1002] Overall system configuration

[1003] The system of the present invention is mainly composed of the following elements.

[1004] 1. Drones: They patrol high-crime areas and are equipped with cameras, audio sensors, environmental sensors, and generative AI.

[1005] 2. Server: Controls the entire system and is responsible for data analysis, generating drone patrol schedules, and emergency notification functions.

[1006] 3. Terminal: Analyzes data from the drone in real time, detects abnormal behavior, and issues instructions for response.

[1007] 4. Users: Includes system administrators, police operators, local residents, etc.

[1008] 5. Emotion recognition: Analyze the user's emotions and adjust the response.

[1009] Drone patrols and data collection

[1010] The drones patrol high-crime areas according to a pre-set patrol schedule, collecting data using various sensors (cameras, audio sensors, environmental sensors) and analyzing it in real time using generative AI.

[1011] Example: While a drone is patrolling a parking lot at night, the camera captures several people transacting something around a car, which is detected as anomalous behavior.

[1012] Data analysis and processing by the server

[1013] The server receives the data sent from the drone and uses the AI ​​to analyze any abnormal behavior. If a serious abnormality is detected, an emergency call is made.

[1014] Hardware / software used: high-performance computers (e.g., servers with NVIDIA GPUs), generative AI models (e.g., TensorFlow).

[1015] Example: A server analyzes video data captured by a drone and determines that a transaction is suspicious. It then sends a warning to the drone saying, "Fraudulent activity is prohibited here."

[1016] Emotion recognition and response adjustment

[1017] The emotion recognition means analyzes the user's voice and video data to recognize their emotions, and adjusts the drone's response based on the emotion recognition results.

[1018] Example: When a drone issues a warning, if the suspicious person responds aggressively, it will keep its distance, but if the person responds non-aggressively, it will continue to pay more attention.

[1019] Example prompt: "I'm patrolling a high-crime area and I've noticed some unusual behavior. What are my next steps?"

[1020] User operation

[1021] Users (system administrators and police operators) operate the server or terminals to configure the system, check patrol schedules, monitor abnormal behavior, and respond quickly to emergency calls.

[1022] This system can monitor the safety of high-crime areas in real time, detect and respond to abnormal behavior immediately, and take user emotions into consideration, enabling more appropriate security management.

[1023] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1024] Step 1:

[1025] The server generates the drone's patrol schedule. The inputs are geographic information of high-crime areas and past crime data. Using this data, the optimal patrol route and time is determined and sent to the drone. The output is the patrol schedule sent to the drone. Specifically, the server's AI algorithm retrieves past crime data from the database and compares it with map information to determine the patrol route.

[1026] Step 2:

[1027] The drone patrols high-crime areas based on a patrol schedule received from a server. The input is the patrol schedule from the server, and the output is collected real-time video, audio, and environmental data. Specifically, the drone automatically flies a set route, and the onboard cameras and sensors continuously record the surrounding situation.

[1028] Step 3:

[1029] The device receives real-time data transmitted from the drone and analyzes it using a generative AI model. The input is video, audio, and environmental data from the drone, and the output is the detection result of abnormal behavior. Specifically, the device's generative AI model analyzes the received data and applies an algorithm to detect abnormal behavior or situations.

[1030] Step 4:

[1031] The server receives the anomaly analysis results sent from the device and determines the type of anomaly and its level of urgency. The input is the anomaly detection result from the device, and the output is instructions for the drone to take action and, if necessary, a report to the police. Specifically, the server instructs the drone to take appropriate action (such as issuing a warning or sending a warning message) based on the analysis results, and if the level of urgency is high, it automatically reports the anomaly to the police.

[1032] Step 5:

[1033] The drone issues warnings and advice on-site based on instructions from the server. The input is an action instruction from the server, and the output is the playback of a voice message or a specific action (for example, a change in flight pattern). Specific actions include the drone playing a specified message over a speaker and warning suspicious individuals as necessary.

[1034] Step 6:

[1035] The emotion recognition means analyzes the subject's reactions and adjusts the drone's response. The input is video and audio data from the drone's camera and microphone, and the output is the emotion recognition results and instructions for the drone to act. Specifically, the AI ​​model analyzes the video and audio data to recognize the subject's emotions. If an aggressive reaction is observed, the drone will maintain its distance, and if the reaction is non-aggressive, it will issue further warnings.

[1036] Step 7:

[1037] The server accumulates the collected data, analyzes long-term crime trends, and generates reports. The input is all data transmitted by the drones, and the output is analysis results and reports. Specifically, the server stores the collected data in a database, periodically analyzes the data, and generates reports that identify crime trends and problem areas.

[1038] Step 8:

[1039] The user (system administrator or police operator) receives the report and modifies the patrol plan or takes additional measures as necessary. The input is the report from the server, and the output is the modified patrol plan or additional measures. In concrete terms, the user checks the contents of the report and sets up new patrol routes or additional crime prevention measures.

[1040] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1041] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1042] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1043] [Third embodiment]

[1044] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1045] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1046] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1047] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1048] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1049] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1050] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1051] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1052] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1053] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1054] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1055] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1056] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system mainly consists of four elements: a drone, a server, a terminal, and a user, all of which work in close coordination with each other.

[1057] System configuration

[1058] 1. Drones

[1059] The drone is equipped with various sensors such as a camera, audio sensor, and environmental sensor, as well as generative AI.

[1060] The drones automatically patrol designated crime-prone areas based on instructions from the server.

[1061] If the battery level gets low, it can automatically return and recharge.

[1062] 2. Server

[1063] The server is responsible for the overall chain of command and data analysis.

[1064] Generate a schedule for the patrol area and send it to the drone.

[1065] It receives data sent from the drone in real time and instructs appropriate actions based on the analysis results.

[1066] It also has a function to report to the police if abnormal behavior or situations are detected.

[1067] 3. Terminal

[1068] The device is equipped with generative AI that quickly analyzes data sent from the drone in real time.

[1069] Detects abnormal behavior and situations and reports the results to the server.

[1070] 4. Users

[1071] Users include system administrators, police operators, and local residents.

[1072] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[1073] Police operators receive emergency calls and respond quickly to the scene.

[1074] Program processing overview

[1075] Preparing and conducting patrols

[1076] The server generates a patrol schedule for high-crime areas based on data from the patrol area and sends it to the drone.

[1077] The drone automatically patrols a set route and collects data using various sensors.

[1078] The device analyzes the data in real time and detects abnormal behavior or situations.

[1079] Detecting and responding to abnormal behavior

[1080] If the device detects an abnormality, it immediately reports the situation to the server.

[1081] The server receives the report and analyzes the type and urgency of the abnormal behavior.

[1082] The server issues warnings and advice to the drone.

[1083] The drone approaches anyone behaving abnormally and uses audio messages and warning lights to warn them.

[1084] Emergency response

[1085] If the server determines that the emergency is high, it will automatically notify the police.

[1086] The server sends the drone's current location and real-time video data to the police.

[1087] The user (police operator) receives the report and arranges for a response at the scene.

[1088] Data Collection and Reporting

[1089] After the drone has completed its designated patrol area, it sends all collected data to the server.

[1090] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[1091] The server generates a report summarizing the analysis results and provides it to the user.

[1092] The user receives the report and takes any necessary measures or adjustments.

[1093] Specific examples

[1094] Example 1: Detecting suspicious activity in a parking lot at night

[1095] The server sends the drone a schedule for patrolling the parking lot at night.

[1096] As the drone patrols the parking lot, it captures several people trading something around a car.

[1097] The device analyzes the video data and determines that the transaction is suspicious.

[1098] The server analyzes the situation and issues instructions to the drone to warn it off.

[1099] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[1100] If it is determined that police are needed, the server will notify them and they will rush to the scene.

[1101] Example 2: Detecting suspected fraudulent calls

[1102] The server sends the drone a patrol schedule for areas prone to fraud.

[1103] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1104] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1105] The server analyzes the situation and provides advice and instructions to the drone.

[1106] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[1107] If necessary, the server will notify the police, who will respond to the scene.

[1108] This invention is designed to automate security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, thereby improving overall community safety and enhancing residents' sense of security.

[1109] The processing flow will be explained below.

[1110] Step 1:

[1111] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[1112] Step 2:

[1113] The server sends the generated patrol schedule to the drone.

[1114] Step 3:

[1115] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[1116] Step 4:

[1117] The server confirms that the drone is ready and sends a signal to activate the drone.

[1118] Step 5:

[1119] The drone will begin flying automatically, following the set patrol route.

[1120] Step 6:

[1121] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[1122] Step 7:

[1123] The data collected by the drone is sent to the device in real time.

[1124] Step 8:

[1125] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[1126] Step 9:

[1127] If the device detects an abnormality, it transfers the data and analysis results to the server.

[1128] Step 10:

[1129] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[1130] Step 11:

[1131] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[1132] Step 12:

[1133] The drone will play messages and turn on warning lights to deter abnormal behavior.

[1134] Step 13:

[1135] The server will reassess the situation and notify the police if necessary.

[1136] Step 14:

[1137] The server sends the drone's current location and real-time video data to the police.

[1138] Step 15:

[1139] The user (police operator) confirms the report and arranges for on-site response.

[1140] Step 16:

[1141] After the drone completes its patrol, it sends all collected data to a server.

[1142] Step 17:

[1143] The server stores the collected data and analyzes long-term crime trends and problem areas.

[1144] Step 18:

[1145] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[1146] Step 19:

[1147] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[1148] Example 1

[1149] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1150] Conventional security patrol systems were unable to efficiently patrol high-crime areas, making it difficult to detect and respond to abnormal behavior or situations in real time. Another issue was delays in reporting emergencies to the police, making it difficult to respond quickly.

[1151] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1152] In this invention, the server includes a means for generating a patrol schedule for high-crime areas based on patrol area data and transmitting it to the drone, a means for the drone to detect abnormal behavior or situations in real time using various sensors and generation AI and transmit that data to the terminal, and a means for issuing warnings or advice when an abnormal situation is detected. This enables highly efficient detection and response to abnormal behavior in real time.

[1153] "Patrol area" refers to a specific area or region in a high-crime neighborhood that will be patrolled by a drone.

[1154] A "drone" is an unmanned aerial vehicle that operates by remote control or automatic pilot and is equipped with a camera and various sensors.

[1155] A "server" is a computer that is responsible for the chain of command and management of the entire system, including collecting data, analyzing it, and sending instructions.

[1156] A "terminal" is a computer that receives and analyzes data sent from the drone.

[1157] "Generative AI" is part of a system that uses artificial intelligence technology to analyze data and make decisions, and refers to an algorithm that utilizes a generative model.

[1158] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is likely to be criminal or dangerous.

[1159] A "warning" is a warning or advice given to discourage certain behavior.

[1160] An "emergency" is a situation in which a serious danger or crime is in progress and requires a rapid response.

[1161] "Reporting" refers to the act of informing the police or other relevant authorities of abnormal behavior or an emergency.

[1162] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system is mainly composed of four elements: a server, a terminal, a drone, and a user, and each element works in close coordination with the others.

[1163] Hardware and software used

[1164] Server: A computer with high-performance data analysis capabilities and the processing power to operate generative AI models. Specifically, it is a server equipped with a large-capacity hard disk and a high-speed processor.

[1165] Device: A computer that receives and analyzes data in real time and runs the generative AI model. This device can be a laptop or a high-performance desktop computer.

[1166] Drone: An unmanned aerial vehicle equipped with cameras, audio sensors, and environmental sensors, with autopilot capabilities, a battery management system, and the ability to return automatically when it needs to be recharged.

[1167] Generative AI model: An artificial intelligence algorithm for detecting abnormal behavior or situations. For example, TensorFlow is used for visual analysis, and PyTorch is used for audio analysis.

[1168] Program processing

[1169] 1. The server collects data on the patrol area and generates a patrol schedule using a generative AI model. The generated schedule includes prompts such as "Patrol the parking lot at night and detect suspicious behavior."

[1170] 2. The server sends the generated patrol schedule to the drone and issues patrol instructions to the drone.

[1171] 3. The drone patrols the set area on autopilot, collecting video data from the camera, audio data from the audio sensor, and temperature and humidity data from the environmental sensors.

[1172] 4. The data collected by the drone is sent to the terminal in real time via the drone's communication module using Wi-Fi or 4G / 5G networks.

[1173] 5. The device quickly analyzes the data it receives using a generative AI model, inputs a prompt such as "Analyze the collected data and detect any abnormalities," and reports any abnormal behavior or abnormal situations to the server.

[1174] 6. The server receives the report and evaluates the type of abnormal behavior (e.g., voyeurism, violent behavior) and the urgency.

[1175] 7. The server issues warnings and advice to the drone, such as "play a warning message" or "turn on a warning light."

[1176] 8. The drone will play a specified voice message (e.g., "Misconduct in this area is prohibited. Please leave immediately.") and turn on a warning light.

[1177] 9. If the server determines that the abnormal behavior is of high urgency, it will automatically notify the police. The notification process will send the drone's current location, real-time video data, and detailed information about the abnormal behavior.

[1178] 10. After the drone has completed its designated patrol area, it will send all collected data to the server.

[1179] 11. The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[1180] 12. The server generates a report summarizing the analysis results and provides it to the user, who then takes necessary measures and makes adjustments based on the report.

[1181] Specific examples of programs

[1182] 1. Detecting suspicious activity in parking lots at night:

[1183] The server sends the drone a schedule for patrolling the parking lot at night.

[1184] As the drone patrols the parking lot, it captures several people conducting transactions around a car.

[1185] The device analyzes the video data and determines that the transaction is suspicious.

[1186] The server analyzes the situation and issues instructions to the drone to warn it off.

[1187] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[1188] If necessary, the server will notify the police, who will rush to the scene.

[1189] 2. Detecting Suspected Fraudulent Calls:

[1190] The server sends the drone a patrol schedule for areas prone to fraud.

[1191] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1192] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1193] The server analyzes the situation and provides advice and instructions to the drone.

[1194] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[1195] If necessary, the server will notify the police, who will respond to the scene.

[1196] In this way, this invention enables efficient security patrols in high-crime areas, enabling highly accurate detection of abnormal behavior and rapid response, thereby improving the safety of the entire area and increasing the sense of security for residents.

[1197] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1198] Step 1:

[1199] The server collects data for the patrol area. Specifically, the server imports past data on crime hotspots and analyzes crime rates by time of day and day of the week. The input is crime occurrence data, and the output is a crime risk map for each area as a result of the analysis.

[1200] Step 2:

[1201] The server uses the generative AI model to generate a schedule for the patrol area. The server inputs a prompt statement, "Please generate a patrol schedule for a specific area," into the AI ​​model and obtains the patrol schedule generated by the AI. The inputs are a crime risk map and the prompt statement, and the output is the patrol schedule.

[1202] Step 3:

[1203] The server sends the generated patrol schedule to the drone. Specifically, it transfers a data packet containing the generated schedule to the drone via Wi-Fi or 4G / 5G networks. The input is the patrol schedule, and the output is a confirmation that the transmission to the drone has been completed.

[1204] Step 4:

[1205] The server issues a "start patrol" command to the drone. The drone starts patrolling the designated area in autopilot mode. The input is the start patrol command, and the output is the drone starting to patrol.

[1206] Step 5:

[1207] The drone collects data in real time using various sensors (camera, audio sensor, environmental sensor). The input is environmental data within the patrol area, and the output is the collected sensor data.

[1208] Step 6:

[1209] The data collected by the drone is sent to the terminal in real time. Specifically, the data is sent to the terminal via Wi-Fi or 4G / 5G networks using the drone's communication module. The input is various sensor data, and the output is confirmation that the data has been sent to the terminal.

[1210] Step 7:

[1211] The data received by the device is quickly analyzed using a generative AI model. The device inputs the prompt "Analyze the collected data and detect abnormalities" into the AI ​​model to detect abnormal behavior or situations. The input is various sensor data and the prompt, and the output is the detection result of abnormal behavior or situations.

[1212] Step 8:

[1213] The device reports abnormal behavior or abnormal conditions to the server. Specifically, it sends a data packet containing the abnormality detection result to the server. The input is the abnormality detection result, and the output is a confirmation that the report has been sent to the server.

[1214] Step 9:

[1215] The server analyzes the received abnormal behavior reports and evaluates their urgency. The input is the abnormal behavior report data, and the output is the urgency evaluation result of the abnormal behavior.

[1216] Step 10:

[1217] The server issues warnings and advice to the drone. For example, it sends instructions such as "play a warning message" or "turn on a warning light." The input is the warning instruction, and the output is confirmation that the instruction has been sent to the drone.

[1218] Step 11:

[1219] The drone plays a specified audio message and turns on a warning light. For example, it might play the message "Misconduct in this location is prohibited. Please leave immediately." The input is the warning instruction, and the output is the actual warning action.

[1220] Step 12:

[1221] If the server determines that the abnormal behavior is of high urgency, it automatically notifies the police. The server sends the drone's current location, real-time video data, and detailed information about the abnormal behavior to the police. The input is the trigger for the emergency call, and the output is confirmation that the call to the police has been completed.

[1222] Step 13:

[1223] After the drone finishes its patrol area, it sends all collected data to the server. The input is the data collected after the patrol, and the output is a confirmation that the data has been sent to the server.

[1224] Step 14:

[1225] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior. The input is the collected data, and the output is the analysis results.

[1226] Step 15:

[1227] The server generates a report summarizing the analysis results and provides it to the user. The user then takes necessary measures and makes adjustments based on the report. The input is the analysis results, and the output is the report.

[1228] (Application example 1)

[1229] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1230] There is a need for efficient patrols, detection of abnormal behavior, and rapid response in high-crime areas. However, current systems rely heavily on manual patrols and monitoring, which are costly and time-consuming and make it difficult to respond in real time. Furthermore, delayed response in emergencies can have fatal consequences. To solve this problem, a more efficient and rapid method is needed.

[1231] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1232] In this invention, the server includes means for patrolling high-crime areas by drones, means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, means for issuing warnings and advice when an abnormal situation is detected, means for notifying the police in an emergency, means for transmitting data collected from the drone to a smartphone in real time, means for delivering the real-time data to a user terminal in streaming format, and means for sending a push notification to the user terminal when abnormal behavior is detected. This enables efficient patrols, rapid detection of abnormal behavior, real-time information provision to users, and rapid response in emergencies.

[1233] A "drone" is an unmanned aerial vehicle that flies remotely or autonomously and is equipped with cameras and sensors to collect data.

[1234] A "high-crime area" is a specific area where crimes occur frequently based on past data and statistics.

[1235] "Abnormal behavior" refers to behavior that differs from normal behavior and is suspected of being criminal or fraudulent.

[1236] An "abnormal situation" is a situation that differs from the normal environment or circumstances and suggests that abnormal or illegal activity is occurring.

[1237] A "sensor" is a device that collects environmental information and converts it into an analog or digital signal.

[1238] "Generative AI" is an artificial intelligence technology that identifies and judges abnormal behavior and situations based on large amounts of data.

[1239] "Warning" refers to the act of giving a warning or advice to a target person.

[1240] "Advice" refers to the act of making suggestions or instructions to encourage appropriate behavior by a person.

[1241] An "emergency" is an extraordinary situation that requires immediate action.

[1242] "Police" refers to a public institution whose primary mission is to prevent crime and maintain public order.

[1243] "Reporting" refers to the act of reporting an incident or abnormal situation to the relevant authorities.

[1244] "Real-time" refers to a state in which data collection and processing occur immediately, without delay.

[1245] A "smartphone" is a mobile phone with computing power and internet connectivity.

[1246] "Streaming format" refers to a technology that transmits data continuously and receives and plays it back in real time.

[1247] A "user terminal" is a digital device that displays data and performs operations.

[1248] "Push notifications" refers to a technology that automatically sends messages from a web server to a client device.

[1249] Overall system configuration

[1250] This invention is a system consisting of a drone, a server, a terminal, and a user. Each element works together to efficiently patrol high-crime areas, detect abnormal behavior, issue warnings, and notify the police in emergencies.

[1251] 1. Drones

[1252] Drones are autonomous unmanned aerial vehicles equipped with cameras, audio sensors, and environmental sensors. They patrol crime-prone areas under instructions from a server and transmit data collected by various sensors to the server in real time. Furthermore, drones are equipped with generative AI, allowing them to analyze some of the data themselves.

[1253] The drone is equipped with a battery management system that allows it to automatically return to a charging station and recharge when the battery level is low.

[1254] 2. Server

[1255] The server is responsible for the chain of command for the entire system and data analysis. It sends patrol area schedules to the drones and receives and analyzes the data sent in real time. The server is equipped with advanced generative AI to detect and analyze abnormal behavior. If an abnormality is detected, the server sends a warning instruction to the drone and, if necessary, sends a push notification to the user's smartphone. It also has a function to automatically notify the police in an emergency. The server can provide real-time data to the user in streaming format. It also accumulates and analyzes the collected data and generates periodic reports.

[1256] 3. Terminal

[1257] The device is equipped with a generative AI that analyzes data sent from the drone in real time. If it detects abnormal behavior or an abnormal situation, it immediately reports the results to the server. The device also provides an interface for users to configure the system, check patrol schedules, and monitor abnormal behavior. It also has the function of sending a push notification to the user's device if abnormal behavior is detected.

[1258] 4. Users

[1259] Users include system administrators, police operators, and local residents. Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior. In an emergency, users can manually notify the police from their smartphones.

[1260] Natural language description of the process

[1261] The drone patrols designated high-crime areas and collects data using various sensors. The collected data is sent to a server in real time via 4G / 5G. On the server, a generating AI analyzes the data and detects abnormal behavior or situations. If an abnormality is detected, the server sends a warning command to the drone and a push notification to the user's smartphone. In the event of an emergency, the server automatically notifies the police. Users can view the real-time data in streaming format on their smartphone and can also make additional reports as needed.

[1262] Examples of specific examples and prompts

[1263] Specific examples

[1264] While a drone is patrolling a parking lot at night, the camera captures several people engaging in suspicious transactions. This video data is sent to a server in real time, and the generating AI determines that the activity is suspicious. The server immediately sends a warning command to the drone, which plays a message saying, "Fraudulent activity is prohibited. Please leave the area immediately." The server also sends a push notification to the user's smartphone, allowing them to review the video in real time. If necessary, the server automatically notifies the police.

[1265] Prompt Sentence Examples

[1266] "While a drone is patrolling a parking lot at night, several people appear to be conducting suspicious transactions. Based on this data, detect suspicious activity, send a warning command to the drone, and send a real-time notification to the user's smartphone. Also, describe the process for automatically notifying the police if necessary."

[1267] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1268] Step 1:

[1269] The server generates a patrol schedule for high-crime areas. It uses area data and time data as input and generates the schedule based on this. The generated schedule data is sent as output to the drone. This sets the drone to patrol the specified area at the specified time.

[1270] Step 2:

[1271] The drone begins patrolling high-crime areas based on the received schedule. It uses schedule data from the server as input and environmental data and video data obtained by sensors and cameras as output, sending them to the server in real time.

[1272] Step 3:

[1273] The server receives data sent from the drone in real time and analyzes it using generative AI. Video data, audio data, and environmental data are used as input, and data processing is performed to analyze and detect abnormal behavior and abnormal situations. The output is the detection result of abnormal behavior and abnormal situations.

[1274] Step 4:

[1275] If the server detects abnormal behavior or an abnormal situation, it sends a warning or advice to the drone. It uses the detection result data as input and sends warning instruction data to the drone as output. Specific actions include having the drone play an audio message such as, "Unauthorized activity is prohibited in this location. Please leave the area immediately."

[1276] Step 5:

[1277] If the server detects abnormal behavior or an abnormal situation, it sends a push notification to the user's smartphone. It uses the detection result data as input and sends a notification message to the user's device as output. Specifically, it sends a notification message to the user saying, "Emergency: Suspicious person has entered the premises!"

[1278] Step 6:

[1279] If the server detects an emergency, it automatically notifies the police. It uses the detection result data and location data as input, and sends a notification message to the police system as output. Specifically, it provides the police with the drone's current location and real-time video data.

[1280] Step 7:

[1281] After the drone completes its patrol, it sends all collected data to the server. It uses the environmental data, video data, and audio data collected during the patrol as input, and sends all data to the server as output.

[1282] Step 8:

[1283] The server stores the collected data and uses generation AI to analyze it. The collected data is used as input, and data processing is performed to extract the frequency and trends of abnormal behavior. The analysis results are obtained as output. Specific operations include generating statistical data on the location and frequency of abnormal behavior.

[1284] Step 9:

[1285] The server generates a report summarizing the analysis results and provides it to the user. It uses the analysis result data as input, processes the data, and formats it into a report format. It then sends the report data to the user's terminal as output.

[1286] Step 10:

[1287] The user receives the report and takes necessary measures and adjustments. The report data is used as input, and the output is used to adjust local safety measures and patrol schedules. Specific actions include setting up new patrol areas and strengthening crime prevention measures.

[1288] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1289] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. This system consists of five main components: a drone, a server, a terminal, a user, and an emotion engine. The detailed functions and linked operations of each component are described below.

[1290] System configuration

[1291] 1. Drones

[1292] The drone is equipped with a camera, audio sensors, environmental sensors, and generative AI.

[1293] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[1294] It detects abnormal behavior and situations and sends the data to the server in real time.

[1295] 2. Server

[1296] The server is responsible for controlling the entire system and analyzing data.

[1297] Generate a schedule for the patrol area and send it to the drone.

[1298] It receives data transmitted from the drone and analyzes it using generative AI to detect abnormal behavior and determine appropriate action.

[1299] In case of an emergency, it will notify the police and provide the drone's location and real-time data.

[1300] 3. Terminal

[1301] The device is equipped with generative AI that analyzes data sent from the drone in real time.

[1302] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[1303] 4. Users

[1304] Users include system administrators, police operators, local residents, etc.

[1305] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[1306] Police operators receive emergency calls and respond quickly to the scene.

[1307] 5. Emotion Engine

[1308] The emotion engine analyzes the user's video and audio data and has the ability to recognize emotions in real time.

[1309] The emotion engine adjusts the response of the drone and the entire system based on the emotions it recognizes.

[1310] Program processing overview

[1311] Preparing and conducting patrols

[1312] The server generates a patrol schedule based on data on the patrol area and sends it to the drone.

[1313] The drone automatically patrols a set route and collects data using various sensors.

[1314] The device analyzes the data in real time and detects abnormal behavior or situations.

[1315] Detecting and responding to abnormal behavior

[1316] If the device detects an abnormality, it transfers the data and analysis results to the server.

[1317] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[1318] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[1319] The emotion engine analyzes the user's emotions and adjusts the drone's response.

[1320] Emergency response

[1321] If the server determines that the emergency is high, it will automatically notify the police.

[1322] The server sends the drone's current location and real-time video data to the police.

[1323] The user (police operator) confirms the report and arranges for on-site response.

[1324] Data Collection and Reporting

[1325] After the drone has completed its designated patrol area, it sends all collected data to the server.

[1326] The server stores the collected data and analyzes long-term crime trends and problem areas.

[1327] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[1328] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[1329] Specific examples

[1330] Example 1: Detecting suspicious activity in a parking lot at night

[1331] The server sends the drone a schedule for patrolling the parking lot at night.

[1332] As the drone patrols the parking lot, it captures several people trading something around a car.

[1333] The device analyzes the video data and determines that the transaction is suspicious.

[1334] The server analyzes the situation and issues instructions to the drone to warn it off.

[1335] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[1336] At the same time, the emotion engine analyzes the suspicious person's reaction and determines whether additional action (e.g., immediate police reporting) is necessary.

[1337] Example 2: Detecting suspected fraudulent calls

[1338] The server sends the drone a patrol schedule for areas prone to fraud.

[1339] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1340] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1341] The server analyzes the situation and provides advice and instructions to the drone.

[1342] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[1343] An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[1344] If necessary, the server will notify the police, who will respond to the scene.

[1345] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[1346] The processing flow will be explained below.

[1347] Step 1:

[1348] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[1349] Step 2:

[1350] The server sends the generated patrol schedule to the drone.

[1351] Step 3:

[1352] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[1353] Step 4:

[1354] The server confirms that the drone is ready and sends a signal to activate the drone.

[1355] Step 5:

[1356] The drone will begin flying automatically, following the set patrol route.

[1357] Step 6:

[1358] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[1359] Step 7:

[1360] The data collected by the drone is sent to the device in real time.

[1361] Step 8:

[1362] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[1363] Step 9:

[1364] If the device detects an abnormality, it transfers the data and analysis results to the server.

[1365] Step 10:

[1366] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[1367] Step 11:

[1368] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[1369] Step 12:

[1370] The emotion engine analyzes the video and audio data of suspicious individuals and recognizes their emotions in real time.

[1371] Step 13:

[1372] Based on the emotions recognized by the emotion engine, the server further adjusts the drone's response.

[1373] Step 14:

[1374] Depending on the emotion the drone recognizes, for example, if tension is detected, it will play an appropriate additional message, such as "This behavior has been recorded. Please leave the area immediately."

[1375] Step 15:

[1376] The server will reassess the situation and notify the police if necessary.

[1377] Step 16:

[1378] The server sends the drone's current location and real-time video data to the police.

[1379] Step 17:

[1380] The user (police operator) confirms the report and arranges for on-site response.

[1381] Step 18:

[1382] After the drone completes its patrol, it sends all collected data to a server.

[1383] Step 19:

[1384] The server stores the collected data and analyzes long-term crime trends and problem areas.

[1385] Step 20:

[1386] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[1387] Step 21:

[1388] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[1389] Example 2

[1390] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1391] Conventional security patrol systems do not adequately detect abnormal behavior in real time or respond immediately in high-crime areas. Furthermore, rather than simply detecting abnormalities, appropriate responses that take into account the user's emotions are required, but this is difficult to achieve with conventional systems. This has led to problems such as a decline in crime prevention capabilities and a lack of security for local residents.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1393] In this invention, the server includes a means for patrolling high-crime areas by drone, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, and an emotion recognition engine means for analyzing user emotions in real time and adjusting the response of the entire system. This enables rapid detection of abnormal behavior, immediate response, and response that takes user emotions into consideration.

[1394] A "drone" is an unmanned aircraft that can be remotely controlled and is equipped with cameras and various sensors to patrol crime-prone areas.

[1395] "Generative AI" is an artificial intelligence system that uses machine learning algorithms to recognize patterns from input data and automatically detect abnormal behavior or situations.

[1396] A "sensor" is a device that detects physical changes and outputs them as data, and in this system this includes cameras, audio sensors, and environmental sensors.

[1397] A "patrol schedule" is a plan that defines the drone's flight routes and time periods in high-crime areas.

[1398] "Warning" is an action that warns or advises the person involved or those around them when abnormal behavior or an abnormal situation is detected.

[1399] "Advice" is an action that instructs the appropriate way to respond to a detected abnormal situation.

[1400] An "emergency call" is the act of immediately notifying the police or relevant agencies when abnormal behavior or an abnormal situation occurs, and encouraging a prompt response.

[1401] An "emotion recognition engine" is a system that analyzes a user's video and audio data and recognizes their emotional state in real time.

[1402] "User" refers to anyone who uses this patrol system, such as a system administrator, police operator, or local resident.

[1403] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. The system is composed of the following main components: a drone, a server, a terminal, a user, and an emotion recognition engine.

[1404] System configuration

[1405] 1. Drones

[1406] Drones are equipped with cameras, audio sensors, environmental sensors, and generative AI. A typical drone is a general-purpose unmanned aerial vehicle (e.g., a general-purpose unmanned flying device) equipped with a camera and various sensors.

[1407] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[1408] Various sensors detect abnormal behavior and situations and send the data to a server in real time.

[1409] 2. Server

[1410] The server is responsible for controlling the entire system and analyzing data. A high-performance server machine (e.g., high-performance computer equipment) is used.

[1411] Generate a schedule for the patrol area and send it to the drone.

[1412] The data transmitted from the drone is analyzed and generative AI is used to detect abnormal behavior and determine appropriate actions.

[1413] In case of an emergency, the system will notify the police and provide the drone's location and real-time data.

[1414] 3. Terminal

[1415] The terminal is equipped with generative AI and analyzes data transmitted in real time from the drone, using high-performance AI computing platforms such as NVIDIA Jetson.

[1416] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[1417] 4. Users

[1418] Users include system administrators, police operators, local residents, and other users of the system.

[1419] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[1420] Police operators receive emergency calls and respond quickly to the scene.

[1421] 5. Emotion Recognition Engine

[1422] The emotion recognition engine analyzes the user's video and audio data and has the ability to recognize emotions in real time. Emotion recognition software such as Affectiva SDK is used.

[1423] The drone and the entire system can then adjust their response based on the emotions they recognize.

[1424] System Operation

[1425] The server generates an optimal patrol schedule based on data on the patrol area and sends it to the drone. The drone automatically patrols a set route and collects data using various sensors. The collected data is sent to the terminal in real time and analyzed by the generating AI. If the terminal detects abnormal behavior, it reports the data and analysis results to the server. The server determines the type and urgency of the abnormal behavior and instructs the drone to take appropriate action. In the event of an emergency, the server automatically notifies the police, and police operators respond promptly to the scene.

[1426] Specific examples

[1427] Example 1: Detecting suspicious activity in a parking lot at night

[1428] The server sends the drone a schedule for patrolling the parking lot at night.

[1429] As the drone patrols the parking lot, it captures several people trading something around a car.

[1430] The device analyzes the video data and determines that the transaction is suspicious.

[1431] The server analyzes the situation and issues instructions to the drone to warn it off.

[1432] As the drone approaches, it plays a message saying, "Unauthorized activity is prohibited. Please leave the area immediately." The emotion engine analyzes the suspicious person's reaction and considers additional measures if necessary.

[1433] Example 2: Detecting suspected fraudulent calls

[1434] The server sends the drone a patrol schedule for areas prone to fraud.

[1435] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1436] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1437] The server analyzes the situation and provides advice and instructions to the drone.

[1438] The drone approaches and plays a message: "This may be a scam. Please end this call and contact the police." An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[1439] Prompt Sentence Examples

[1440] "What should you do if a drone detects unauthorized activity in a parking lot at night?"

[1441] "Please tell me how the device and server work together when the drone detects a conversation that is suspected to be fraudulent."

[1442] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[1443] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1444] Step 1:

[1445] The server generates an optimal tour schedule based on the data of the tour area.

[1446] Input: Geographic data of high-crime areas and historical crime records

[1447] Data processing: Generative AI models are used to analyze geographic data and crime records to optimize patrol routes and times.

[1448] Output: Tour schedule

[1449] Specific operation: The server retrieves crime data from the database for the past year, analyzes it using a generative AI model (e.g., a general-purpose AI platform), generates an optimal patrol schedule, and transmits it to the drone.

[1450] Step 2:

[1451] The drone receives the patrol schedule and automatically patrols the set route.

[1452] Input: Tour schedule sent from the server

[1453] Data calculation: Uses GPS data and internal map data to navigate routes

[1454] Output: Route execution status and collected data

[1455] How it works: The drone travels to a designated starting point, flies a pre-defined route based on GPS and internal map data, and collects data from its surroundings using cameras and audio sensors.

[1456] Step 3:

[1457] The device receives data transmitted from the drone in real time and analyzes it using a generative AI model.

[1458] Input: Video data, audio data, and environmental data from drones

[1459] Data calculation: Analyzes video data and audio data to identify abnormal behavior

[1460] Output: Detection results and analysis data

[1461] Specific operation: The device receives a data stream from the drone in real time. The video data is analyzed using a TensorFlow model to identify specific behavioral patterns (e.g., a person running away). The audio data is analyzed using a voice recognition system (e.g., a general-purpose voice analysis system) to detect abnormal conversations.

[1462] Step 4:

[1463] If the device detects abnormal behavior, it sends the data and analysis results to the server.

[1464] Input: Analysis results and anomaly detection data

[1465] Data calculation: Determining the type and urgency of abnormal behavior

[1466] Output: Notifications and detailed data to send to the server

[1467] Specific operation: When the device identifies abnormal behavior (e.g., suspicious transaction activity), it sends a notification to the server with detailed data attached.

[1468] Step 5:

[1469] The server determines the type and urgency of abnormal behavior and sends appropriate action instructions to the drone.

[1470] Input: Analysis results and anomaly detection data from the device

[1471] Data calculations: assessing urgency and determining appropriate actions

[1472] Output: Instructions to the drone

[1473] Specific operation: The server receives the analysis data and determines the urgency of the abnormal behavior as "high." It then sends a "warning" command to the drone, playing a message saying, "That behavior is illegal. Please stop immediately."

[1474] Step 6:

[1475] An emotion recognition engine analyzes the user's emotions and adjusts the system's overall response.

[1476] Input: Video and audio data from a drone or device

[1477] Data Computing: Recognizing Emotional States Using Emotion Analysis Algorithms

[1478] Output: System-wide response adjustment instructions

[1479] Specific operation: The emotion recognition engine analyzes video and audio data to determine the reaction of suspicious individuals, and adjusts the drone's and the system's overall response as needed.

[1480] Step 7:

[1481] If the server determines that an emergency situation exists, it will automatically notify the police.

[1482] Input: Emergency data from drones and devices

[1483] Data calculation: generating and sending message content

[1484] Output: Police report details and real-time data

[1485] Specific operation: The server determines the abnormal behavior as "highly urgent" and initiates a police notification system. The notification includes the drone's current location, detailed emergency data, and a real-time video link.

[1486] Step 8:

[1487] After the drone completes its patrol area, it sends all collected data to the server.

[1488] Input: Drone-collected data

[1489] Data calculation: storing in a database and analyzing

[1490] Output: Long-term data analysis results and reports

[1491] Specific operation: When the drone completes its patrol, it transmits the collected data to a server via Wi-Fi or 4G / 5G. The server stores the data in a database and performs trend analysis using AI. A report is generated and sent to the system administrator.

[1492] (Application example 2)

[1493] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1494] Improving safety in crime-prone areas requires efficient, real-time patrols and the detection of abnormal behavior. However, conventional systems are slow to detect and respond to abnormal behavior, making it difficult to quickly detect and deal with suspicious individuals and criminals. They also lack the ability to respond flexibly to on-site situations or take measures that take user emotions into consideration. This makes it difficult to ensure the safety and security of local residents.

[1495] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1496] In this invention, the server includes a means for patrolling high-crime areas using drones, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, an emotion recognition means for analyzing the user's emotions, and a means for adjusting responses based on the emotion recognition results. This enables abnormal behavior to be detected in real time, enabling prompt and appropriate responses. Furthermore, flexible measures that take the user's emotions into account can increase the safety of the entire area and the sense of security of residents.

[1497] A "drone" is an unmanned aerial vehicle equipped with sensors and generative AI to patrol high-crime areas and detect abnormal behavior or situations.

[1498] A "sensor" is a device installed on a drone to detect changes in the environment or abnormal behavior, and is a device that collects data such as audio, video, and temperature.

[1499] "Generative AI" is artificial intelligence that analyzes collected data and detects abnormal behavior and situations in real time.

[1500] "Emotion recognition means" is a technology that analyzes the user's video and audio data and recognizes emotions in real time.

[1501] "Response adjustment means based on emotion recognition results" is a technology that adjusts the response of the drone and the entire system based on the emotional data analyzed by the emotion recognition means.

[1502] A "patrol schedule" is a server-generated plan of routes and time settings for drones to efficiently patrol high-crime areas.

[1503] "Real-time analysis means" is a technology that analyzes data collected from drones in real time and quickly detects abnormal behavior.

[1504] "Warning and advisory measures" are technologies that allow drones to issue warning messages to targets when abnormal behavior is detected.

[1505] "Emergency reporting means" is a technology that allows drones or systems to automatically report to the police when abnormal behavior is deemed to be an emergency.

[1506] As an embodiment of the present invention, the following system is constructed. Details of the system and the processing contents of each means will be explained using specific examples.

[1507] Overall system configuration

[1508] The system of the present invention is mainly composed of the following elements.

[1509] 1. Drones: They patrol high-crime areas and are equipped with cameras, audio sensors, environmental sensors, and generative AI.

[1510] 2. Server: Controls the entire system and is responsible for data analysis, generating drone patrol schedules, and emergency notification functions.

[1511] 3. Terminal: Analyzes data from the drone in real time, detects abnormal behavior, and issues instructions for response.

[1512] 4. Users: Includes system administrators, police operators, local residents, etc.

[1513] 5. Emotion recognition: Analyze the user's emotions and adjust the response.

[1514] Drone patrols and data collection

[1515] The drones patrol high-crime areas according to a pre-set patrol schedule, collecting data using various sensors (cameras, audio sensors, environmental sensors) and analyzing it in real time using generative AI.

[1516] Example: While a drone is patrolling a parking lot at night, the camera captures several people transacting something around a car, which is detected as anomalous behavior.

[1517] Data analysis and processing by the server

[1518] The server receives the data sent from the drone and uses the AI ​​to analyze any abnormal behavior. If a serious abnormality is detected, an emergency call is made.

[1519] Hardware / software used: high-performance computers (e.g., servers with NVIDIA GPUs), generative AI models (e.g., TensorFlow).

[1520] Example: A server analyzes video data captured by a drone and determines that a transaction is suspicious. It then sends a warning to the drone saying, "Fraudulent activity is prohibited here."

[1521] Emotion recognition and response adjustment

[1522] The emotion recognition means analyzes the user's voice and video data to recognize their emotions, and adjusts the drone's response based on the emotion recognition results.

[1523] Example: When a drone issues a warning, if the suspicious person responds aggressively, it will keep its distance, but if the person responds non-aggressively, it will continue to pay more attention.

[1524] Example prompt: "I'm patrolling a high-crime area and I've noticed some unusual behavior. What are my next steps?"

[1525] User operation

[1526] Users (system administrators and police operators) operate the server or terminals to configure the system, check patrol schedules, monitor abnormal behavior, and respond quickly to emergency calls.

[1527] This system can monitor the safety of high-crime areas in real time, detect and respond to abnormal behavior immediately, and take user emotions into consideration, enabling more appropriate security management.

[1528] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1529] Step 1:

[1530] The server generates the drone's patrol schedule. The inputs are geographic information of high-crime areas and past crime data. Using this data, the optimal patrol route and time is determined and sent to the drone. The output is the patrol schedule sent to the drone. Specifically, the server's AI algorithm retrieves past crime data from the database and compares it with map information to determine the patrol route.

[1531] Step 2:

[1532] The drone patrols high-crime areas based on a patrol schedule received from a server. The input is the patrol schedule from the server, and the output is collected real-time video, audio, and environmental data. Specifically, the drone automatically flies a set route, and the onboard cameras and sensors continuously record the surrounding situation.

[1533] Step 3:

[1534] The device receives real-time data transmitted from the drone and analyzes it using a generative AI model. The input is video, audio, and environmental data from the drone, and the output is the detection result of abnormal behavior. Specifically, the device's generative AI model analyzes the received data and applies an algorithm to detect abnormal behavior or situations.

[1535] Step 4:

[1536] The server receives the anomaly analysis results sent from the device and determines the type of anomaly and its level of urgency. The input is the anomaly detection result from the device, and the output is instructions for the drone to take action and, if necessary, a report to the police. Specifically, the server instructs the drone to take appropriate action (such as issuing a warning or sending a warning message) based on the analysis results, and if the level of urgency is high, it automatically reports the anomaly to the police.

[1537] Step 5:

[1538] The drone issues warnings and advice on-site based on instructions from the server. The input is an action instruction from the server, and the output is the playback of a voice message or a specific action (for example, a change in flight pattern). Specific actions include the drone playing a specified message over a speaker and warning suspicious individuals as necessary.

[1539] Step 6:

[1540] The emotion recognition means analyzes the subject's reactions and adjusts the drone's response. The input is video and audio data from the drone's camera and microphone, and the output is the emotion recognition results and instructions for the drone to act. Specifically, the AI ​​model analyzes the video and audio data to recognize the subject's emotions. If an aggressive reaction is observed, the drone will maintain its distance, and if the reaction is non-aggressive, it will issue further warnings.

[1541] Step 7:

[1542] The server accumulates the collected data, analyzes long-term crime trends, and generates reports. The input is all data transmitted by the drones, and the output is analysis results and reports. Specifically, the server stores the collected data in a database, periodically analyzes the data, and generates reports that identify crime trends and problem areas.

[1543] Step 8:

[1544] The user (system administrator or police operator) receives the report and modifies the patrol plan or takes additional measures as necessary. The input is the report from the server, and the output is the modified patrol plan or additional measures. In concrete terms, the user checks the contents of the report and sets up new patrol routes or additional crime prevention measures.

[1545] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1546] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1547] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1548] [Fourth embodiment]

[1549] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1550] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1551] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1552] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1553] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1554] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1555] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1556] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1557] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1558] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1559] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1560] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1561] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1562] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system mainly consists of four elements: a drone, a server, a terminal, and a user, all of which work in close coordination with each other.

[1563] System configuration

[1564] 1. Drones

[1565] The drone is equipped with various sensors such as a camera, audio sensor, and environmental sensor, as well as generative AI.

[1566] The drones automatically patrol designated crime-prone areas based on instructions from the server.

[1567] If the battery level gets low, it can automatically return and recharge.

[1568] 2. Server

[1569] The server is responsible for the overall chain of command and data analysis.

[1570] Generate a schedule for the patrol area and send it to the drone.

[1571] It receives data sent from the drone in real time and instructs appropriate actions based on the analysis results.

[1572] It also has a function to report to the police if abnormal behavior or situations are detected.

[1573] 3. Terminal

[1574] The device is equipped with generative AI that quickly analyzes data sent from the drone in real time.

[1575] Detects abnormal behavior and situations and reports the results to the server.

[1576] 4. Users

[1577] Users include system administrators, police operators, and local residents.

[1578] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[1579] Police operators receive emergency calls and respond quickly to the scene.

[1580] Program processing overview

[1581] Preparing and conducting patrols

[1582] The server generates a patrol schedule for high-crime areas based on data from the patrol area and sends it to the drone.

[1583] The drone automatically patrols a set route and collects data using various sensors.

[1584] The device analyzes the data in real time and detects abnormal behavior or situations.

[1585] Detecting and responding to abnormal behavior

[1586] If the device detects an abnormality, it immediately reports the situation to the server.

[1587] The server receives the report and analyzes the type and urgency of the abnormal behavior.

[1588] The server issues warnings and advice to the drone.

[1589] The drone approaches anyone behaving abnormally and uses audio messages and warning lights to warn them.

[1590] Emergency response

[1591] If the server determines that the emergency is high, it will automatically notify the police.

[1592] The server sends the drone's current location and real-time video data to the police.

[1593] The user (police operator) receives the report and arranges for a response at the scene.

[1594] Data Collection and Reporting

[1595] After the drone has completed its designated patrol area, it sends all collected data to the server.

[1596] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[1597] The server generates a report summarizing the analysis results and provides it to the user.

[1598] The user receives the report and takes any necessary measures or adjustments.

[1599] Specific examples

[1600] Example 1: Detecting suspicious activity in a parking lot at night

[1601] The server sends the drone a schedule for patrolling the parking lot at night.

[1602] As the drone patrols the parking lot, it captures several people trading something around a car.

[1603] The device analyzes the video data and determines that the transaction is suspicious.

[1604] The server analyzes the situation and issues instructions to the drone to warn it off.

[1605] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[1606] If it is determined that police are needed, the server will notify them and they will rush to the scene.

[1607] Example 2: Detecting suspected fraudulent calls

[1608] The server sends the drone a patrol schedule for areas prone to fraud.

[1609] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1610] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1611] The server analyzes the situation and provides advice and instructions to the drone.

[1612] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[1613] If necessary, the server will notify the police, who will respond to the scene.

[1614] This invention is designed to automate security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, thereby improving overall community safety and enhancing residents' sense of security.

[1615] The processing flow will be explained below.

[1616] Step 1:

[1617] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[1618] Step 2:

[1619] The server sends the generated patrol schedule to the drone.

[1620] Step 3:

[1621] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[1622] Step 4:

[1623] The server confirms that the drone is ready and sends a signal to activate the drone.

[1624] Step 5:

[1625] The drone will begin flying automatically, following the set patrol route.

[1626] Step 6:

[1627] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[1628] Step 7:

[1629] The data collected by the drone is sent to the device in real time.

[1630] Step 8:

[1631] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[1632] Step 9:

[1633] If the device detects an abnormality, it transfers the data and analysis results to the server.

[1634] Step 10:

[1635] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[1636] Step 11:

[1637] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[1638] Step 12:

[1639] The drone will play messages and turn on warning lights to deter abnormal behavior.

[1640] Step 13:

[1641] The server will reassess the situation and notify the police if necessary.

[1642] Step 14:

[1643] The server sends the drone's current location and real-time video data to the police.

[1644] Step 15:

[1645] The user (police operator) confirms the report and arranges for on-site response.

[1646] Step 16:

[1647] After the drone completes its patrol, it sends all collected data to a server.

[1648] Step 17:

[1649] The server stores the collected data and analyzes long-term crime trends and problem areas.

[1650] Step 18:

[1651] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[1652] Step 19:

[1653] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[1654] Example 1

[1655] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1656] Conventional security patrol systems were unable to efficiently patrol high-crime areas, making it difficult to detect and respond to abnormal behavior or situations in real time. Another issue was delays in reporting emergencies to the police, making it difficult to respond quickly.

[1657] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1658] In this invention, the server includes a means for generating a patrol schedule for high-crime areas based on patrol area data and transmitting it to the drone, a means for the drone to detect abnormal behavior or situations in real time using various sensors and generation AI and transmit that data to the terminal, and a means for issuing warnings or advice when an abnormal situation is detected. This enables highly efficient detection and response to abnormal behavior in real time.

[1659] "Patrol area" refers to a specific area or region in a high-crime neighborhood that will be patrolled by a drone.

[1660] A "drone" is an unmanned aerial vehicle that operates by remote control or automatic pilot and is equipped with a camera and various sensors.

[1661] A "server" is a computer that is responsible for the chain of command and management of the entire system, including collecting data, analyzing it, and sending instructions.

[1662] A "terminal" is a computer that receives and analyzes data sent from the drone.

[1663] "Generative AI" is part of a system that uses artificial intelligence technology to analyze data and make decisions, and refers to an algorithm that utilizes a generative model.

[1664] "Abnormal behavior" refers to behavior that deviates from normal patterns of behavior and is likely to be criminal or dangerous.

[1665] A "warning" is a warning or advice given to discourage certain behavior.

[1666] An "emergency" is a situation in which a serious danger or crime is in progress and requires a rapid response.

[1667] "Reporting" refers to the act of informing the police or other relevant authorities of abnormal behavior or an emergency.

[1668] This invention relates to a security patrol system that uses drones to efficiently patrol high-crime areas, detect abnormal behavior, alert people, and notify the police in emergencies. This system is mainly composed of four elements: a server, a terminal, a drone, and a user, and each element works in close coordination with the others.

[1669] Hardware and software used

[1670] Server: A computer with high-performance data analysis capabilities and the processing power to operate generative AI models. Specifically, it is a server equipped with a large-capacity hard disk and a high-speed processor.

[1671] Device: A computer that receives and analyzes data in real time and runs the generative AI model. This device can be a laptop or a high-performance desktop computer.

[1672] Drone: An unmanned aerial vehicle equipped with cameras, audio sensors, and environmental sensors, with autopilot capabilities, a battery management system, and the ability to return automatically when it needs to be recharged.

[1673] Generative AI model: An artificial intelligence algorithm for detecting abnormal behavior or situations. For example, TensorFlow is used for visual analysis, and PyTorch is used for audio analysis.

[1674] Program processing

[1675] 1. The server collects data on the patrol area and generates a patrol schedule using a generative AI model. The generated schedule includes prompts such as "Patrol the parking lot at night and detect suspicious behavior."

[1676] 2. The server sends the generated patrol schedule to the drone and issues patrol instructions to the drone.

[1677] 3. The drone patrols the set area on autopilot, collecting video data from the camera, audio data from the audio sensor, and temperature and humidity data from the environmental sensors.

[1678] 4. The data collected by the drone is sent to the terminal in real time via the drone's communication module using Wi-Fi or 4G / 5G networks.

[1679] 5. The device quickly analyzes the data it receives using a generative AI model, inputs a prompt such as "Analyze the collected data and detect any abnormalities," and reports any abnormal behavior or abnormal situations to the server.

[1680] 6. The server receives the report and evaluates the type of abnormal behavior (e.g., voyeurism, violent behavior) and the urgency.

[1681] 7. The server issues warnings and advice to the drone, such as "play a warning message" or "turn on a warning light."

[1682] 8. The drone will play a specified voice message (e.g., "Misconduct in this area is prohibited. Please leave immediately.") and turn on a warning light.

[1683] 9. If the server determines that the abnormal behavior is of high urgency, it will automatically notify the police. The notification process will send the drone's current location, real-time video data, and detailed information about the abnormal behavior.

[1684] 10. After the drone has completed its designated patrol area, it will send all collected data to the server.

[1685] 11. The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior.

[1686] 12. The server generates a report summarizing the analysis results and provides it to the user, who then takes necessary measures and makes adjustments based on the report.

[1687] Specific examples of programs

[1688] 1. Detecting suspicious activity in parking lots at night:

[1689] The server sends the drone a schedule for patrolling the parking lot at night.

[1690] As the drone patrols the parking lot, it captures several people conducting transactions around a car.

[1691] The device analyzes the video data and determines that the transaction is suspicious.

[1692] The server analyzes the situation and issues instructions to the drone to warn it off.

[1693] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[1694] If necessary, the server will notify the police, who will rush to the scene.

[1695] 2. Detecting Suspected Fraudulent Calls:

[1696] The server sends the drone a patrol schedule for areas prone to fraud.

[1697] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1698] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1699] The server analyzes the situation and provides advice and instructions to the drone.

[1700] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[1701] If necessary, the server will notify the police, who will respond to the scene.

[1702] In this way, this invention enables efficient security patrols in high-crime areas, enabling highly accurate detection of abnormal behavior and rapid response, thereby improving the safety of the entire area and increasing the sense of security for residents.

[1703] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1704] Step 1:

[1705] The server collects data for the patrol area. Specifically, the server imports past data on crime hotspots and analyzes crime rates by time of day and day of the week. The input is crime occurrence data, and the output is a crime risk map for each area as a result of the analysis.

[1706] Step 2:

[1707] The server uses the generative AI model to generate a schedule for the patrol area. The server inputs a prompt statement, "Please generate a patrol schedule for a specific area," into the AI ​​model and obtains the patrol schedule generated by the AI. The inputs are a crime risk map and the prompt statement, and the output is the patrol schedule.

[1708] Step 3:

[1709] The server sends the generated patrol schedule to the drone. Specifically, it transfers a data packet containing the generated schedule to the drone via Wi-Fi or 4G / 5G networks. The input is the patrol schedule, and the output is a confirmation that the transmission to the drone has been completed.

[1710] Step 4:

[1711] The server issues a "start patrol" command to the drone. The drone starts patrolling the designated area in autopilot mode. The input is the start patrol command, and the output is the drone starting to patrol.

[1712] Step 5:

[1713] The drone collects data in real time using various sensors (camera, audio sensor, environmental sensor). The input is environmental data within the patrol area, and the output is the collected sensor data.

[1714] Step 6:

[1715] The data collected by the drone is sent to the terminal in real time. Specifically, the data is sent to the terminal via Wi-Fi or 4G / 5G networks using the drone's communication module. The input is various sensor data, and the output is confirmation that the data has been sent to the terminal.

[1716] Step 7:

[1717] The data received by the device is quickly analyzed using a generative AI model. The device inputs the prompt "Analyze the collected data and detect abnormalities" into the AI ​​model to detect abnormal behavior or situations. The input is various sensor data and the prompt, and the output is the detection result of abnormal behavior or situations.

[1718] Step 8:

[1719] The device reports abnormal behavior or abnormal conditions to the server. Specifically, it sends a data packet containing the abnormality detection result to the server. The input is the abnormality detection result, and the output is a confirmation that the report has been sent to the server.

[1720] Step 9:

[1721] The server analyzes the received abnormal behavior reports and evaluates their urgency. The input is the abnormal behavior report data, and the output is the urgency evaluation result of the abnormal behavior.

[1722] Step 10:

[1723] The server issues warnings and advice to the drone. For example, it sends instructions such as "play a warning message" or "turn on a warning light." The input is the warning instruction, and the output is confirmation that the instruction has been sent to the drone.

[1724] Step 11:

[1725] The drone plays a specified audio message and turns on a warning light. For example, it might play the message "Misconduct in this location is prohibited. Please leave immediately." The input is the warning instruction, and the output is the actual warning action.

[1726] Step 12:

[1727] If the server determines that the abnormal behavior is of high urgency, it automatically notifies the police. The server sends the drone's current location, real-time video data, and detailed information about the abnormal behavior to the police. The input is the trigger for the emergency call, and the output is confirmation that the call to the police has been completed.

[1728] Step 13:

[1729] After the drone finishes its patrol area, it sends all collected data to the server. The input is the data collected after the patrol, and the output is a confirmation that the data has been sent to the server.

[1730] Step 14:

[1731] The server analyzes the collected data and evaluates the frequency and trends of abnormal behavior. The input is the collected data, and the output is the analysis results.

[1732] Step 15:

[1733] The server generates a report summarizing the analysis results and provides it to the user. The user then takes necessary measures and makes adjustments based on the report. The input is the analysis results, and the output is the report.

[1734] (Application example 1)

[1735] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1736] There is a need for efficient patrols, detection of abnormal behavior, and rapid response in high-crime areas. However, current systems rely heavily on manual patrols and monitoring, which are costly and time-consuming and make it difficult to respond in real time. Furthermore, delayed response in emergencies can have fatal consequences. To solve this problem, a more efficient and rapid method is needed.

[1737] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1738] In this invention, the server includes means for patrolling high-crime areas by drones, means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, means for issuing warnings and advice when an abnormal situation is detected, means for notifying the police in an emergency, means for transmitting data collected from the drone to a smartphone in real time, means for delivering the real-time data to a user terminal in streaming format, and means for sending a push notification to the user terminal when abnormal behavior is detected. This enables efficient patrols, rapid detection of abnormal behavior, real-time information provision to users, and rapid response in emergencies.

[1739] A "drone" is an unmanned aerial vehicle that flies remotely or autonomously and is equipped with cameras and sensors to collect data.

[1740] A "high-crime area" is a specific area where crimes occur frequently based on past data and statistics.

[1741] "Abnormal behavior" refers to behavior that differs from normal behavior and is suspected of being criminal or fraudulent.

[1742] An "abnormal situation" is a situation that differs from the normal environment or circumstances and suggests that abnormal or illegal activity is occurring.

[1743] A "sensor" is a device that collects environmental information and converts it into an analog or digital signal.

[1744] "Generative AI" is an artificial intelligence technology that identifies and judges abnormal behavior and situations based on large amounts of data.

[1745] "Warning" refers to the act of giving a warning or advice to a target person.

[1746] "Advice" refers to the act of making suggestions or instructions to encourage appropriate behavior by a person.

[1747] An "emergency" is an extraordinary situation that requires immediate action.

[1748] "Police" refers to a public institution whose primary mission is to prevent crime and maintain public order.

[1749] "Reporting" refers to the act of reporting an incident or abnormal situation to the relevant authorities.

[1750] "Real-time" refers to a state in which data collection and processing occur immediately, without delay.

[1751] A "smartphone" is a mobile phone with computing power and internet connectivity.

[1752] "Streaming format" refers to a technology that transmits data continuously and receives and plays it back in real time.

[1753] A "user terminal" is a digital device that displays data and performs operations.

[1754] "Push notifications" refers to a technology that automatically sends messages from a web server to a client device.

[1755] Overall system configuration

[1756] This invention is a system consisting of a drone, a server, a terminal, and a user. Each element works together to efficiently patrol high-crime areas, detect abnormal behavior, issue warnings, and notify the police in emergencies.

[1757] 1. Drones

[1758] Drones are autonomous unmanned aerial vehicles equipped with cameras, audio sensors, and environmental sensors. They patrol crime-prone areas under instructions from a server and transmit data collected by various sensors to the server in real time. Furthermore, drones are equipped with generative AI, allowing them to analyze some of the data themselves.

[1759] The drone is equipped with a battery management system that allows it to automatically return to a charging station and recharge when the battery level is low.

[1760] 2. Server

[1761] The server is responsible for the chain of command for the entire system and data analysis. It sends patrol area schedules to the drones and receives and analyzes the data sent in real time. The server is equipped with advanced generative AI to detect and analyze abnormal behavior. If an abnormality is detected, the server sends a warning instruction to the drone and, if necessary, sends a push notification to the user's smartphone. It also has a function to automatically notify the police in an emergency. The server can provide real-time data to the user in streaming format. It also accumulates and analyzes the collected data and generates periodic reports.

[1762] 3. Terminal

[1763] The device is equipped with a generative AI that analyzes data sent from the drone in real time. If it detects abnormal behavior or an abnormal situation, it immediately reports the results to the server. The device also provides an interface for users to configure the system, check patrol schedules, and monitor abnormal behavior. It also has the function of sending a push notification to the user's device if abnormal behavior is detected.

[1764] 4. Users

[1765] Users include system administrators, police operators, and local residents. Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior. In an emergency, users can manually notify the police from their smartphones.

[1766] Natural language description of the process

[1767] The drone patrols designated high-crime areas and collects data using various sensors. The collected data is sent to a server in real time via 4G / 5G. On the server, a generating AI analyzes the data and detects abnormal behavior or situations. If an abnormality is detected, the server sends a warning command to the drone and a push notification to the user's smartphone. In the event of an emergency, the server automatically notifies the police. Users can view the real-time data in streaming format on their smartphone and can also make additional reports as needed.

[1768] Examples of specific examples and prompts

[1769] Specific examples

[1770] While a drone is patrolling a parking lot at night, the camera captures several people engaging in suspicious transactions. This video data is sent to a server in real time, and the generating AI determines that the activity is suspicious. The server immediately sends a warning command to the drone, which plays a message saying, "Fraudulent activity is prohibited. Please leave the area immediately." The server also sends a push notification to the user's smartphone, allowing them to review the video in real time. If necessary, the server automatically notifies the police.

[1771] Prompt Sentence Examples

[1772] "While a drone is patrolling a parking lot at night, several people appear to be conducting suspicious transactions. Based on this data, detect suspicious activity, send a warning command to the drone, and send a real-time notification to the user's smartphone. Also, describe the process for automatically notifying the police if necessary."

[1773] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1774] Step 1:

[1775] The server generates a patrol schedule for high-crime areas. It uses area data and time data as input and generates the schedule based on this. The generated schedule data is sent as output to the drone. This sets the drone to patrol the specified area at the specified time.

[1776] Step 2:

[1777] The drone begins patrolling high-crime areas based on the received schedule. It uses schedule data from the server as input and environmental data and video data obtained by sensors and cameras as output, sending them to the server in real time.

[1778] Step 3:

[1779] The server receives data sent from the drone in real time and analyzes it using generative AI. Video data, audio data, and environmental data are used as input, and data processing is performed to analyze and detect abnormal behavior and abnormal situations. The output is the detection result of abnormal behavior and abnormal situations.

[1780] Step 4:

[1781] If the server detects abnormal behavior or an abnormal situation, it sends a warning or advice to the drone. It uses the detection result data as input and sends warning instruction data to the drone as output. Specific actions include having the drone play an audio message such as, "Unauthorized activity is prohibited in this location. Please leave the area immediately."

[1782] Step 5:

[1783] If the server detects abnormal behavior or an abnormal situation, it sends a push notification to the user's smartphone. It uses the detection result data as input and sends a notification message to the user's device as output. Specifically, it sends a notification message to the user saying, "Emergency: Suspicious person has entered the premises!"

[1784] Step 6:

[1785] If the server detects an emergency, it automatically notifies the police. It uses the detection result data and location data as input, and sends a notification message to the police system as output. Specifically, it provides the police with the drone's current location and real-time video data.

[1786] Step 7:

[1787] After the drone completes its patrol, it sends all collected data to the server. It uses the environmental data, video data, and audio data collected during the patrol as input, and sends all data to the server as output.

[1788] Step 8:

[1789] The server stores the collected data and uses generation AI to analyze it. The collected data is used as input, and data processing is performed to extract the frequency and trends of abnormal behavior. The analysis results are obtained as output. Specific operations include generating statistical data on the location and frequency of abnormal behavior.

[1790] Step 9:

[1791] The server generates a report summarizing the analysis results and provides it to the user. It uses the analysis result data as input, processes the data, and formats it into a report format. It then sends the report data to the user's terminal as output.

[1792] Step 10:

[1793] The user receives the report and takes necessary measures and adjustments. The report data is used as input, and the output is used to adjust local safety measures and patrol schedules. Specific actions include setting up new patrol areas and strengthening crime prevention measures.

[1794] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1795] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. This system consists of five main components: a drone, a server, a terminal, a user, and an emotion engine. The detailed functions and linked operations of each component are described below.

[1796] System configuration

[1797] 1. Drones

[1798] The drone is equipped with a camera, audio sensors, environmental sensors, and generative AI.

[1799] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[1800] It detects abnormal behavior and situations and sends the data to the server in real time.

[1801] 2. Server

[1802] The server is responsible for controlling the entire system and analyzing data.

[1803] Generate a schedule for the patrol area and send it to the drone.

[1804] It receives data transmitted from the drone and analyzes it using generative AI to detect abnormal behavior and determine appropriate action.

[1805] In case of an emergency, it will notify the police and provide the drone's location and real-time data.

[1806] 3. Terminal

[1807] The device is equipped with generative AI that analyzes data sent from the drone in real time.

[1808] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[1809] 4. Users

[1810] Users include system administrators, police operators, local residents, etc.

[1811] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[1812] Police operators receive emergency calls and respond quickly to the scene.

[1813] 5. Emotion Engine

[1814] The emotion engine analyzes the user's video and audio data and has the ability to recognize emotions in real time.

[1815] The emotion engine adjusts the response of the drone and the entire system based on the emotions it recognizes.

[1816] Program processing overview

[1817] Preparing and conducting patrols

[1818] The server generates a patrol schedule based on data on the patrol area and sends it to the drone.

[1819] The drone automatically patrols a set route and collects data using various sensors.

[1820] The device analyzes the data in real time and detects abnormal behavior or situations.

[1821] Detecting and responding to abnormal behavior

[1822] If the device detects an abnormality, it transfers the data and analysis results to the server.

[1823] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[1824] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[1825] The emotion engine analyzes the user's emotions and adjusts the drone's response.

[1826] Emergency response

[1827] If the server determines that the emergency is high, it will automatically notify the police.

[1828] The server sends the drone's current location and real-time video data to the police.

[1829] The user (police operator) confirms the report and arranges for on-site response.

[1830] Data Collection and Reporting

[1831] After the drone has completed its designated patrol area, it sends all collected data to the server.

[1832] The server stores the collected data and analyzes long-term crime trends and problem areas.

[1833] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[1834] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[1835] Specific examples

[1836] Example 1: Detecting suspicious activity in a parking lot at night

[1837] The server sends the drone a schedule for patrolling the parking lot at night.

[1838] As the drone patrols the parking lot, it captures several people trading something around a car.

[1839] The device analyzes the video data and determines that the transaction is suspicious.

[1840] The server analyzes the situation and issues instructions to the drone to warn it off.

[1841] The drone approaches and plays a message saying, "Unauthorized activity is prohibited in this location. Please leave immediately."

[1842] At the same time, the emotion engine analyzes the suspicious person's reaction and determines whether additional action (e.g., immediate police reporting) is necessary.

[1843] Example 2: Detecting suspected fraudulent calls

[1844] The server sends the drone a patrol schedule for areas prone to fraud.

[1845] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1846] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1847] The server analyzes the situation and provides advice and instructions to the drone.

[1848] The drone approaches and plays the message, "This may be a scam. Please end this call and contact the police."

[1849] An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[1850] If necessary, the server will notify the police, who will respond to the scene.

[1851] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[1852] The processing flow will be explained below.

[1853] Step 1:

[1854] The server references a database of patrol areas and generates a patrol schedule for high-crime areas.

[1855] Step 2:

[1856] The server sends the generated patrol schedule to the drone.

[1857] Step 3:

[1858] The server checks the drone's remaining battery level, GPS signal, and the operating status of each sensor.

[1859] Step 4:

[1860] The server confirms that the drone is ready and sends a signal to activate the drone.

[1861] Step 5:

[1862] The drone will begin flying automatically, following the set patrol route.

[1863] Step 6:

[1864] The drone collects data about its surroundings using cameras, audio sensors, and environmental sensors.

[1865] Step 7:

[1866] The data collected by the drone is sent to the device in real time.

[1867] Step 8:

[1868] The device uses generated AI to analyze the received data and detect abnormal behavior or situations.

[1869] Step 9:

[1870] If the device detects an abnormality, it transfers the data and analysis results to the server.

[1871] Step 10:

[1872] The server determines the type and urgency of abnormal behavior and instructs the drone to take action based on that.

[1873] Step 11:

[1874] The drone follows instructions from the server and approaches suspicious people to warn them and give them advice.

[1875] Step 12:

[1876] The emotion engine analyzes the video and audio data of suspicious individuals and recognizes their emotions in real time.

[1877] Step 13:

[1878] Based on the emotions recognized by the emotion engine, the server further adjusts the drone's response.

[1879] Step 14:

[1880] Depending on the emotion the drone recognizes, for example, if tension is detected, it will play an appropriate additional message, such as "This behavior has been recorded. Please leave the area immediately."

[1881] Step 15:

[1882] The server will reassess the situation and notify the police if necessary.

[1883] Step 16:

[1884] The server sends the drone's current location and real-time video data to the police.

[1885] Step 17:

[1886] The user (police operator) confirms the report and arranges for on-site response.

[1887] Step 18:

[1888] After the drone completes its patrol, it sends all collected data to a server.

[1889] Step 19:

[1890] The server stores the collected data and analyzes long-term crime trends and problem areas.

[1891] Step 20:

[1892] The server compiles the data analysis results, generates a report, and sends it to the user (administrator or operator).

[1893] Step 21:

[1894] The user receives the report and takes any necessary additional measures or amends the patrol plan.

[1895] Example 2

[1896] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1897] Conventional security patrol systems do not adequately detect abnormal behavior in real time or respond immediately in high-crime areas. Furthermore, rather than simply detecting abnormalities, appropriate responses that take into account the user's emotions are required, but this is difficult to achieve with conventional systems. This has led to problems such as a decline in crime prevention capabilities and a lack of security for local residents.

[1898] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1899] In this invention, the server includes a means for patrolling high-crime areas by drone, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, and an emotion recognition engine means for analyzing user emotions in real time and adjusting the response of the entire system. This enables rapid detection of abnormal behavior, immediate response, and response that takes user emotions into consideration.

[1900] A "drone" is an unmanned aircraft that can be remotely controlled and is equipped with cameras and various sensors to patrol crime-prone areas.

[1901] "Generative AI" is an artificial intelligence system that uses machine learning algorithms to recognize patterns from input data and automatically detect abnormal behavior or situations.

[1902] A "sensor" is a device that detects physical changes and outputs them as data, and in this system this includes cameras, audio sensors, and environmental sensors.

[1903] A "patrol schedule" is a plan that defines the drone's flight routes and time periods in high-crime areas.

[1904] "Warning" is an action that warns or advises the person involved or those around them when abnormal behavior or an abnormal situation is detected.

[1905] "Advice" is an action that instructs the appropriate way to respond to a detected abnormal situation.

[1906] An "emergency call" is the act of immediately notifying the police or relevant agencies when abnormal behavior or an abnormal situation occurs, and encouraging a prompt response.

[1907] An "emotion recognition engine" is a system that analyzes a user's video and audio data and recognizes their emotional state in real time.

[1908] "User" refers to anyone who uses this patrol system, such as a system administrator, police operator, or local resident.

[1909] This invention relates to a security patrol system that uses drones to patrol high-crime areas, detect abnormal behavior, issue warnings, notify the police in emergencies, and recognize user emotions. The system is composed of the following main components: a drone, a server, a terminal, a user, and an emotion recognition engine.

[1910] System configuration

[1911] 1. Drones

[1912] Drones are equipped with cameras, audio sensors, environmental sensors, and generative AI. A typical drone is a general-purpose unmanned aerial vehicle (e.g., a general-purpose unmanned flying device) equipped with a camera and various sensors.

[1913] The drone receives a patrol schedule from the server and automatically patrols designated high-crime areas.

[1914] Various sensors detect abnormal behavior and situations and send the data to a server in real time.

[1915] 2. Server

[1916] The server is responsible for controlling the entire system and analyzing data. A high-performance server machine (e.g., high-performance computer equipment) is used.

[1917] Generate a schedule for the patrol area and send it to the drone.

[1918] The data transmitted from the drone is analyzed and generative AI is used to detect abnormal behavior and determine appropriate actions.

[1919] In case of an emergency, the system will notify the police and provide the drone's location and real-time data.

[1920] 3. Terminal

[1921] The terminal is equipped with generative AI and analyzes data transmitted in real time from the drone, using high-performance AI computing platforms such as NVIDIA Jetson.

[1922] The system reports the results of abnormal behavior detection to the server and receives instructions from the server as necessary.

[1923] 4. Users

[1924] Users include system administrators, police operators, local residents, and other users of the system.

[1925] Users operate the server and terminals to configure the system, check patrol schedules, and monitor abnormal behavior.

[1926] Police operators receive emergency calls and respond quickly to the scene.

[1927] 5. Emotion Recognition Engine

[1928] The emotion recognition engine analyzes the user's video and audio data and has the ability to recognize emotions in real time. Emotion recognition software such as Affectiva SDK is used.

[1929] The drone and the entire system can then adjust their response based on the emotions they recognize.

[1930] System Operation

[1931] The server generates an optimal patrol schedule based on data on the patrol area and sends it to the drone. The drone automatically patrols a set route and collects data using various sensors. The collected data is sent to the terminal in real time and analyzed by the generating AI. If the terminal detects abnormal behavior, it reports the data and analysis results to the server. The server determines the type and urgency of the abnormal behavior and instructs the drone to take appropriate action. In the event of an emergency, the server automatically notifies the police, and police operators respond promptly to the scene.

[1932] Specific examples

[1933] Example 1: Detecting suspicious activity in a parking lot at night

[1934] The server sends the drone a schedule for patrolling the parking lot at night.

[1935] As the drone patrols the parking lot, it captures several people trading something around a car.

[1936] The device analyzes the video data and determines that the transaction is suspicious.

[1937] The server analyzes the situation and issues instructions to the drone to warn it off.

[1938] As the drone approaches, it plays a message saying, "Unauthorized activity is prohibited. Please leave the area immediately." The emotion engine analyzes the suspicious person's reaction and considers additional measures if necessary.

[1939] Example 2: Detecting suspected fraudulent calls

[1940] The server sends the drone a patrol schedule for areas prone to fraud.

[1941] As the drone patrols, it uses a voice sensor to detect people making phone calls on the street.

[1942] The device analyzes the voice data and determines that the conversation is suspected of being fraudulent.

[1943] The server analyzes the situation and provides advice and instructions to the drone.

[1944] The drone approaches and plays a message: "This may be a scam. Please end this call and contact the police." An emotion engine analyzes the caller's reaction and adjusts the drone's behavior as needed.

[1945] Prompt Sentence Examples

[1946] "What should you do if a drone detects unauthorized activity in a parking lot at night?"

[1947] "Please tell me how the device and server work together when the drone detects a conversation that is suspected to be fraudulent."

[1948] This invention automates security patrols in high-crime areas, detecting and responding to abnormal behavior in real time with high efficiency, and also enables responses that take user emotions into consideration, thereby improving the safety of the entire area and increasing the sense of security for residents.

[1949] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1950] Step 1:

[1951] The server generates an optimal tour schedule based on the data of the tour area.

[1952] Input: Geographic data of high-crime areas and historical crime records

[1953] Data processing: Generative AI models are used to analyze geographic data and crime records to optimize patrol routes and times.

[1954] Output: Tour schedule

[1955] Specific operation: The server retrieves crime data from the database for the past year, analyzes it using a generative AI model (e.g., a general-purpose AI platform), generates an optimal patrol schedule, and transmits it to the drone.

[1956] Step 2:

[1957] The drone receives the patrol schedule and automatically patrols the set route.

[1958] Input: Tour schedule sent from the server

[1959] Data calculation: Uses GPS data and internal map data to navigate routes

[1960] Output: Route execution status and collected data

[1961] How it works: The drone travels to a designated starting point, flies a pre-defined route based on GPS and internal map data, and collects data from its surroundings using cameras and audio sensors.

[1962] Step 3:

[1963] The device receives data transmitted from the drone in real time and analyzes it using a generative AI model.

[1964] Input: Video data, audio data, and environmental data from drones

[1965] Data calculation: Analyzes video data and audio data to identify abnormal behavior

[1966] Output: Detection results and analysis data

[1967] Specific operation: The device receives a data stream from the drone in real time. The video data is analyzed using a TensorFlow model to identify specific behavioral patterns (e.g., a person running away). The audio data is analyzed using a voice recognition system (e.g., a general-purpose voice analysis system) to detect abnormal conversations.

[1968] Step 4:

[1969] If the device detects abnormal behavior, it sends the data and analysis results to the server.

[1970] Input: Analysis results and anomaly detection data

[1971] Data calculation: Determining the type and urgency of abnormal behavior

[1972] Output: Notifications and detailed data to send to the server

[1973] Specific operation: When the device identifies abnormal behavior (e.g., suspicious transaction activity), it sends a notification to the server with detailed data attached.

[1974] Step 5:

[1975] The server determines the type and urgency of abnormal behavior and sends appropriate action instructions to the drone.

[1976] Input: Analysis results and anomaly detection data from the device

[1977] Data calculations: assessing urgency and determining appropriate actions

[1978] Output: Instructions to the drone

[1979] Specific operation: The server receives the analysis data and determines the urgency of the abnormal behavior as "high." It then sends a "warning" command to the drone, playing a message saying, "That behavior is illegal. Please stop immediately."

[1980] Step 6:

[1981] An emotion recognition engine analyzes the user's emotions and adjusts the system's overall response.

[1982] Input: Video and audio data from a drone or device

[1983] Data Computing: Recognizing Emotional States Using Emotion Analysis Algorithms

[1984] Output: System-wide response adjustment instructions

[1985] Specific operation: The emotion recognition engine analyzes video and audio data to determine the reaction of suspicious individuals, and adjusts the drone's and the system's overall response as needed.

[1986] Step 7:

[1987] If the server determines that an emergency situation exists, it will automatically notify the police.

[1988] Input: Emergency data from drones and devices

[1989] Data calculation: generating and sending message content

[1990] Output: Police report details and real-time data

[1991] Specific operation: The server determines the abnormal behavior as "highly urgent" and initiates a police notification system. The notification includes the drone's current location, detailed emergency data, and a real-time video link.

[1992] Step 8:

[1993] After the drone completes its patrol area, it sends all collected data to the server.

[1994] Input: Drone-collected data

[1995] Data calculation: storing in a database and analyzing

[1996] Output: Long-term data analysis results and reports

[1997] Specific operation: When the drone completes its patrol, it transmits the collected data to a server via Wi-Fi or 4G / 5G. The server stores the data in a database and performs trend analysis using AI. A report is generated and sent to the system administrator.

[1998] (Application example 2)

[1999] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2000] Improving safety in crime-prone areas requires efficient, real-time patrols and the detection of abnormal behavior. However, conventional systems are slow to detect and respond to abnormal behavior, making it difficult to quickly detect and deal with suspicious individuals and criminals. They also lack the ability to respond flexibly to on-site situations or take measures that take user emotions into consideration. This makes it difficult to ensure the safety and security of local residents.

[2001] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2002] In this invention, the server includes a means for patrolling high-crime areas using drones, a means for installing sensors and generating AI for detecting abnormal behavior and abnormal situations, a means for issuing warnings and advice when an abnormal situation is detected, a means for notifying the police in an emergency, an emotion recognition means for analyzing the user's emotions, and a means for adjusting responses based on the emotion recognition results. This enables abnormal behavior to be detected in real time, enabling prompt and appropriate responses. Furthermore, flexible measures that take the user's emotions into account can increase the safety of the entire area and the sense of security of residents.

[2003] A "drone" is an unmanned aerial vehicle equipped with sensors and generative AI to patrol high-crime areas and detect abnormal behavior or situations.

[2004] A "sensor" is a device installed on a drone to detect changes in the environment or abnormal behavior, and is a device that collects data such as audio, video, and temperature.

[2005] "Generative AI" is artificial intelligence that analyzes collected data and detects abnormal behavior and situations in real time.

[2006] "Emotion recognition means" is a technology that analyzes the user's video and audio data and recognizes emotions in real time.

[2007] "Response adjustment means based on emotion recognition results" is a technology that adjusts the response of the drone and the entire system based on the emotional data analyzed by the emotion recognition means.

[2008] A "patrol schedule" is a server-generated plan of routes and time settings for drones to efficiently patrol high-crime areas.

[2009] "Real-time analysis means" is a technology that analyzes data collected from drones in real time and quickly detects abnormal behavior.

[2010] "Warning and advisory measures" are technologies that allow drones to issue warning messages to targets when abnormal behavior is detected.

[2011] "Emergency reporting means" is a technology that allows drones or systems to automatically report to the police when abnormal behavior is deemed to be an emergency.

[2012] As an embodiment of the present invention, the following system is constructed. Details of the system and the processing contents of each means will be explained using specific examples.

[2013] Overall system configuration

[2014] The system of the present invention is mainly composed of the following elements.

[2015] 1. Drones: They patrol high-crime areas and are equipped with cameras, audio sensors, environmental sensors, and generative AI.

[2016] 2. Server: Controls the entire system and is responsible for data analysis, generating drone patrol schedules, and emergency notification functions.

[2017] 3. Terminal: Analyzes data from the drone in real time, detects abnormal behavior, and issues instructions for response.

[2018] 4. Users: Includes system administrators, police operators, local residents, etc.

[2019] 5. Emotion recognition: Analyze the user's emotions and adjust the response.

[2020] Drone patrols and data collection

[2021] The drones patrol high-crime areas according to a pre-set patrol schedule, collecting data using various sensors (cameras, audio sensors, environmental sensors) and analyzing it in real time using generative AI.

[2022] Example: While a drone is patrolling a parking lot at night, the camera captures several people transacting something around a car, which is detected as anomalous behavior.

[2023] Data analysis and processing by the server

[2024] The server receives the data sent from the drone and uses the AI ​​to analyze any abnormal behavior. If a serious abnormality is detected, an emergency call is made.

[2025] Hardware / software used: high-performance computers (e.g., servers with NVIDIA GPUs), generative AI models (e.g., TensorFlow).

[2026] Example: A server analyzes video data captured by a drone and determines that a transaction is suspicious. It then sends a warning to the drone saying, "Fraudulent activity is prohibited here."

[2027] Emotion recognition and response adjustment

[2028] The emotion recognition means analyzes the user's voice and video data to recognize their emotions, and adjusts the drone's response based on the emotion recognition results.

[2029] Example: When a drone issues a warning, if the suspicious person responds aggressively, it will keep its distance, but if the person responds non-aggressively, it will continue to pay more attention.

[2030] Example prompt: "I'm patrolling a high-crime area and I've noticed some unusual behavior. What are my next steps?"

[2031] User operation

[2032] Users (system administrators and police operators) operate the server or terminals to configure the system, check patrol schedules, monitor abnormal behavior, and respond quickly to emergency calls.

[2033] This system can monitor the safety of high-crime areas in real time, detect and respond to abnormal behavior immediately, and take user emotions into consideration, enabling more appropriate security management.

[2034] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2035] Step 1:

[2036] The server generates the drone's patrol schedule. The inputs are geographic information of high-crime areas and past crime data. Using this data, the optimal patrol route and time is determined and sent to the drone. The output is the patrol schedule sent to the drone. Specifically, the server's AI algorithm retrieves past crime data from the database and compares it with map information to determine the patrol route.

[2037] Step 2:

[2038] The drone patrols high-crime areas based on a patrol schedule received from a server. The input is the patrol schedule from the server, and the output is collected real-time video, audio, and environmental data. Specifically, the drone automatically flies a set route, and the onboard cameras and sensors continuously record the surrounding situation.

[2039] Step 3:

[2040] The device receives real-time data transmitted from the drone and analyzes it using a generative AI model. The input is video, audio, and environmental data from the drone, and the output is the detection result of abnormal behavior. Specifically, the device's generative AI model analyzes the received data and applies an algorithm to detect abnormal behavior or situations.

[2041] Step 4:

[2042] The server receives the anomaly analysis results sent from the device and determines the type of anomaly and its level of urgency. The input is the anomaly detection result from the device, and the output is instructions for the drone to take action and, if necessary, a report to the police. Specifically, the server instructs the drone to take appropriate action (such as issuing a warning or sending a warning message) based on the analysis results, and if the level of urgency is high, it automatically reports the anomaly to the police.

[2043] Step 5:

[2044] The drone issues warnings and advice on-site based on instructions from the server. The input is an action instruction from the server, and the output is the playback of a voice message or a specific action (for example, a change in flight pattern). Specific actions include the drone playing a specified message over a speaker and warning suspicious individuals as necessary.

[2045] Step 6:

[2046] The emotion recognition means analyzes the subject's reactions and adjusts the drone's response. The input is video and audio data from the drone's camera and microphone, and the output is the emotion recognition results and instructions for the drone to act. Specifically, the AI ​​model analyzes the video and audio data to recognize the subject's emotions. If an aggressive reaction is observed, the drone will maintain its distance, and if the reaction is non-aggressive, it will issue further warnings.

[2047] Step 7:

[2048] The server accumulates the collected data, analyzes long-term crime trends, and generates reports. The input is all data transmitted by the drones, and the output is analysis results and reports. Specifically, the server stores the collected data in a database, periodically analyzes the data, and generates reports that identify crime trends and problem areas.

[2049] Step 8:

[2050] The user (system administrator or police operator) receives the report and modifies the patrol plan or takes additional measures as necessary. The input is the report from the server, and the output is the modified patrol plan or additional measures. In concrete terms, the user checks the contents of the report and sets up new patrol routes or additional crime prevention measures.

[2051] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2052] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2053] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2054] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2055] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2056] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2057] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2058] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2059] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2060] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2061] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2062] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2063] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2064] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2065] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2066] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2067] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2068] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2069] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2070] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2071] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2072] The following is further disclosed regarding the above embodiment.

[2073] (Claim 1)

[2074] Drones can be used to patrol high-crime areas,

[2075] A means of incorporating sensors and generative AI to detect abnormal behavior and situations;

[2076] A means of issuing warnings and advice when an abnormal situation is detected;

[2077] How to notify the police in an emergency;

[2078] A system including:

[2079] (Claim 2)

[2080] 10. The system of claim 1, further comprising means for generating and transmitting a patrol schedule for the drone to the drone.

[2081] (Claim 3)

[2082] 10. The system of claim 1, further comprising: a generative AI means for analyzing data collected from the drone in real time.

[2083] (Claim 4)

[2084] 10. The system of claim 1, further comprising means for the server to evaluate the situation and instruct the drone on appropriate action if abnormal behavior is detected.

[2085] (Claim 5)

[2086] 10. The system of claim 1, further comprising means for transmitting drone location information and real-time data to police upon an emergency call.

[2087] (Claim 6)

[2088] 10. The system of claim 1, further comprising means for transmitting all collected data to a server for analysis and reporting after the tour is completed.

[2089] "Example 1"

[2090] (Claim 1)

[2091] A means for generating a patrol schedule for high-crime areas based on the patrol area data and transmitting the schedule to the drone;

[2092] Drones can be used to patrol high-crime areas,

[2093] Using various sensors and generating AI, drones can detect abnormal behavior and situations in real time and transmit that...

Claims

1. Drones can be used to patrol high-crime areas, A means of incorporating sensors and generative AI to detect abnormal behavior and situations; A means of issuing warnings and advice when an abnormal situation is detected; How to notify the police in an emergency; A system including:

2. The system of claim 1 , further comprising means for generating and transmitting a patrol schedule for the drone to the drone.

3. 10. The system of claim 1, further comprising: a generative AI means for analyzing data collected from the drone in real time.

4. The system of claim 1 , further comprising means for the server to evaluate the situation and instruct the drone on appropriate action if abnormal behavior is detected.

5. 10. The system of claim 1, further comprising means for transmitting drone location information and real-time data to police during an emergency call.

6. 10. The system of claim 1, further comprising means for transmitting all collected data to a server for analysis and reporting after the tour is completed.

Images