SUBJECT-SELECTION AUTONOMOUS AIRCRAFT-BASED SECURITY MONITORING SYSTEM FOR INDOOR SPACES

Abstract

The invention relates to an AI-powered system for selectively tracking a protected target by processing image, depth, location, and preferably audio data from an autonomous aerial vehicle operating indoors where GPS signals are unavailable, limiting tracking behavior towards non-target individuals, and detecting identified security anomalies.

Description

SECURITY IN INDOOR SPACES WITH SUBJECT-SELECTION AUTONOMOUS AIRCRAFT TRACKING SYSTEM Technical Field to Which the Invention Relates The invention is an aerial device that moves autonomously indoors where GPS signals are unavailable. by processing image, depth, location and preferably audio data through the medium selectively targeting a specific individual to be protected, targeting individuals outside the target group. restricting surveillance behavior and identifying security anomalies It is related to an artificial intelligence-powered system for this purpose. State of the Art Fixed-position cameras are commonly used for monitoring child or person safety inside the home. Imaging devices are used. Fixed cameras, from a specific field of view They can capture images continuously or on an event basis; cameras with a movable head are the same. It can be directed horizontally and vertically from the installation point. However, a single fixed installation point, furniture, walls, doors and different room geometries Because of this, it cannot cover the entire living area simultaneously; different areas can be monitored. Multiple camera setups may be required. Increasing the number of cameras, the amount of image data to be processed and stored and the system infrastructure requirements It can increase. Indoor floor robots capable of movement, environmental mapping, and autonomous pathfinding. It offers different solutions in terms of ground conditions. However, this type of movement depends on the ground. The observation point and freedom of movement of the platforms are largely based on a two-dimensional plane. It is limited. Areas such as tables, countertops, window frames, or areas at different heights. monitoring areas such as these from varying viewing angles, platforms working on the ground It can be limited depending on its physical location. Target tracking functions are also known in autonomous or semi-autonomous aerial vehicles. These types of systems track the target's location when a person or marker is detected. The aircraft can update its movement according to the target. However, in general... the roles of individuals within the frame in the target tracking approach within the context of usage differentiation and the direct application of this difference to follow behavior in each use 1 This is not provided in the scenario. In a home environment, in addition to a person to be protected, a caregiver is required. The presence of other individuals, such as family members or visitors, requires constant mobile monitoring. This leads to unnecessary monitoring and an undesirable form of work in terms of privacy. It can open. Known approaches to protecting image privacy involve showing people's faces. blurring, masking specific image regions, or accessing the recording Methods such as limiting their powers can be used. These approaches although it is intended for subsequent processing or storage of the image, the decision to physically track a mobile robotic platform in the context of people's use Changing roles based on performance is a separate control problem, especially in a closed environment. In a moving aircraft, the perception result and the flight motion are interconnected. Because of this, the privacy decision also affects navigation and flight control. It becomes an input. Another technical difficulty in indoor aircraft is when the GPS signal is weak or The reason is that it is not usable. Therefore, the environment can be monitored with visual or distance-based sensors. mapping, identifying obstacles and mapping the aircraft's own position on the map It needs to be calculated accordingly. When visual tracking is temporarily interrupted... The tracking cycle may be interrupted until the target is found again. Similarly, the aircraft's limited battery capacity ensures safety based on continuous flight. limiting operating time in applications and ensuring safety during charging. This creates the risk of its functions becoming completely disabled. For these reasons, a platform that can move in three dimensions in an enclosed space is only Instead of monitoring the environment or random individuals, it selectively tracks the target to be protected. physical surveillance of non-target individuals when they are alone. It can stop its behavior; it can perform safe navigation without GPS, and preferably... Visual, auditory, and signal-based different data sources depending on the working condition. There is a need for an integrated security monitoring architecture that it can utilize. Brief Description and Objectives of the Invention The overall purpose of the invention is to provide autonomous aerial navigation in enclosed environments where GPS signals are unavailable. In security monitoring carried out with the vehicle, the usage context of the detected individuals active targets to be protected by using their roles as a technical classification input. 2 The same system uses tracking and non-tracking behavior towards non-target individuals. to carry out within. One aim of the invention is to identify individuals detected in image data by their target class to be protected. to separate individuals based on the class of persons whose privacy needs to be protected and the result of this classification The goal is to directly convert this into an air traffic control decision. This way, the target will be protected. When located, the aircraft moves according to the target position, the target to be protected If no one is present and only a person in the privacy category is present, then the target The pursuit is halted. Another purpose of the invention is to enable autonomous operation in enclosed spaces without the need for GPS data. The purpose is to provide positioning and navigation. Preferably with image and depth data. By evaluating the LiDAR data together, the two and three-dimensional geometric shapes of the environment are revealed. extracting its characteristics, determining the aircraft's position within this environment. and safe movement commands taking into account obstacles and virtual boundaries. The aim is to create. Another purpose of the invention is to enable image-based target tracking over physical obstacles or selective barriers. In cases where privacy is interrupted due to work status, low energy related to the target. Using the received Bluetooth signal strength, the approximate location of the target can be determined. to determine and monitor target search behavior until visual tracking is re-established It is to support. Another purpose of the invention is to extend the aircraft's limited battery life to include safety functions. The goal is to manage without completely cutting off access. The target is to keep the system in a sleeping, immobile, or secure state. Upon determination of this, the aircraft will return to the charging station; during charging switched from image-based flight tracking to acoustic observation mode based on microphone data. and resume active flight in the event of a detected acoustic anomaly. that is intended. Another purpose of the invention is to provide information regarding the physical state of the target to be protected and the environmental conditions. The goal is to evaluate different types of anomalies within a common decision architecture. In this context... the target falling or remaining motionless for a long time, crying or screaming-like sounds by detecting environmental events such as smoke, loss of visibility or flame formation at a distance. The aim is to send an alarm to the user. 3 Another purpose of the invention is to enable the autonomous aerial vehicle to physically reach the target to be protected. The goal is to prevent it from approaching dangerously close. A three-dimensional security perimeter around the target. the definition of the zone and the aircraft's retreat or altitude relative to this zone by performing a changeover movement, it ensures a safe connection between the mobile platform and the target. The aim is to maintain distance. Detailed Description of the Invention The invention enables the autonomous aerial vehicle to monitor a target that needs to be protected in an enclosed environment. a system that combines image processing and flight control within the same decision cycle. It is a security monitoring system. The system consists of at least one autonomous aerial vehicle capable of moving in three axes. The vehicle has at least one image sensing unit that captures images of the environment, and it monitors the movement of the aircraft. a flight control unit that manages and processes the sensing data and sends it to the flight control unit. The aircraft fuselage contains at least one processing unit that provides decision commands. familiar flight components such as propulsion motors, propellers, and the power source necessary for flight It may include the following components. It is a preferred choice for use in environments where children are present. In practice, the propellers provide direct contact between the user and the rotating elements. in order to reduce it, within a protective cage or environmental protection structure It can be positioned. The processing unit performs at least part of the image processing and decision-making operations on the aircraft. It is structured in such a way as to perform this operation on it. This structure ensures that each of the image data... low latency without needing to send the decision to a remote server It allows for the establishment of a control loop. In an application, the processing unit is general. It is a single-board computer for this purpose and has a separate artificial intelligence accelerator for the processing unit. It is connected. The artificial intelligence accelerator, object detection and classification model. It can take over at least a portion of the computational load from the processor. The image sensing unit is capable of providing depth information as well as a picture of the surrounding environment. It can be in stereo or RGB-D camera form. The system also includes the surrounding area. LiDAR sensor for distance-based identification of obstacles and room boundaries. It is available for use. The system will obtain depth data, distance data, or both from the environment. It may include at least one sensing unit structured in this way. The sensing unit in question... The unit includes a stereo or RGB-D camera providing depth data, and a camera providing distance data. 4 In the form of a LiDAR sensor or a sensor structure using them in combination. It is possible. Image data and depth and / or distance data are perceived as the same. It can be obtained from one unit or from separate sensing units. In the preferred approach, two-dimensional boundaries of the environment are determined using LiDAR data. Three-dimensional visual feature points and depth information obtained from a stereo camera. An environmental model is being created. This data is visualized by the positioning and mapping module. simultaneous positioning and mapping of the aircraft's GPS signal It calculates its own position in an environment where it is not present and provides an up-to-date environmental map. It is used for movement on it. Within the system, image detection Image, depth, or distance obtained from the unit and preferably from a LiDAR sensor. by processing the data using sensor fusion method, a two or three-dimensional map of the environment is created. at least one that extracts and calculates the real-time position of the aircraft on this map The module includes a positioning and mapping module. This positioning and... The mapping module can be in the form of a standalone processing unit, or it can be a direct processing unit. also in the form of a software / functional algorithm block running on the unit It can be designed. Aircraft reconnaissance during the system's initial use or when a terrain map is unavailable. It can be operated in exploration mode. In exploration mode, image, depth and distance data are displayed. The collected materials are used to create a three-dimensional map of the house or indoor space. This map room boundaries, passageways, fixed obstacles and user-defined features The defined virtual boundaries can be saved. The user can choose areas such as the kitchen, window sill, and pool. The surrounding area, the fireplace area, or other areas deemed risky according to the regulations are prohibited. or it can be defined as an area requiring attention. The definition of a restricted area is: This can be directly related to map coordinates, as well as semantic elements in the image data. It can also be associated with domain classes determined by segmentation. During normal operation, image frames received from the image sensing unit are processed. It is analyzed in terms of person or object identification within the unit. A basic principle in an application. Object detection model, transfer learning to differentiate between child and adult classes They are being retrained. The face is being examined for more detailed verification of the target's identity. recognition can be used or perceived from the image, such as the person's body proportions. Derivative biometric features can be considered as classification input. A lightweight deep learning-based object detection model in proof-of-concept application and While image processing libraries can be used, the scope of the request is limited to a specific artificial intelligence. It is not dependent on the neural network architecture or software library. As a result of person identification, the processing unit will assign a protected target class or for each detected person. It assigns a privacy class. The image shows the person belonging to the target class to be protected. In that case, the image coordinates, depth information, or of the person in question The target location is calculated using its position on the environmental map. Processing unit flight by evaluating the difference between the target location and the aircraft's current location It generates a movement command to the control unit to track the target. Flight control This unit converts this command into engine control signals for the aircraft, thereby controlling the aircraft. It guides the target in a three-dimensional environment according to its movement. According to the selective tracking logic of the invention, if a person belonging to the target class to be protected is not detected and The processing unit is active only when the image contains a person belonging to the privacy category. It does not generate a target tracking command. Instead, it sends the aircraft's target tracking command to the flight control unit. to maintain its current position, move to a safe waiting position, or command to terminate the tracking movement according to the specified work status. This is transmitted. Thus, when the adult or caregiver is alone in the room, the air... The device does not track the individual via mobile. In a preferred application. The system does not record video; it only sends a live stream to the remote user's device. It is configured to provide image transmission. The target to be protected Privacy is protected by not initiating image transmission if the image is not present. Transferring images of people in the class to a remote user device This prevents the remote user from viewing the protected target. If a command is given, the aircraft will protect its target autonomously. searching and transmitting real-time images upon target detection. It is being carried out. The target to be protected and the person in the privacy category together. If the target is visible in the image, tracking of the target to be protected continues. To prevent the aircraft from getting too close to the target during tracking, the target A three-dimensional security zone can be defined around it. A preferred option. In practice, this safety zone is a virtual zone centered on the target and with a radius of at least 1.5 meters. It is defined as a sphere. The target approaches the aircraft or air If the vehicle's planned movement violates this security zone, the processing unit 6 changing its normal follow-up route and instructing the flight control unit to move away from the target. or gives a command to increase altitude. Thus, physical target tracking is achieved through selective target tracking. Approach safety is managed concurrently within the same navigation cycle. The target temporarily disappears from the image, moves to another room, or the image goes blank. If the line is interrupted by an obstacle, the system enters a redundant target finding mode. It can pass through. For this purpose, periodically low levels are carried on the target to be protected. a wearable BLE wristband or BLE that broadcasts powered Bluetooth (BLE) advertising packets A transmitter unit in the form of a label can be used. These BLE signals are transmitted autonomously. The process is received by a BLE receiver module integrated onto the aircraft. The unit measures the received signal strength of the signal received from the transmitting unit on the target to be protected. RSSI (Reverse Stability Indicator) value and fixed points positioned in different regions of the indoor environment. By jointly evaluating the reference IDs received from BLE beacon units search area containing the target to be protected, on a room or zone basis. It can restrict access. In a preferred application, this applies to different areas of an enclosed space. Three fixed BLE beacon units are used. The processing unit, word According to the assessment results, the subject is the aircraft's placement in the designated area or room. to ensure orientation and re-establish visual contact with the target to be protected. The system then switches to image-based target tracking mode. The system detects not only the location but also the activity status of the target to be protected. It can evaluate movement or posture information obtained from the image. using whether the target is active / moving or passive / sleeping and in a safe state. It can be determined whether the target is sleeping or in a safe / motionless state. If provided, the aircraft can reduce the energy consumption and noise of continuous flight. It is autonomously directed to the charging station. This decision is based on battery level. A state-sensitive energy system that can be evaluated independently or in conjunction with battery level. It can be implemented as a management function. Instead of completely deactivating the system while the aircraft is at the charging station, an acoustic system is activated. It can switch to observation mode. This is possible either on the aircraft or while the system is in charging mode. Audio data received by at least one microphone on site is analyzed in the processing unit. This is done in one application, such as mel-frequency cepstrum coefficients are extracted from the audio signal. Frequency domain features are extracted and a deep learning-based audio classification model is used. with acoustic events such as crying, screaming, high-intensity impact, or glass breaking in the background 7 It deviates from the plan sound. These acoustic phenomena are secondary, confirming the visual anomaly. as a trigger or as a standalone safety trigger during charging It is available for use. When an acoustic anomaly is detected during charging, the processing unit signals the charging process. It can make an emergency takeoff decision without waiting for the completion of aircraft charging. It leaves the station and is directed to the area associated with the anomaly, and the image A verification or target acquisition process based on data acquisition is initiated. In this working method... The system's energy management involves switching between the active flight phase and the passive charging and resting phase. It creates a security loop that does this. The posture and movement history of the target to be protected are considered within the scope of visual anomaly detection. can be evaluated. In an application, the image limiter surrounding the target. The sudden change in the aspect ratio of the box is used to determine the probability of it falling. It is used. In addition, the target must remain motionless for a certain period of time or Entering the predefined restricted area coordinates is considered an anomaly. These anomaly classes are based solely on camera imagery. It can be evaluated on its own, or in conjunction with acoustic data. The system sends alarms or notifications generated by the processing unit to the remote user. It may contain at least one communication unit that enables the transmission of information to the device. The communication unit transmits data to a remote user device over a wireless communication network. It can be arranged in a way that facilitates communication. Within the scope of environmental hazard assessment, the image shows visibility loss due to smoke density. Features such as smoke-like patterns or flame flashes are considered hazardous. It can be classified. Visual acuity is significantly impaired due to smoke. If a decrease in intensity or the detection of smoke / flames occurs, the system identifies the target in the image. Instead of viewing its absence as a simple "missing target," it should be considered a fire or It can switch to emergency protocol. In this protocol, the relevant situation is marked as a critical alarm. is being marked and sent via the communication unit to the remote parent or user device. An alarm message is being sent. Alarm transmission in an application is done via MQTT messaging. It is carried out via a protocol, however the invention is a specific communication It is not limited to the protocol. 8 The aircraft's own operational status can also be taken into account in the safety decision. Air the vehicle unexpectedly falling, sensor or control connection being interrupted or the system deviating from its predicted flight status, a simple communication interruption. Instead, it can be classified as a critical device anomaly. In this case, the user is informed. An alarm indicating that the aircraft is disabled and a physical inspection may be required. It is being transmitted. In a concrete operational scenario, the system initially operates in three phases in home environment exploration mode. It is mapped in three dimensions. The user can map areas such as the kitchen and window surrounds. It defines it as a risky area. Image processing when the child and caregiver are in the same room. The model categorizes both individuals, with the child identified as the target for protection, and The aircraft tracks the child between rooms according to the child's movements. When the kitchen enters the defined risky zone, the system generates a positional anomaly and It sends an alarm to the user when the caregiver is then left alone in the room. Because the child is not in the image, the aircraft does not follow the caregiver, and this is the preferred method. It also stops the image transfer function in the application. In the same scenario, when the child falls asleep, the activity analysis results indicate that the aircraft... It returns to the charging station and switches to acoustic monitoring mode. During charging... If a crying or loud crashing sound is detected, the system will make an emergency takeoff decision. The process shifts back to visual observation. The visual aspect changes as the child moves into another room. If the connection is lost, a low-power Bluetooth signal strength will be used to approximately room temperature. This room is being used to re-establish aircraft visual detection. It is being directed. If the child approaches the aircraft, the virtual safety perimeter around the target is protected. The violation of the area is determined and the aircraft retreats or increases altitude. It is performing a maneuver. Thanks to the described structure, the system reduces the field of view resulting from fixed camera placement. while reducing its limitations with three-dimensional mobility, the mobile platform integrates with every element present in the environment. Instead of automatically tracking a person, it uses selective physical tracking based on the classification result. It employs tracking behavior. Visual positioning and mapping are used in flight. autonomous movement in GPS-neutral environments combined with control; visual tracking Switching between low-energy Bluetooth signal strengths enables target re-finding; active flight The transition between acoustic monitoring during charging and battery limitation ensures safety despite battery limitations. It ensures that its functions are maintained through different sensor modes. Physical, acoustic and 9 Evaluation of environmental anomalies in a common processing architecture and dynamic security. Applying the navigation decision to the region also concerns target safety and platform safety. It integrates them within the same system.

Claims

1. A target to be protected in a closed environment without the need for GPS data. It is a security monitoring system designed for autonomous monitoring, and its feature is; - at least one autonomous aerial vehicle capable of moving in three axes in an enclosed space vehicle, - at least one image sensing unit that receives image data from the environment, - Obtain depth data, distance data, or both from the environment. at least one sensing unit configured to do so, - at least one flight control unit that controls the movement of the aircraft, - using at least one of the depth and distance data along with image data. creating an environmental map and the aircraft's use of that environment at least one positioning that determines its location on the map and mapping module and - individuals detected in image data received from the image sensing unit in terms of the target class to be protected and the class of individuals whose privacy will be respected When a person belonging to the target class to be protected is identified by classification, the statement is made. The subject of the tracking command is a person belonging to the target class to be protected. a person was identified who was not identified and who belongs to the category of person whose privacy must be protected. If this happens, the tracking action can be stopped or the current location can be determined. at least one process that generates a protection command and transmits it to the flight control unit. unit It includes.

2. It is a security monitoring system according to Claim 1, and its feature is that it uses LiDAR data. defining the two-dimensional boundaries of the environment and obtained from a stereo or RGB-D camera. Using the acquired visual feature and depth data, a three-dimensional environmental map can be created. creates and determines the position of the aircraft on the environmental map in question. visual simultaneous positioning and mapping using sensor fusion. It includes a positioning and mapping module that performs this function. 11 3. A security monitoring system compliant with Claim 1, featuring image processing and person identification. with the processing unit that performs classification operations on the aircraft at least a portion of the image processing load from that processing unit. It includes an AI accelerator that takes over and executes the process.

4. It is a security monitoring system compliant with Claim 1, and its feature is; detection in image data. an image trained to separate the people being viewed into child and adult categories classification model, facial recognition process or biometric data derived from an image privacy of the target class to be protected by using at least one of the rates It includes a processing unit that identifies the class of person to be monitored.

5. A security monitoring system compliant with Claim 1, characterized by its ability to determine the class of target to be protected. no person to whom this information was obtained and who belongs to the category of person whose privacy must be protected if detected, image transfer to the remote user device. The obstacle is that it contains a processing unit.

6. A security monitoring system compliant with Claim 1, whose feature is; on the environmental map or Identifying at least one predefined restricted area in the image data and The situation where the target to be protected enters the prohibited area is considered an anomaly. It contains a processing unit that identifies [this].

7. It is a security monitoring system in accordance with Claim 6, and its feature is; to identify the restricted area. semantics applied to coordinates or image data on a map The processing unit that defines the region class determined as a result of segmentation. It includes.

8. Security monitoring system according to Claim 1, its feature is; image of the target to be protected. from the data, it determines the activity status and whether the target to be protected is dormant or not. air upon determination that it is in a safe state of immobility It includes a processing unit that enables the vehicle to be directed to a charging station.

9. It is a security monitoring system according to Claim 8, and its feature is; aircraft charging. at least one microphone that picks up audio data while it is at the station and the said 12 It identifies a predefined acoustic anomaly by processing microphone data. and upon identification of the acoustic anomaly in question, the aircraft's charging process An emergency takeoff command that allows the vehicle to leave the charging station before completion. It contains a processing unit that constitutes it.

10. It is a security monitoring system according to Claim 1, and its feature is; on the target to be protected. at least one transmitter unit that is carried and emits a low-energy Bluetooth signal, At least one BLE receiver module is positioned on the autonomous aerial vehicle, enclosed. Multiple fixed BLE beacons positioned in different regions of the environment The BLE receiver in question will be affected if the unit and image-based target tracking are interrupted. the signal strength of the transmitter unit of the target to be protected, received by the module by evaluating the value and reference IDs of the fixed BLE beacon units the area or room where the target to be protected is located and the aircraft's statement The subject matter is that it includes a processing unit that enables navigation to the area or room.

11. It is a security monitoring system according to Claim 1, and its feature is that it receives audio data from the environment. a small microphone and the aspect ratio of the limiting box of the target to be protected in the image the microphone in question determines the probability of falling based on the change in the ratio Detecting crying or screaming events based on frequency characteristics extracted from the data. It contains a processing unit.

12. It is a security monitoring system compliant with Claim 1, and its feature is; smoke in the image data. visibility loss, smoke patterns, or flame flashes depending on the density identifying as an environmental hazard and determining the environmental hazard in question a process that creates an emergency alarm different from a loss of target situation. It includes the unit.

13. According to claim 12, it is a security monitoring system, the characteristic of which is; by the processing unit. The created emergency alarm is transmitted remotely via a messaging protocol. It contains a communication unit that transmits information to the user's device.

14. A security monitoring system in accordance with Claim 1, whose characteristic is; the target to be protected. defining a three-dimensional virtual safety zone around the aircraft entering the security zone in question or the aircraft of the target to be protected 13 If it approaches the target, the flight control unit will be notified of departure from the target or altitude. It contains a processing unit that transmits increment commands.

15. A security monitoring system compliant with Claim 14, characterized by its target-centered approach to protection. and a three-dimensional virtual security system in the shape of a sphere with a radius of at least 1.5 meters. It contains a processing unit that defines the region.

16. It is a safety monitoring system compliant with Claim 1, and its feature is; aircraft crash or interruption of the control or communication link of the aircraft is critical the processing unit that identifies the device anomaly and the critical device in question It includes a communication unit that transmits an anomaly alarm to a remote user. 14