Procedure for monitoring an area

Abstract

The invention relates to a method (100) for monitoring an area (40), in particular an area, building or part of a building, comprising the steps: • Determining (110) a plurality of sensor data of different modalities and / or from different locations within the monitored area (40), wherein the sensor data characterize physical measurements of the monitored area or parts of the monitored area, • Providing (120) the majority of the sensor data as input to a machine learning system (5), • Classifying (130) by the machine learning system (5) whether, based on the sensor data in the monitored area, a dangerous situation, in particular fire, water and / or burglary, exists, or that no dangerous situation exists.

Description

[0001] The invention relates to a method for monitoring an area, an intelligent monitoring system, a computer program and a machine-readable memory. State of the art

[0002] There are surveillance systems, colloquially known as alarm systems, that trigger without a dangerous situation. In particular, they trigger a burglar alarm without a burglar and a fire alarm without a fire. False triggering has negative consequences, such as a loss of confidence in the reliability of the surveillance system or the costs of dispatching personnel, especially firefighters, police, or employees, to inspect the system. Disclosure of the invention

[0003] The invention relates to a method for monitoring an area, in particular a territory, a building, or a sub-area. The sub-area can in particular comprise a part of a building or a part of a territory. A building can be a residential building, a commercial building, or an industrial building. An area can also comprise an area with multiple buildings.

[0004] The procedure includes the steps listed below.

[0005] In one process step, a plurality of sensor data from different modalities and / or different locations within the monitored area is collected. The combination of sensor data from different modalities enables comprehensive monitoring.

[0006] Sensor data with different modalities refers to different types and properties of sensor data. The sensor data can include, in particular, the time of day, weather data, airborne and structure-borne sound, air pressure, humidity, temperature, and the presence of people, but also sensor data from motion sensors, heating sensors, video sensors, lidar sensors, ultrasound sensors, radar sensors, heat sensors, smoke sensors, heat sensors, fire alarms of any kind, water sensors (for detecting burst pipes), and energy consumers in the home, e.g., coffee machines, stoves, heaters, televisions, lights / lamps, radios, computers, and microphones. The sensor data acquired here includes, in particular, current, voltage, magnetic field sensors, and interference in local radio systems, particularly Wi-Fi, Zigbee, Z-Wave, and Bluetooth. The sensor data primarily includes physical measurements that characterize the monitored area or parts thereof.

[0007] Characterization refers to the fact that the sensor data describes or represents the monitored area or parts of the monitored area using physical measurements. The sensor data serves to provide information about the state or properties of the monitored area.

[0008] A further method step is providing the majority of the sensor data as input to a machine learning system. The machine learning system, in particular, has an interface or interacts with such an interface. The interface allows the sensor data to be received and / or retrieved. The interface can be used to receive or retrieve the sensor data, in particular from one or more sensors or devices with one or more sensors. The received sensor data can, in particular, be provided to the machine learning system.

[0009] In a further process step, the machine learning system performs classification. Based on the sensor data, the machine learning system determines whether a hazardous situation, particularly fire, water, and / or burglary, exists in the monitored area, or whether no hazardous situation exists. Preferably, the machine learning system can analyze and classify whether a hazardous situation such as fire, water, or burglary exists based on the sensor data.

[0010] Advantageously, false alarms can be prevented in particular by the method according to the invention.

[0011] The measures listed in the subclaims result in advantageous further developments and improvements of the method specified in the main claim.

[0012] An advantageous further development of the process comprises the following steps: In one process step, a target value is received. The target value is used to classify whether a hazardous situation exists or not.

[0013] Examples of hazardous situations include burglary, fire (especially smoke, preferably gases), and water. It is advantageous if the target value corresponds to a predefined classification. In particular, a target value is defined for sensor data, a plurality of sensor data, or a combination of sensor data.

[0014] According to an advantageous further training, the target value is set by a human.

[0015] The next step in the process involves supervised training of the machine learning system. The machine learning system is trained based on the classification determined by the machine learning system and the received target value. Specifically, the machine learning system is trained based on the target value and the classification. This improves its future classification ability.

[0016] According to an advantageous development, known configurations, in particular of a smart home system and / or building control system, are advantageously used. Configurations include, for example, a spatial allocation or grouping of sensors and / or the topological arrangement of the sensors and / or system-specific sensory measurement ranges. The configurations are preferably predefined or result from the sensor data. The configurations can also be used in classification or training.

[0017] An advantageous further development is that the method step of receiving a target value comprises providing a message depending on the result of the classification, in particular in the form of a video stream, an audio stream and / or a notification.

[0018] In particular, a video stream, an audio stream, or a notification containing, for example, a recorded video or audio is provided to a person, in particular shown, preferably played. Based on the message, the person decides whether a dangerous situation exists and, if so, what the dangerous situation is. The decision corresponds to the target value.

[0019] An advantageous further development is that the machine learning system includes a neural network that performs the classification.

[0020] In particular, the neural network can be trained using the feedback target value.

[0021] A particularly advantageous development is that the target value is automatically specified and provided after a defined time, and / or a defined number of classifications, and / or based on a defined confidence value, and / or based on a defined event. In particular, such a defined event can be a reduction in the frequency of provided messages.

[0022] An advantageous further development is that if the classification classifies a dangerous situation, in particular a fire (smoke), water, and / or burglary, at least one action is performed. An action can, in particular, be alerting for assistance.

[0023] The invention also relates to an intelligent monitoring system configured and configured to carry out the method. In particular, the intelligent monitoring system comprises a building control system or a smart home system. In particular, the monitoring system has a central unit, in particular a smart home central unit, which is configured and configured to carry out a plurality of the method steps.

[0024] According to an advantageous further development, the central unit is designed and configured to communicate with a remote processing unit, in particular a server cloud or a server, via WiFi, Ethernet, LAN, WAN, in particular to carry out calculations there.

[0025] Furthermore, the invention relates to a computer program which is set up and / or designed to carry out all steps of the method.

[0026] Furthermore, the invention relates to a machine-readable storage medium on which the computer program is stored.

[0027] Embodiments are shown in the figures and explained in more detail in the following description.

[0028] They show: Fig. 1 an example of a monitored area designed as a building, Fig. 2 a method according to the invention.

[0029] Fig. Figure 1 shows, as an example, a building 40, in particular a single-family home, for an area 40. The building 40 can also be configured as an area, an apartment building, a commercial building, an industrial building, an apartment, etc.

[0030] The exemplary building 40 has several building sections, especially rooms. Fig. 1, a building section 41 is shown as an example room 41. To simplify the illustration, the exterior walls and windows of building section 41 are not shown.

[0031] In the area 40, several devices 20 are formed, for example, which can in particular be part of a smart home system or a building control system.

[0032] A smart home system is a system designed to improve the quality of living, security, and efficient energy use. A smart home system is preferably based on at least two or more devices, particularly those that are interconnected and preferably controllable. A smart home system can also consist of just a single controllable device.

[0033] Preferably, the smart home devices can communicate with each other via a central unit and / or directly.

[0034] When it comes to industrial and / or commercial applications, people usually don't talk about smart home systems, but rather about Industry 4.0, a building control system, or an automation system. The goal of such a commercial and / or industrial system is to use the area, building, and / or part of a building, more efficiently through the use of sensors, actuators, and communication tools, particularly to increase security, energy efficiency, and efficiency through automation.

[0035] The industrial / commercial building control system and the smart home system therefore usually differ only in scaling. The goals of a smart home system or a building control system are therefore very similar. Often, the devices used are the same or are used in both systems. Bus systems such as KNX are used, for example, in smart home systems in private buildings, but also in commercial and / or industrial buildings.

[0036] Devices 20 are understood to be apparatuses that are designed and configured to control facilities in an area 40 and / or to acquire sensor data and / or to display information and / or to interact with the user. The devices 20 particularly increase efficiency, safety, comfort, and / or usability. The devices particularly fulfill one of the following functions: heating, air conditioning, cooling, ventilating, switching, controlling, or regulating. Examples of devices 20 are light sources, blinds, heaters, refrigerators, washing machines, ovens, but also control elements for building control, for example for controlling HVAC systems, white goods appliances, multimedia systems, etc., but also temperature controllers, in particular room temperature controllers, room thermostats, wall thermostats, control units, or switching elements for switching lights, electricity in general, HVAC systems, etc.

[0037] Furthermore, modern smart home systems or building control systems can be controlled remotely, for example by a mobile input device, in particular a smartphone, tablet or mobile phone, or from a server, in particular a cloud, or via a server, in particular via a cloud.

[0038] The devices 20 are networked with each other and can receive and / or transmit messages, in particular signals, data packets, information, control commands and / or environmental variables, etc. The devices 20 have an interface, in particular a communication interface. Communication is wired, in particular via a bus system, Powerline, M-Bus, Ethernet, EIB, CAN, KNX, EMS, 1-Wire, DALI, DMX, OpenTherm, or wireless, preferably Bluetooth, NFC, RFID, ANT+, Dash7, GPRS, UMTS, 5G, LTE, LoRaWAN, WIMAX, Zigbee, Z-Wave, Matter, Thread, 868 MHz or WIFI, or optical, in particular via pulse-modulated light. For this purpose, the interface has the hardware components required for communication using a previously mentioned technology or protocol, such as processing means, amplifiers, antennas, as well as the required software.

[0039] An example is Fig. 1 shows a motion sensor 22. A motion sensor 22 is also understood to be a presence detector. The motion sensor 22 detects infrared radiation, electromagnetic waves, or ultrasound as a physical quantity. A motion sensor 22 detects the environment by detecting movements based on changes in infrared radiation, electromagnetic waves, or ultrasound. If a person, animal, and / or object moves, the motion sensor 22 provides corresponding sensor data.

[0040] A human-machine interface (HMI) 24 is also shown. The HMI serves, in particular, as an operating unit, particularly as an HVAC operating unit. It enables the operation and / or control of heating, ventilation, and air conditioning systems or individual or multiple components thereof.

[0041] According to further training, the HMI 24 is designed to collect and provide sensor data.

[0042] An HMI 24 can also be designed as a device operating unit 24, which serves to operate and control one or more devices 20, in particular white goods devices and / or multimedia devices and / or smart home devices, or building control devices, preferably monitoring, control and / or regulating devices.

[0043] For example, a smart light switch also represents a device 20.

[0044] Furthermore, a thermostatic valve 26 on a radiator is shown as an example. A thermostatic valve 26 is a device 20 used to control, in particular, regulate, the temperature in an area. The thermostatic valve 26 can also be used to control underfloor heating and / or infrared heating and / or an air conditioning system and / or a ventilation system.

[0045] Like other devices 20, the thermostatic valve 26 can be controlled via an HMI or other control methods and allows automatic adjustment of the room temperature based on preset settings or sensor data.

[0046] Furthermore, door / window sensors 28 are formed by way of example. A door and / or window sensor 28 is a device 20 used to monitor the status of doors and windows. It typically consists of two parts: a magnetic sensor and a magnet. The magnetic sensor is attached to the door or window frame, while the magnet is attached to the movable part of the door or window. When the door or window is opened, the magnet moves away from the magnetic sensor and triggers an alarm or notification to inform the user of the change in status. According to one development, the door / window sensor has a rotation rate and / or motion sensor and / or acceleration sensor.

[0047] Door and window sensors can be integrated into smart home systems or building control systems in particular to enable security functions such as intrusion detection or the automation of lighting and heating.

[0048] The devices 20 provide a variety of sensor data.

[0049] Sensor data is information collected by various sensors in the monitored area, in particular in building 40. These sensors or devices 20 with sensors can monitor various aspects of the area, such as temperature, humidity, light intensity, movement, noise, air quality, and much more. The sensors collect sensor data, in particular regarding these parameters, continuously, constantly, or intermittently, in particular event-driven, and can send it to connected devices 20 or make it available for retrieval.

[0050] The sensor data can be used to trigger various actions. For example, based on the sensor data, the heating or air conditioning can be controlled to optimize the room temperature. It can also adjust the lighting to regulate the amount of light entering the room. Furthermore, sensor data can also be used for security purposes, such as detecting intrusions or triggering smoke and carbon monoxide detectors. The sensor data is crucial for automation and control, as it allows you to monitor the condition of the home and respond accordingly to improve comfort, energy efficiency, and security.

[0051] Preferably, the sensor data has different modalities and / or is collected from different locations. For example, smoke is detected at the ceiling, whereas the door opening is detected at the door, the window opening at the window, etc.

[0052] The sensor data characterize physical measurements of the monitored area.

[0053] In Fig. 2 shows a method 100 according to the invention. The method 100 comprises a plurality of method steps.

[0054] In method step 110, a plurality of sensor data of different modalities and / or different locations within the monitored area are determined.

[0055] Multimodal sensor data refers to different types and properties of sensor data. Each sensor can provide specific information about a particular property or phenomenon. Here are some examples of sensor data from different modalities: • Temperature sensor: Detects the temperature in the room or at a specific location. • Motion sensor / presence sensor: Detects movement or changes in a specific area. • Light sensor: Measures the light intensity or brightness in a room. • Air quality sensor: Monitors air quality by measuring parameters such as humidity, CO2 levels or VOCs (volatile organic compounds). • Noise sensor: Detects noise or sound levels in the environment. • Humidity sensor: Measures the moisture content in the air or in a material. • Pressure sensor: Measures the pressure, for example in a piping system or in a tire. • Accelerometer: Detects acceleration or movement in a specific direction. • GPS sensor: Tracks the geographical position or movement of an object.

[0056] In a further method step 120, the majority of the sensor data is provided as input to a machine learning system 5. The machine learning system 5 is part of an intelligent monitoring system 1. The machine learning system 5 has, in particular, a neural network.

[0057] In a further method step 130, the sensor data is classified by the machine learning system 5. Based on the sensor data, it is classified whether a hazardous situation exists in the monitored area or not. Examples of hazardous situations include fire, water, and / or burglaries. But also smoke or an excessively high or too low concentration of one or more gases.

[0058] In an optional method step 140, a message is provided. Providing here means generating the message. Furthermore, providing means sending the generated message or making it available for retrieval. For example, if there is a high probability of a dangerous situation, or if the current situation cannot be classified by the system, or if it is classified as a particularly unknown situation, a message can optionally be provided.

[0059] The delivery can, in particular, take the form of a video stream, an audio stream, and / or a notification. The notification can include text describing the situation or a recorded video or audio file. The message is, in particular, delivered to a human. The human can then determine a target value based on the message, in particular, perform a classification.

[0060] In an optional method step 150, a target value is received. The target value classifies the presence of a hazardous situation. The target value corresponds, in particular, to a predefined classification.

[0061] The target value can be set, in particular, by a human. The target value can also be set by a machine learning system and / or artificial intelligence. Providing a target value, i.e., reacting to a message, can be automated, in particular, after a defined number of classifications, a predefined confidence value, or a defined event. A defined event can, for example, be that a defined number of determinations have been made by one or more humans.

[0062] According to an advantageous further development, a pre-classification can be performed, which is then confirmed or rejected by the human. The human confirms or rejects the pre-classification. In particular, in the case of rejection, the human specifies a classification in the form of a target value. The machine learning system can thus preferably be actively supported and / or accelerated during training.

[0063] An optional method step 115 is executed after method step 110. In the optional method step 115, the known sensor data is clustered, in particular grouped. Clustering results in a basic model. Based on this basic model, the machine learning system then classifies the acquired sensor data.

[0064] For example, a smoke signal from a fire detector when the window is closed may indicate a fire in the building. Furthermore, an opening window when no prior motion or presence detection within the building indicates a burglary.

[0065] In particular, the known sensor data and known sensor data combinations are added to clusters. New or unclassifiable sensor data are assigned to a target value in process steps 140 to 160. The target value classifies whether a hazardous situation exists or not. Supervised training of the machine learning system 5 is performed. In particular, the clusters and / or the classification are optimized in this way.

[0066] Preferably, method step 115 is no longer carried out once the basic model has been created.

[0067] Method steps 140 and 150 can also be carried out after method step 120 and / or replace method step 130.

[0068] In an optional step 160, an action is executed depending on the classification of one or more sensor data or a sensor data combination. The action is executed in particular if a potential hazard has been detected.

[0069] Such an action can be: • A notification to the security service or the police. • Activation of acoustic alarm signals in and around the house, in particular the activation of a siren 29. • Automatic closing or opening of relevant escape doors and windows • A recording of footage by surveillance cameras 27 for later use in investigations. • Informing selected emergency contacts of the user. • A status update in the home control app on a communication device, which makes the user's whereabouts clear in the event of an alarm.

Claims

[1] Method (100) for monitoring an area (40), in particular an area, building or sub-area, comprising the steps: • Determining (110) a plurality of sensor data of different modalities and / or from different locations within the monitored area (40), wherein the sensor data characterize physical measurements of the monitored area or parts of the monitored area, • Providing (120) the majority of the sensor data as input to a machine learning system (5), • Classifying (130) by the machine learning system (5) whether, based on the sensor data in the monitored area, a dangerous situation, in particular fire, water and / or burglary, exists, or that no dangerous situation exists. [2] Method (100) according to claim 1, characterized by the steps: • Receiving (150) a target value, wherein the target value is determined in particular by humans, and wherein the target value classifies the presence of a dangerous situation or no dangerous situation, and wherein the target value corresponds to a predetermined classification, • Supervised training (160) of the machine learning system (5) depending on the classification determined by the machine learning system and the received target value. [3] Method (100) according to claim 2, characterized by that receiving (150) a target value, providing (140) a message depending on the result of the classification, in particular in the form of a video stream, an audio stream and / or a notification, wherein the message is transmitted to a person who determines the target value based on the message. [4] Method (100) according to the preceding claim, characterized bythat the machine learning system (5) comprises a neural network which performs the classification. [5] Method (100) according to one of the preceding claims, characterized by that the target value is automatically specified and provided after a defined time, and / or a defined number of classifications and / or due to a defined event. [6] Method (100) according to one of the preceding claims, characterized by that if the classification classifies a dangerous situation, in particular fire, water and / or burglary, at least one action is carried out. [7] Intelligent monitoring system (1) which is arranged and designed to carry out the method (100) according to one of the preceding claims. [8] Computer program which is arranged to carry out all steps of the method (100) of one of claims 1 to 6. [9] A machine-readable storage medium on which the computer program according to claim 8 is stored.

Images