Time-of-flight based 3d monitoring system with flexible monitoring zone definition

Abstract

The invention relates to a time-of-flight based 3D surveillance system with flexible surveillance zone defining functionality. The invention relates to a surveillance system that detects and / or characterizes movements within or around a monitored infrastructure, such as a building or facility that is monitored to detect intruders. An improved trade-off between rigorous area surveillance and the number of false alarms is provided by improved control of the 3D surveillance device. Input functionality is provided to a user to define a 3D sub-zone within a 3D environment model. A change functionality allows the user to generate a redefined sub-zone by dragging one of the corner points of the 3D sub-zone to a different location within the 3D visualization of the 3D environment model, whereby the shape of the 3D sub-zone is deformed. The input and change functionality serve to provide spatial parameters associated with the redefined sub-zone to the 3D surveillance device and to cause the 3D surveillance device to generate an action in case a movement within the redefined sub-zone is detected by means of 3D measurement data.

Description

Technical Field

[0001] The present invention relates to a surveillance system for detecting and / or characterizing movement within or around a monitored infrastructure (e.g., a building or facility to be monitored for intruder detection).

[0002] As an example, the surveillance system according to the invention is used to monitor neural points within a city, such as train stations, airports, city parks, or other busy public places. Furthermore, the surveillance system is used to monitor or protect restricted or hazardous areas, such as industrial plants, construction sites, commercial complexes, and private residences.

[0003] As another example, the monitoring system according to the invention is used to support the operation of a facility, such as the monitoring of a warehouse or parking lot. Background Technology

[0004] Starting with initial passive observation (i.e., merely providing a remote representation of a scene to be monitored by a human system operator), modern surveillance systems have increasingly evolved into proactive systems that can autonomously identify objects and / or situations related to the scene to be monitored, such as automatically issuing alarms or marking scenes for human operators to view.

[0005] Typically, a distributed network of cameras is used, where live video streams from multiple cameras are displayed on a set of monitors, and / or where different camera views can be selected by the operator of the surveillance system (e.g., security personnel at a train station or commercial building complex). Furthermore, modern systems can be configured to automatically identify and track people or other moving objects (e.g., vehicles) to notify the operator of any suspicious movement.

[0006] Specifically, when a moving object leaves the field of view of the first camera, the automatically generated tracking information is used to switch the moving object from one camera to another; that is, the object's path can be automatically tracked within the distributed network of cameras.

[0007] In addition to displaying essentially unedited footage and path information from the cameras, the surveillance results are typically embedded as real-time textures in an integrated 3D model of the environment, allowing the situation to be examined from any viewpoint (e.g., independent of the movement of people or objects).

[0008] Modern surveillance systems often utilize a variety of different surveillance sensors. For example, thermal imaging cameras can be used to monitor infrastructure at night and / or to identify objects or critical events based on temperature, such as detecting fires or the status of vehicle or other machine engines.

[0009] In another example, lidar (light detection and ranging) devices and / or optical barriers provide intrusion and / or movement detection, where moving objects are detected as they pass through an observation plane within the infrastructure. However, compared to camera imaging, the spatial coverage of lidar devices and / or optical barriers is typically limited, for example, where different observation planes are placed only at entrances (e.g., doors and windows), or where different observation planes are at least several meters apart.

[0010] A specific problem with existing surveillance systems involves handling changing environmental conditions (such as changes in ambient light) and / or adapting to modifications of the environment, where the placement of permitted objects must be distinguished from the placement of prohibited objects within the infrastructure to be monitored.

[0011] These requirements have led to a new category of surveillance systems using 3D coordinate measuring units (3D measurement units) that implement techniques and principles known from the field of surveying for precise three-dimensional geometric measurement of the environment (e.g., principles used by high-end laser scanning stations such as the Leica RTC360 or Leica BLK360). As an example, the Leica BLK247 is an exemplary example of this new category of intelligent 3D surveillance systems. These new 3D surveillance systems use 3D coordinate measuring units configured to provide 3D coordinate measurement data based on the time-of-flight principle in an all-round view. For example, the 3D coordinate measuring unit provides a field of view of 360 degrees around a first axis and at least 130 degrees around a second axis perpendicular to the first axis, wherein the 3D coordinate measurement data is generated at a point acquisition rate of at least 150,000 points per second. For example, the Leica BLK247 provides a point acquisition rate of 200,000 points per second in a field of view of 360 degrees around the first axis and 270 degrees around the second axis.

[0012] Because it utilizes the time-of-flight principle, it provides the detection and object tracking of any 3D changes under any lighting conditions (e.g., in bright backlight and darkness). Radiation wavelengths within the infrared range can be used for time-of-flight measurements, allowing the use of materials opaque to the visual wavelength range to cover the 3D coordinate measurement unit, thus hiding details about the 3D surveillance technology used from the human observer. Therefore, it can provide discreet and noise-free 3D surveillance that is invisible to the eye.

[0013] Furthermore, the use of time-of-flight-based 3D coordinate measurement units and 3D-based object classification increases the sensitivity and spatial resolution of object detection and recognition. As an example, spatial resolution and object recognition can be tuned in principle to ensure that objects with dimensions in the centimeter range or even smaller are precisely detected and recognized.

[0014] The full field of view and the data acquisition method (e.g., where 3D coordinate measurement data is inherently generated relative to a common coordinate system) provide a direct location reference for events detected by surveillance data (3D coordinate measurement data) within the environment. For example, a 3D surveillance device can be used to generate a digital model of the environment and monitor events within that digital model by utilizing the same 3D coordinate measurement unit. This allows for the flexible definition and later adjustment of specific surveillance areas (sub-regions of the full field of view) or multiple different sub-regions monitored by the same 3D surveillance device without having to move the 3D surveillance device.

[0015] Typically, an environment comprises monitored areas with different security levels (e.g., different access authorizations), where potential "holes" in the security network that allow intruders to enter undetected can occur, particularly in the transition areas between different monitored areas. Therefore, it is often beneficial to limit the transitions between areas to the smallest possible size, for example, by arranging different areas as close to each other as possible.

[0016] The increased sensitivity and spatial resolution have brought new challenges. Even small, slow movements can be detected and may trigger false alarms. For example, a complex of buildings to be monitored may include or be adjacent to green areas, such as forests, farmland, or areas with ornamental plants. The improved sensitivity and spatial resolution can now even trigger alarms due to the detection of vegetation moving into a protected area due to wind. Even the growth of the plants themselves can lead to increased alarms. For instance, the monitoring area is typically located close to the ground, for example, to prevent someone from crouching “under” the monitoring area (between the ground and the monitoring area). However, in the case of green areas, the ground may “grow into” the monitoring area, thus triggering false alarms, for example, if plants are moved by the wind. As another example, when monitoring boundary walls, birds may frequently land on and move around on top of the wall, which could trigger an alarm if a potential intruder is assumed to be climbing over the wall.

[0017] Therefore, a certain tolerance zone between the protected area and its surrounding environment is used to prevent frequent false alarms. However, this reduces the tightness of the shielding. Summary of the Invention

[0018] The purpose of this invention is to provide improved environmental monitoring that overcomes the shortcomings of existing technologies.

[0019] One specific objective is to provide a surveillance system that allows for an optimal trade-off between monitoring in a tightly controlled area and the number of false alarms caused, for example, by intrusion into transitional areas between different surveillance zones.

[0020] Another objective is to provide a monitoring system that reduces, for example, incorrect monitoring of specific areas due to improper operation by inexperienced users.

[0021] These objectives are achieved by implementing at least some of the characterizing features of the following aspects. Features that further develop the invention in alternative or advantageous ways are described in other aspects.

[0022] The present invention relates to a method for controlling a 3D monitoring device, the 3D monitoring device being configured to generate 3D measurement data of the spatial volume of an environment and to use the 3D measurement data and a 3D environment model to monitor the spatial volume of the environment.

[0023] As an example, a 3D monitoring device includes a 3D coordinate measurement unit configured to capture the spatial volume of an environment using 3D coordinate measurement data based on the time-of-flight principle. To detect objects within the spatial volume of the monitored environment, the 3D monitoring device may further include an object detector configured to detect objects within a defined sub-region of the monitored environment based on the 3D coordinate measurement data of the 3D monitoring device (e.g., based on 3D change detection).

[0024] For example, a 3D monitoring device is implemented to be placed in a corner of a room and is intended to monitor the spatial volume within a 90-degree by 90-degree field of view (to generate 3D coordinate measurement data with a 90-degree by 90-degree field of view). If the 3D monitoring device is intended to be mounted on a wall, it can be implemented to provide monitoring of a 90-degree by 180-degree field of view. In another implementation, for example, where the 3D monitoring device is intended to be mounted on the ceiling, the 3D coordinate measurement unit provides a 360-degree by 90-degree field of view. As an example, the 3D coordinate measurement unit is configured to provide a field of view of 360 degrees about a first axis (e.g., a vertical axis) and at least 130 degrees (e.g., 270 degrees about the second axis) about a second axis perpendicular to the first axis.

[0025] Specifically, the 3D monitoring device is configured to provide the generation and updating of 3D point clouds at a rate of at least one 3D point cloud sampling point per second for each angular field of view, with an elevation angle of half a degree and an azimuth angle of one degree. For example, the 3D monitoring device is configured to generate 3D coordinate measurement data at a point acquisition rate of at least 150,000 points per second.

[0026] Therefore, the entire field of view can be monitored virtually in real time by 3D coordinate measuring equipment, and a virtually real-time representation of the monitored infrastructure can be displayed to the user based on the 3D measurement data. Specifically, rapid updates of the 3D point cloud provide improved change detection based on the 3D measurement data.

[0027] The method includes the following steps: reading input data associated with a 3D monitoring device, wherein the input data provides coordinate information associated with the position of the 3D monitoring device within a 3D environment model. For example, the 3D environment model is provided in the form of a point cloud or as a vector-file-model (e.g., a computer-aided design (CAD) model), wherein the position of the 3D monitoring device within the 3D environment model is obtained using the coordinate information provided along with the input data. Alternatively or additionally, 3D measurement data from the 3D monitoring device is used to generate a 3D measurement model, wherein the position of the 3D monitoring device (e.g., essentially) is known / provided by the 3D measurement data.

[0028] The input data is then used to generate a graphical user interface (GUI) on an electronic graphic display. This GUI provides a 3D visualization of the 3D environment model, including the location of the 3D monitoring device and, for example, an indication of the 3D monitoring device's field of view. Thus, the user of the 3D monitoring device can examine the environment accessible by the device and possible monitoring areas. In a further step, input functionality is provided via the GUI (e.g., by means of touchscreen control), where the user can define 3D sub-regions within the 3D environment model. For example, sub-regions can be expected to be specifically monitored and / or treated differently relative to the rest of the environment, for instance, defining minimum and / or maximum sizes of alarm objects (and objects issuing alarms), minimum and / or maximum speeds of alarm objects, or specific movement patterns to be compared with the movement of detected objects. Furthermore, sub-regions can be assigned different point densities or sensitivities and / or different repetition frequencies to refresh the point cloud portion represented by the sub-region.

[0029] For example, an electronic graphic display is implemented by an external display (e.g., the display of a personal computer, tablet, or smartphone), wherein the 3D monitoring device is configured to establish a (wired or wireless) data and communication connection with the external display or a computing unit associated with the external display. Various different technologies existing in the art can be used to establish this data and communication connection. For example, the connection can be implemented in a so-called "direct" manner (device-to-device) via, for example, Bluetooth or a peer-to-peer architecture, or the connection can be implemented in an "indirect" manner via, for example, a server-client architecture.

[0030] The 3D sub-region has the shape of a 3D body spanned by connections of at least four corner points. A graphical user interface provides the user with the ability to modify the shape of the sub-region by dragging one of its corner points to a different location within the 3D visualization of the 3D environment model, thereby generating a redefined sub-region. The spatial parameters associated with the redefined sub-region within the 3D environment model are then provided to a 3D monitoring device, which generates an action (e.g., an alarm) upon detecting movement within the redefined sub-region using 3D measurement data.

[0031] For example, spatial parameters provide 3D monitoring equipment with information that enables the identification of the boundaries or complete outlines of redefined subregions to deduce whether movement has occurred within the redefined subregions. Alternatively or additionally, spatial parameters may provide specific processing and evaluation rules that allow for the direct identification of whether movement exists within the redefined subregions (e.g., without knowing the specific boundaries of the redefined subregions).

[0032] By dragging, the shape of the 3D sub-region deforms, which differs from simply expanding or shrinking the 3D sub-region. The deformation results in a change in the angle of incidence between two adjacent surfaces of the 3D body defining the 3D sub-region. Of course, the method can include additional steps to rapidly expand or shrink the 3D sub-region without deforming its shape. Additionally, other generally known measures for viewing and / or arranging 3D sub-regions within a 3D environment model can be implemented, such as providing lateral (xyz) movement and adaptive scaling of the 3D visualization of the 3D environment model, for example, utilizing a single scan or tap encoding (e.g., similar to a smartphone).

[0033] As an example, the graphical user interface provides different views for setting up a 3D visualization of a 3D environment model, where dragging one corner point can be performed in each of the different views.

[0034] For example, by allowing / forcing the deformation of the shape of a 3D subregion by dragging a corner point, intuitive user operations are provided for (re)defining 3D subregions. Users can select the best view of the 3D visualization to move and place specific corner points, where only restricted and explicitly defined movements (or no movements at all, see below) of other (not directly affected) corner points or boundaries of the 3D subregion are performed, which, for example, prevents incorrect monitoring of region definition.

[0035] As an example, if deformation is not allowed (e.g., dragging causes a 3D sub-region to expand or shrink as a whole), the new placement of the dragged corner points can be optimized. However, simultaneously, due to the overall shrinkage of the contour, the altered contour of the redefined sub-region may intrude into other areas of the 3D environment model or leave gaps in the safety net. Therefore, a complex iterative step of resizing the 3D sub-region along with rotation and linear repositioning will be required. Furthermore, optimizing the fit of each boundary of the redefined sub-region to the 3D environment model may still be impossible.

[0036] In one implementation, the 3D sub-region is spanned by a top surface and a bottom surface, and by connections between the top and bottom surfaces, each of which includes at least three corners. All edges (edges of the 3D body) of the shape of the 3D sub-region formed by the connections between the corner points of the top and bottom surfaces are parallel to each other. For example, in addition to the top and bottom surfaces, the lateral surfaces defining the 3D sub-region are vertically arranged relative to a defined horizontal / vertical orientation within the 3D environment model. Dragging one corner point causes a movement of the corner points connected to it (the corner point of the 3D sub-region), such that the edge of the 3D sub-region defined by the connections between one corner point and its connected corner points remains parallel to the other edges of the shape of the 3D sub-region formed by the connections between the corner points of the top and bottom surfaces.

[0037] In another embodiment, each of the top and bottom surfaces is a planar polygonal surface with at least three corners, for example, wherein the top and bottom surfaces are arbitrarily inclined relative to each other.

[0038] As an example, in another implementation, the change function is configured such that by dragging a corner point of the top or bottom surface, the entire top or bottom surface is tilted, such that it includes a new position of one of the corner points. Alternatively or additionally, the tilt of the top or bottom surface is set by an input function via a graphical user interface, for example, by defining the tilt angle via a menu, and the tilt angle of the top or bottom surface remains fixed thereon.

[0039] In another implementation, the positions of all corner points in a 3D sub-region within the 3D environment model, except for one (dragged) corner point, are not affected by the dragging of that one corner point.

[0040] As an example, the change function provides a choice of different dragging modes, wherein, in the so-called parallel mode, the edge of the 3D sub-region defined by the connection of one corner point to the connected corner points remains parallel to the other edges of the shape of the 3D sub-region as described above; and in the so-called free mode, the positions of all corner points of the 3D sub-region within the 3D environment model, except for the (draggled) corner point, are not affected by the dragging of that corner point.

[0041] In another implementation, the change function provides the deletion and addition of corner points in the 3D subregion, for example, where adding includes clicking the edge at any location on the edge of the 3D subregion (e.g., dragging a "new" point on the edge immediately after clicking), and / or where adding includes clicking the surface at any location on the surface of the 3D subregion (e.g., dragging a "new" point on the surface immediately after clicking). For example, the change function is configured such that clicking and dragging the surface introduces kinks in the surface or allows the surface to warp in a defined manner.

[0042] As an example, addition and deletion are configured to generate or delete parallel (e.g., vertical) edges of 3D subregions as described above. Thus, each time an additional corner point is added on the top surface, a corresponding added corner point is introduced on the bottom surface (and vice versa), such that the edge defined by the connection of two added corner points is parallel to the shape of the 3D subregion and other edges generated by the connection of the corner points on the top and bottom surfaces.

[0043] In other words, improved and flexible monitoring area definition capabilities are provided for the aforementioned new category of 3D surveillance equipment. Referring to the example mentioned at the beginning regarding building complexes that include or are adjacent to green areas to be monitored, the boundaries of the green areas can be updated frequently, for example, based on the growth status of the vegetation or based on weather and wind conditions. This allows for frequent updates to the monitoring area definition, providing an optimal trade-off between minimizing false alarms and robust shielding.

[0044] The method according to the invention provides intuitive control of 3D monitoring devices and intuitive definition and redefinition of 3D sub-regions, enabling users to quickly adapt monitoring conditions to changing environments or situations, such as in the event of an alarm.

[0045] For example, in the case of generating a 3D measurement model using 3D measurement data from a 3D monitoring device, where the 3D measurement is updated based on 3D measurement data of the spatial volume of the environment to be monitored at defined intervals and / or repeatedly generated when a defined event is detected within the 3D environment model, knowledge of the real-time conditions of the environment can be used to perform the definition and redefinition of 3D sub-regions.

[0046] In another implementation, the method includes the steps of: storing the movement history of movements within the 3D environment model; and analyzing the redefined sub-region when a change function is performed to provide feedback on the movement history within the redefined sub-region via a graphical user interface. For example, this allows a user to manually find the appropriate distance from the redefined sub-region to the surface of the 3D environment model, which may result in (non-alarming) movement detection, such as a green area.

[0047] In another implementation, the redefined sub-region is analyzed in real time while providing spatial parameters to the 3D monitoring device for motion detection within the redefined sub-region 5, and real-time feedback on motion detection is provided via a graphical user interface.

[0048] For example, this allows for fine-tuning of the boundaries of redefined 3D sub-regions for potentially moving objects near the environment. Referring again to the example mentioned at the beginning where the group of buildings to be monitored includes or is adjacent to a green area, using...

[0049] Users can finely adjust the edge of a 3D sub-region toward a green area (e.g., plants moving in the wind) by dragging one corner point and receive real-time feedback on detected movement within the adjusted sub-region. This allows users to iteratively find the optimal trade-off between small transition areas (which strictly define the area from the outside) and the number of false alarms (which are valid for the current environmental conditions). For example, in each instance of dragging one corner point of the 3D sub-region, the spatial parameters associated with the redefined sub-region...

[0050] Data is provided to the 3D monitoring device in real time, so that with each drag, the user receives immediate feedback on the movement within the redefined sub-area.

[0051] In another implementation, the input function provides the definition of the 3D sub-region by selecting optional options from a list of automatic shapes for the 3D sub-region or by arbitrarily defining the corner points of the 3D sub-region (e.g., by means of a drawing function for arbitrarily drawing 3D sub-regions).

[0052] In another embodiment, the method includes the steps of: analyzing 3D measurement data, and based thereon, providing via a graphical user interface an indication of a portion of the 3D environment model having a movement history associated with a defined movement category among a plurality of movement categories.

[0053] For example, different movement classifications indicate different threat levels for events; for instance, an event frequency exceeding a defined tolerance frequency triggers a defined warning level. Movement that typically occurs irregularly and is associated with objects in a non-moving environment...

[0054] For example, wind causes plants to move, and birds fly across the environment. Conversely, an intruder moving through the environment will trigger a continuous movement alert. Even in cases where an intruder attempts to move irregularly, statistical methods can be used to analyze the movement history of a part of the environment to distinguish between non-suspected and suspicious movement histories.

[0055] Users can then specifically target problematic sections of the environment, for example, by re-defining the monitoring area for these problematic sections. As an example, the input functionality includes automatically providing (suggested) 3D sub-regions around the sections within the 3D environment model that are associated with the defined movement classification.

[0056] Based on movement history (e.g., through statistical analysis of movement history), the sensitivity level used to issue alarms can be increased or decreased. This could, for example, be beneficial for optimizing the processing load on 3D monitoring equipment. Therefore, in another implementation, a defined movement classification is associated with an evaluation rule used to classify 3D measurement data to generate an action upon detection of movement within a redefined sub-region. The graphical user interface then provides feedback to the user to confirm or modify the evaluation rule and provides the confirmed or modified evaluation rule to the 3D monitoring equipment, enabling the 3D monitoring equipment to generate an action based on the confirmed or modified evaluation rule.

[0057] In another embodiment, the method includes the following steps: providing a snapping-in step via a graphical user interface, wherein, during the dragging of a corner point, on the one hand, the relative geometric arrangement of the corner point and / or the associated surface of the 3D sub-region including the corner point is analyzed, and on the other hand, the relative geometric arrangement of the region (e.g., surface) of the 3D environment model is analyzed. Based on the relative geometric arrangement, a defined snapping-in arrangement of the corner point or associated surface is suggested such that the corner point or associated surface is attached to the region of the 3D environment model in a defined manner. For example, if a sidewall of the 3D sub-region is to be dragged close to a wall of the environment, this is automatically identified, such that the sidewall of the 3D sub-region is automatically arranged (and snapped) parallel to the wall of the environment.

[0058] In another implementation, the locking step includes automatically determining the distance of the locking arrangement relative to a region of the 3D environment model. For example, a user-defined minimum distance between the locking arrangement and a region of the 3D environment model can be provided via user input through a graphical user interface. For example, to define the minimum distance, the user can use feedback regarding movement history within the redefined sub-regions as described above, for example, feedback obtained from a stored movement history of movements within the 3D environment model (see above).

[0059] Alternatively or additionally, the distance is automatically determined by statistical movement classification associated with the region of the 3D environment model, and based on this, the minimum distance automatically obtained between the snap-in arrangement and the region of the 3D environment model is provided.

[0060] As an example, the minimum distance differs for regions with different movement expectations. A larger minimum distance may exist between the boundaries of a 3D subregion facing a green area with vegetation (which has more frequent plant movement) than between the boundaries of rigid (e.g., concrete) walls of a 3D subregion facing the environment.

[0061] The present invention further relates to a system comprising a 3D monitoring device and a computing unit, wherein the computing unit is configured to provide data communication with the 3D monitoring device and an electronic graphic display, for example, wherein the 3D monitoring device includes the computing unit.

[0062] The system is configured to perform the method according to one embodiment of the above embodiments, wherein the 3D monitoring device includes a 3D coordinate measurement unit configured to capture the spatial volume of the environment using 3D coordinate measurement data based on the time-of-flight principle. The system (e.g., the 3D monitoring device) includes an object detector (object detection algorithm) configured to detect objects within a defined sub-region of the monitored spatial volume of the environment based on the 3D coordinate measurement data of the 3D monitoring device. For example, a 3D change detection algorithm is used for object detection.

[0063] The computing unit is configured as follows:

[0064] Read the input data associated with the 3D monitoring device, where the input data provides coordinate information associated with the position of the 3D monitoring device within the 3D environment model.

[0065] The generation of a graphical user interface on an electronic graphic display provides a 3D visualization of a 3D environment model, and the 3D visualization includes indication of the location of a 3D monitoring device.

[0066] The system provides input functionality to the user via a graphical user interface to define 3D sub-regions within a 3D environment model. Each 3D sub-region has the shape of a 3D body spanned by connections of at least four corner points.

[0067] The graphical user interface provides users with the ability to modify a 3D subregion by dragging one corner point of the 3D subregion to a different location within the 3D visualization of the 3D environment model, thereby generating a redefined subregion. This involves shape deformation of the 3D subregion, and...

[0068] It provides spatial parameters associated with redefined sub-regions within a 3D environment model, and enables 3D monitoring devices to generate actions, particularly alarms, when movement within the redefined sub-regions is detected using 3D measurement data.

[0069] In one embodiment, the 3D coordinate measurement unit is implemented as a lidar unit, which is configured to provide 3D coordinate measurement data by performing distance measurement through emitting a laser beam and detecting the return portion of the laser beam. The lidar unit includes a base, a support, and a rotating body. The support is mounted on the base such that it is rotatable about a first axis, and the rotating body is arranged and configured to rotate about a second axis perpendicular to the first axis and provide variable deflection of the outgoing and returning portions of the laser beam, thereby providing rotation of the laser beam about the second axis.

[0070] As an example, the rotating body rotates about a second axis at a frequency of at least 50 Hz, and the laser beam rotates about a first axis at a frequency of at least 0.5 Hz, wherein the laser beam is emitted as a pulsed laser beam, for example, wherein the pulsed laser beam comprises 1.2 million pulses per second. Specifically, the 3D coordinate measurement unit is configured to have a field of view of 360 degrees about the first axis and 130 degrees about the second axis, and generates 3D coordinate measurement data at a point acquisition rate of at least 150,000 points per second.

[0071] In another implementation, the 3D monitoring device is configured to provide at least one of selective storage, selective processing, and selective generation of 3D coordinate measurement data based on a defined sub-region (e.g., based on a redefined sub-region). As an example, the 3D monitoring device is configured to preferentially generate and / or process 3D coordinate measurement data covering the defined sub-region. Alternatively or additionally, the 3D monitoring device is provided with information for defining movement classification within the defined sub-region or association evaluation rules for classifying the 3D measurement data (e.g., see above), which enables the 3D monitoring device to process and analyze the 3D measurement data of the defined sub-region in a specific manner, for example, to obtain actions such as alarms associated with movement within the defined sub-region.

[0072] The present invention further relates to a computer program product comprising program code, which, when executed by a computing unit of a system according to one embodiment of the embodiments described above, causes the system to perform the method described according to one embodiment of the above embodiments. Attached Figure Description

[0073] The methods, systems, and computer program products according to different aspects of the invention are described or explained in more detail below by way of example only, with reference to the schematic examples shown in the accompanying drawings. Identical elements are labeled with the same reference numerals in the drawings. The described embodiments are generally not shown to scale and should not be construed as limiting the invention. Specifically,

[0074] Figure 1 This is an exemplary embodiment of a 3D monitoring device that can be used in the monitoring system according to the present invention;

[0075] Figure 2 It is a so-called dual-axis laser scanner. Figure 1 An exemplary implementation of a 3D coordinate measurement unit;

[0076] Figure 3 The problems of using 3D monitoring devices in, for example, quasi-static changing environments are illustrated schematically;

[0077] Figure 4 schematically depicted Figure 3 A solution to the problem described;

[0078] Figure 5 schematically depicted Figure 3 Alternative solutions to the problem described;

[0079] Figure 6 An implementation of the function change is schematically depicted, wherein dragging one corner point causes the connected corner points to move, such that the side surface and the previously parallel edge remain parallel to the other edge and the other side surface, respectively.

[0080] Figure 7 An alternative implementation of the function change is illustrated schematically, in which the positions of all corner points except the dragged corner point are unaffected by dragging.

[0081] Figure 8 The illustration depicts the addition of a new corner point within the modified functionality;

[0082] Figure 9 Different steps of implementing the method of the present invention are illustrated schematically;

[0083] Figure 10Different steps of another embodiment of the method of the present invention are illustrated schematically, wherein the 3D environment model is generated based on 3D measurement data from a 3D monitoring device. Detailed Implementation

[0084] Figure 1 An exemplary embodiment of a 3D monitoring device 1 that can be used in a monitoring system according to the present invention is shown. The top of the figure shows a side view of the 3D monitoring device 1, and the bottom of the figure shows a top view of the 3D monitoring device 1.

[0085] 3D surveillance equipment 1 includes a common sensor platform 2, which supports a time-of-flight 3D coordinate measurement unit 3 (e.g., regarding...). Figure 2 (as described) and, for example, an additional multispectral imaging unit comprising a multispectral camera 4 arranged on a circumferential region surrounding the cover of the 3D coordinate measuring unit 3.

[0086] In the example shown, the multispectral imaging unit includes two visual imaging cameras 4 and four thermal imaging cameras 5, each visual camera 4 having a field of view of at least 180 degrees and each of the four thermal imaging cameras 5 having a field of view of at least 80 degrees.

[0087] As an example, the 3D monitoring device 1 thus provides motion detection based on determining the deviation of the 3D point cloud from a frequently updated 3D background model and identifying changes in the 3D point cloud generated by the 3D measurement unit 1 based on visual and temperature information. Specifically, based on data from the 3D monitoring device 1, the system of the present invention is capable of frequently updating the background model to account for substantially static changes in the environment, such as slowly growing green areas, when defining different monitoring areas.

[0088] Figure 2 This illustrates a form of so-called dual-axis laser scanner. Figure 1 An exemplary embodiment of the 3D coordinate measurement unit 3 is described. The laser scanner includes a base 6 and a support 7, the support 7 being rotatably mounted on the base 6 about a vertical axis 8. Typically, the rotation of the support 7 about the vertical axis 8 is also referred to as azimuth rotation, regardless of whether the laser scanner or the vertical axis 8 is precisely vertically aligned.

[0089] At the heart of the laser scanner is an optical distance measurement unit 9, which is configured to perform distance measurement by emitting a pulsed laser beam 10 and detecting the return portion of the pulsed laser beam using a receiving unit including a photosensitive sensor, for example, where the pulsed laser beam comprises 1.2 million pulses per second. Thus, pulse echoes are received from backscattering surface points in the environment, and the distance to the surface point can be obtained based on analysis of the emission and return times, shapes, and / or phases of the emitted pulses.

[0090] The scanning movement of the laser beam 10 is performed by rotating the support 7 relative to the base 6 about a vertical axis 8 and by means of a rotating body 11, which is rotatably mounted on the support 7 and rotates about a horizontal axis 12. As an example, both the transmitted laser beam 10 and the returning portion of the laser beam are deflected by means of a reflective surface 13 of the rotating body 11. Alternatively, the transmitted laser radiation originates from the side opposite to the reflective surface 13 (i.e., from the interior of the rotating body 11) and is emitted into the environment via a channel region within the reflective surface.

[0091] To determine the emission direction of the distance measuring beam 10, many different angle determination units are known in the prior art. For example, the emission direction can be detected by means of an angle encoder configured to acquire angle data for detecting the absolute angular position and / or relative angular change of the support 7 or the rotating body 11, respectively. Another possibility is to determine the angular position of the support 7 or the rotating body 11 separately by detecting only the complete rotation and using knowledge of the set rotation frequency.

[0092] Visualization of data from a 3D coordinate measurement unit can be based on known data processing steps and / or display options, such as presenting the acquired data as a 3D point cloud or generating a 3D vector file model.

[0093] The laser scanner is configured to ensure that the measurement operation of the laser scanner covers a total field of view of 360 degrees in the azimuth direction defined by the rotation of the support 7 about the vertical axis 8 and at least 130 degrees in the tilt direction defined by the rotation of the rotating body 11 about the horizontal axis 12. In other words, regardless of the azimuth angle of the support 7 about the vertical axis 8, the laser beam 10 can cover a vertical field of view with a spread angle of at least 130 degrees in the tilt direction.

[0094] As an example, the total field of view typically refers to the central reference point of the laser scanner defined by the intersection of the vertical axis 8 and the horizontal axis 12. Here, the laser scanner is hidden behind a cover 14, which is opaque for the visual wavelength range but transparent for the (typically infrared) wavelength of the laser beam 10.

[0095] Figure 3The illustration depicts the use of a 3D monitoring device in, for example, a quasi-static changing environment. Figure 1 The problem is that, for example, a 3D monitoring area 15 of the environment is defined to be monitored by a 3D monitoring device, which then issues an alarm if movement is detected within the 3D monitoring area 15.

[0096] Typically, the monitored area extends close to the ground, for example, to prevent someone from crouching "below" the monitored area. In the example shown, monitored area 15 includes green area 16, for example, an area with ornamental plants.

[0097] In the state depicted at the top of the diagram, the plants in green area 16 are cut relatively short. Therefore, for example, monitoring area 15 begins above green area 16. In any case, given the short cut, not much plant movement is expected (e.g., due to wind).

[0098] In the state shown at the bottom of the figure, the plants in green area 16 have grown and now protrude into monitoring area 15. In this state, the plants may frequently trigger false movement alarms from the 3D monitoring equipment. For example, monitoring area 15 is located in the inner courtyard of a building complex, and tall plants can now move significantly (e.g., caused by wind), in which case the 3D monitoring equipment detects plant movement.

[0099] Figure 4 One solution to the problem is described, in which multiple different monitoring areas 15', 15' are defined around the vegetation area 16 (thus excluding the vegetation area 16). However, completely excluding the vegetation area 16 can prevent false alarms, but at the same time introduces vulnerabilities into the security network.

[0100] Ideally, such as Figure 5 As shown, newly defined monitoring areas 15', 15”, 15”' (or newly defined single monitoring areas with altered (e.g., more complex) shapes) are defined to account for changes in the state of the green area 16. The newly defined areas (or newly defined single areas) are closely adjacent to the green area 16, thereby achieving an optimal trade-off between false alarms and small tolerance areas between the monitoring areas 15', 15”, 15”' on all sides of the (unmonitored) green area 16.

[0101] For the sake of simplicity, Figure 5 Only vertical optimization is shown, which is provided, for example, by two laterally adjacent monitoring areas 15', 15' and a monitoring area 15"' starting immediately above the growth height of the plant in the green area 16. It goes without saying that similar optimizations can be achieved to further reduce lateral tolerances or for optimal shielding from the front and rear of the green area 16 (not shown).

[0102] Figure 6 An embodiment is schematically depicted in which a 3D sub-region 17 is spanned by a planar polygonal top surface 18 and a planar polygonal bottom surface 19, and by a connection between the planar polygonal top surface 18 and the planar polygonal bottom surface 19. The edges 20 of the shape of the 3D sub-region 17 formed by the connection of the corner points of the top surface 18 and the corner points of the bottom surface 19 are parallel to each other. For example, the side surfaces 21 defining the 3D sub-region 17 are all arranged vertically relative to a defined horizontal / vertical orientation within the 3D environment model.

[0103] The top of the figure depicts the state of the 3D sub-region 17 before dragging one of the corner points 22, and the bottom of the figure depicts the state of the redefined sub-region 24 after dragging one of the corner points 22.

[0104] In this embodiment, dragging one corner point of corner point 22 (here, the corner point of top surface 18) causes the connected corner point 23 (and possibly other corner points) to move, such that side surface 21 and previously parallel edge 20 remain parallel to the other edge 20 and other side surface 21, respectively.

[0105] Figure 7 Another implementation is schematically depicted, wherein the 3D sub-region 17 is initially designed in accordance with the above-described implementation. Figure 6 The implementation method is described in a similar manner. However, here, the positions of all corner points of the 3D sub-region 17, except for the dragged corner point 22, are not affected by the dragging of the dragged corner point 22.

[0106] Similarly, the top of the figure depicts the state of the 3D sub-region 17 before dragging one of the corner points 22, and the bottom of the figure depicts the state of the redefined sub-region 24 after dragging one of the corner points 22.

[0107] Figure 8 The diagram schematically depicts the addition of a new corner point 25 to a 3D subregion 17, with the top of the diagram showing the state of the 3D subregion 17 before the addition (and dragging) of the new corner point 25, and the bottom of the diagram showing the state of the redefined subregion 24 after the addition (and dragging) of the new corner point 25.

[0108] As an example, the addition involves clicking at any location on one of the edges (here, the edge of the top surface 18). Clicking adds a corner point 25 that can be dragged to a new location.

[0109] In the illustrated embodiment, the addition and dragging of new corner points 25 results in the generation of new vertical edges 26 in the redefined 3D sub-region 24, that is, corresponding added corner points 27 are automatically generated on the bottom surface 19, such that the new edges 26 are parallel to other edges generated by connecting the corner points of the top surface and the corner points of the bottom surface.

[0110] Figure 9 The schematic depiction illustrates the use of controls such as references. Figure 1 The present invention describes an implementation of the method for a 3D monitoring device.

[0111] The 3D monitoring device generates 3D measurement data 28 of the environment, wherein the 3D monitoring device is configured to provide the 3D measurement data 28 to an external computing unit 29, which has stored program code for executing the method described above. In the first step, the computing unit 29 reads 30 input data 31 associated with the 3D monitoring device, wherein the input data 31 provides information about the 3D environment model and coordinate information associated with the position of the 3D monitoring device within the 3D environment model.

[0112] In the next step, input data 31 is used to generate a graphical user interface on an electronic graphic display that provides a 3D visualization 32 of the 3D environment model, wherein the visualization 32 includes an indication of the position of the 3D monitoring device within the 3D environment model.

[0113] In a further step, the graphical user interface provides input functionality 33, whereby the user can define 3D sub-regions within the 3D environment model. The 3D sub-regions have the shape of a 3D body spanned by connections of at least four corner points.

[0114] After defining a 3D subregion within the 3D environment model, the program provides the user with a modification function 34, whereby the user can generate a redefined 3D subregion by dragging one of the corner points of the 3D subregion to a different location within the 3D visualization of the 3D environment model. This causes the shape of the 3D subregion to deform, where only restricted and explicitly defined movement (or no movement at all, see above) of the other corner points or boundaries of the 3D subregion is permitted.

[0115] Based on the redefined 3D sub-region, the computing unit 29 calculates spatial parameters 35, which provide geometric information of the redefined sub-region and its position within the 3D environment model. The spatial parameters 35 are then provided to a 3D monitoring device, which, upon detecting movement and / or spatial changes within the redefined sub-region, generates an action (e.g., an alarm).

[0116] Figure 10The schematic depiction illustrates the use of controls such as references. Figure 1 Another embodiment of the method of the present invention for the described 3D monitoring device.

[0117] Here, the method is different from that used for... Figure 9 The difference in the described implementation lies in that the computing unit 29 is configured to generate a 3D measurement model using 3D measurement data 28 provided by the 3D monitoring device, for example, wherein the 3D measurement model is repeatedly generated at defined time intervals. Therefore, the computing unit 29 generates and periodically updates the input data 31 associated with the 3D monitoring device. For example, the computing unit may use the measurement data 28 to generate a 3D point cloud or 3D vector file model for use as a 3D environment model. Since the 3D environment model is generated based on the 3D measurement data of the 3D monitoring device, the position of the 3D monitoring device within the 3D environment model is inherently known.

[0118] The remainder of steps 30, 32, 33, 34, and 35 are referenced. Figure 9 The steps described are similar.

[0119] Although the invention has been illustrated above with reference to some preferred embodiments, it should be understood that many modifications and combinations of different features of the embodiments can be made. All such modifications fall within the scope of the appended claims.

Claims

1. A method for controlling a 3D monitoring device (1), the 3D monitoring device (1) being configured to generate 3D measurement data (28) of the spatial volume of an environment and using the 3D measurement data (28) and a 3D environment model to monitor the spatial volume of the environment, the method comprising frequently updating the definition of a monitoring area provided by the 3D monitoring device (1) through the following steps: Read (30) the input data (31) associated with the 3D monitoring device (1), wherein, The input data (31) provides coordinate information associated with the position of the 3D monitoring device (1) within the 3D environment model. A graphical user interface is generated on an electronic graphic display to provide a 3D visualization (32) of the 3D environment model, wherein the 3D visualization (32) includes an indication of the location of the 3D monitoring device (1). The graphical user interface provides input functionality (33) to define a 3D sub-region (17) within the 3D environment model, wherein the 3D sub-region (17) has the shape of a 3D body spanned by the connection of at least four corner points (22, 23). The graphical user interface provides the user with the ability to modify (34) the shape of the 3D sub-region (17) by dragging one of the corner points (22) of the 3D sub-region (17) to a different position within the 3D visualization (32) of the 3D environment model, thereby deforming the shape of the 3D sub-region (17). The 3D monitoring device (1) is provided with spatial parameters (35) associated with the redefined sub-region (24) within the 3D environment model, and the 3D monitoring device (1) generates an action upon detection of movement within the redefined sub-region (24) by means of the 3D measurement data (28). The 3D sub-region (17) is spanned by a top surface (18) and a bottom surface (19), and by a connection between the top surface (18) and the bottom surface (19). Each of the top surface (18) and the bottom surface (19) includes at least three corner points (22, 23). All edges (20) of the shape of the 3D sub-region (17) formed by the connection between the corner points of the top surface (18) and the corner points of the bottom surface (19) are connected. This parallelism, wherein the dragging of one of the corner points (22) causes the movement of the connected corner point (23), such that the edge of the 3D sub-region (17) defined by the connection between the corner point (22) and the connected corner point (23) remains parallel to the other edges (20) of the shape of the 3D sub-region (17) formed by the connection between the corner point of the top surface (18) and the corner point of the bottom surface (19).

2. The method according to claim 1, wherein, The action described is an alarm.

3. The method according to claim 1 or 2, wherein, The change function (34) provides the deletion and addition of corner points (25, 27) of the 3D sub-region.

4. The method according to claim 3, wherein, The additions include: Click the edge (20) at any location on the edge of the 3D sub-region (17), and / or, Click the surface at any location on the surface of the 3D sub-region (17).

5. The method according to claim 1 or 2, wherein the method comprises the following steps: A 3D measurement model is generated using the 3D measurement data (28) from the 3D monitoring device (1).

6. The method according to claim 5, wherein, The 3D measurement model is generated based on the 3D measurement data (28) used to monitor the spatial volume of the environment, at defined intervals and / or repeatedly when a defined event is detected within the 3D environment model.

7. The method according to claim 1 or 2, wherein the method comprises the following steps: Store the movement history within the 3D environment model; And when performing the change function (34), analyze the redefined sub-region (24) to provide feedback on the movement history within the redefined sub-region (24) via the graphical user interface.

8. The method according to claim 1 or 2, wherein the method comprises the following steps: The 3D monitoring device (1) is provided with the spatial parameters to perform motion detection within the redefined sub-region (24) in real time, and real-time feedback on the motion detection is provided via the graphical user interface.

9. The method according to claim 1 or 2, wherein the method comprises the following steps: The 3D measurement data (28) is analyzed, and based thereon, an indication is provided via the graphical user interface of a portion of the 3D environment model having a movement history associated with a defined movement category among a plurality of movement categories.

10. The method according to claim 9, wherein, The input functionality includes automatically providing the 3D sub-region (17) around the portion of the 3D environment model associated with the defined movement classification.

11. The method according to claim 9, wherein, The defined movement classification is associated with an evaluation rule used to classify the 3D measurement data (28) to generate the action if movement is detected within the redefined sub-region (24). The graphical user interface provides users with feedback functionality to confirm or modify the evaluation rules, and The confirmed or modified evaluation rules are provided to the 3D monitoring device (1), so that the 3D monitoring device (1) generates the action based on the confirmed or modified evaluation rules.

12. The method according to claim 1 or 2, wherein the method comprises the following steps: A snap-in step is provided via the graphical user interface, wherein, during the dragging of one of the corner points (22), the relative geometric arrangement of the following items is analyzed, and based on this, a defined snap-in arrangement of the associated surface of the one of the corner points (22) or the 3D sub-region (17) including the one of the corner points (22) is suggested, such that the one of the corner points (22) or the associated surface is attached to the region of the 3D environment model in a defined manner: The corner point (22) and / or the associated surface, and The area of ​​the 3D environment model.

13. The method according to claim 12, wherein, The insertion step includes automatically determining the distance of the insertion arrangement relative to the region of the 3D environment model, wherein, The user-defined minimum distance between the card placement and the area of ​​the 3D environment model is provided via user input through the graphical user interface, and / or The automatic distance determination includes a motion statistics classification associated with the region of the 3D environment model, and based on this, provides an automatically obtained minimum distance between the snap-in arrangement and the region of the 3D environment model.

14. A system comprising a 3D monitoring device (1) and a computing unit (29), wherein, The computing unit (29) is configured to provide data communication with the 3D monitoring device (1) and the electronic graphics display, wherein the system is configured to provide frequent updates to the definition of the monitoring area provided by the 3D monitoring device (1), wherein, The 3D monitoring device (1) includes a 3D coordinate measurement unit (3), which is configured to capture the spatial volume of the environment by generating 3D coordinate measurement data (28) based on the time-of-flight principle. The system includes an object detector configured to detect objects within a defined sub-region of a monitored spatial volume of the environment based on 3D coordinate measurement data (28) from the 3D monitoring device (1). The computing unit (29) is configured as follows: o Read (30) input data (31) associated with the 3D monitoring device (1), wherein the input data (31) provides coordinate information associated with the position of the 3D monitoring device (1) within the 3D environment model. o Provides the generation of a graphical user interface on the electronic graphic display, wherein the graphical user interface provides a 3D visualization (32) of the 3D environment model, and the 3D visualization (32) includes an indication of the location of the 3D monitoring device (1). o Provides input functionality (33) to the user via the graphical user interface to define a 3D sub-region (17) within the 3D environment model, wherein the 3D sub-region (17) has the shape of a 3D body spanned by the connection of at least four corner points (22, 23). o Provides the user with a modification function (34) via the graphical user interface to generate a redefined subregion (24) by dragging one of the corner points (22) of the 3D subregion (17) to a different position within the 3D visualization (32) of the 3D environment model, thereby deforming the shape of the 3D subregion (17), wherein the 3D subregion (17) is spanned by a connection between a top surface (18) and a bottom surface (19), wherein each of the top surface (18) and the bottom surface (19) includes at least three corner points (22, 23), wherein the 3 All edges (20) of the shape of the 3D sub-region (17) formed by the connection of the corner points of the top surface (18) and the bottom surface (19) are parallel to each other, wherein the dragging of one of the corner points (22) causes the movement of the connected corner point (23), such that the edge of the 3D sub-region (17) defined by the connection of the corner point (22) and the connected corner point (23) remains parallel to the other edges (20) of the shape of the 3D sub-region (17) formed by the connection of the corner point (22) and the bottom surface (19), and o Provides spatial parameters (35) associated with the redefined sub-region (24) within the 3D environment model, and causes the 3D monitoring device (1) to generate an action if movement within the redefined sub-region (24) is detected by means of the 3D coordinate measurement data (28).

15. The system according to claim 14, wherein, The 3D monitoring device (1) includes the computing unit (29). The object detector is configured to detect objects within a defined sub-region of the monitored spatial volume of the environment based on 3D coordinate measurement data (28) from the 3D monitoring device (1) and on 3D change detection. The action mentioned is an alarm.

16. The system according to claim 14, wherein, The 3D coordinate measurement unit (3) is implemented as a lidar unit, which is configured to provide the 3D coordinate measurement data (28) by performing distance measurement through emitting a laser beam (10) and detecting the return portion of the laser beam (10), wherein, The lidar unit includes a base (6), a support (7), and a rotating body (11), wherein the support (7) is mounted on the base (6) in such a way that the support (7) can rotate about a first axis (8), and the rotating body (11) is arranged and configured to rotate about a second axis (12) perpendicular to the first axis (8) and provide variable deflection of the output and return portions of the laser beam (10), thereby providing rotation of the laser beam (10) about the second axis (12).

17. The system according to any one of claims 14 to 16, wherein, The 3D monitoring device (1) is configured to provide at least one of selective storage, selective processing and selective generation of the 3D coordinate measurement data (28) based on the defined sub-region.

18. The system according to claim 17, wherein, The 3D monitoring device (1) is configured to provide at least one of selective storage, selective processing and selective generation of the 3D coordinate measurement data (28) based on the redefined sub-region (24).

19. A computer program product comprising program code, which, when executed by a computing unit (29) of a system according to any one of claims 14 to 18, causes the system to perform the method according to any one of claims 1 to 13.

Images