MODELING A SITUATION

Description

[0001] According to the prior art, a first image sensor has a first sensor effective range. The first image sensor creates first image sensor data from this first sensor effective range in a form comprising at least image data, which first image sensor data are forwarded to downstream units, such as an alarm unit, in a format known from the prior art or in an equally known form. The first image sensor can comprise a unit for converting the first image sensor data into a format suitable for processing in the alarm unit or a corresponding form, which also represents prior art to this disclosure and is not part of the invention discussed here. The inventive method described below can merely build on this.

[0002] The first image sensor data describes the sensor effective range of the first image sensor in the form of an image or a video, which image or video can be processed and saved as image data. Further data can be created from the image data using common methods, which further data describe the sensor effective range or properties of the sensor effective range.

[0003] By definition, the first sensor range includes a first location.

[0004] In the alarm unit, the incoming initial image sensor data—regardless of the format and form in which it is stored—can be analyzed and classified to determine whether the first image sensor is recording or recorded a normal or unusual situation. State-of-the-art methods can be used here, which are typically based on object recognition and / or pattern recognition.

[0005] It is known in the art that the first image sensor creates first real image sensor data. For example, the first image sensor creates an image of a building or a portion of a building or another definable area that can be stored in the form of image data.

[0006] It is also known in the prior art that, using plans or a design or images of a building or another definable area as input data for the method described below, a model of the building or area is created. Within the scope of the following disclosure of the method according to the invention, no distinction is made between the term "building" and the term "area." Taking into account first image sensor properties of the image sensor arranged in the area or directed toward this area, such a model can be created. This model describes at least the effective range of the first image sensor and can be stored as image sensor model data.

[0007] The image sensor model data includes at least image data of the model. The image sensor data may include additional data, which additional data describe properties of the model.

[0008] Taking into account the virtual first image sensor entered in the plan and based on the image sensor model data, virtual image data can be created. This virtual image data describes an image that a real first image sensor located in a real area corresponding to the plans would capture. This, as well as the aforementioned creation of the model and the image sensor model data, is usually carried out using state-of-the-art computer programs.

[0009] DE102015205135A1 does not disclose a method in which the sensor data are supplemented with virtual object movement data or in which simulation data for further processing is created from the sensor data and the virtual object movement data.

[0010] DE1012015205135A1 relates to a method for evaluating (second) data with regard to a possible degree of automation. DE1012015205135A1

[029] mentions that the degree of automation is determined depending on simulated properties of objects. There is no indication in DE1012015205135A1 that the (second) data are supplemented by the simulated or virtual properties of objects or that simulation data are created from the (second) data and the simulated or virtual properties of the objects.

[0011] DE10207105903A1 discloses a method for detecting the collision of two objects, applying methods known from the prior art for training neural networks. DE10207105903A1 contains no reference to supplementing the data with virtual motion data or creating simulation data from the data and the virtual motion data.

[0012] DE102019122790A1 concerns a method for controlling a robot. There is no indication of the generation of an alarm situation described by the simulation data.

[0013] There is no indication in US10503976 of the generation of an alarm situation described by the simulation data.

[0014] DE102008055932A1 discloses a method in which the object polygons actually detected by a sensor, which object polygons represent detected objects, are modified in order to determine the sensor properties with regard to the detection of the objects. The method is limited to modifying the parameters of the polygons (see DE102008055932A1

[0028] ). The principle of modifying the parameters of the polygons differs fundamentally from the principle of the method discussed here, which supplements the sensor data with virtual object movement data or creates simulation data from the sensor data and the virtual movement data.

[0015] In a closed-loop HIL system, data is inserted behind an image sensor output in order to test the subsequent system.

[0016] DE10047082, which is cited as prior art for DE102008055932A1, describes a simulation system for longitudinal control in vehicles. The described system includes a computer unit containing environmental parameters, which is configured to load a stored situation and start a closed-loop simulation. It is possible to embed additional vehicles with a speed profile in a virtual environment. This document contains no indication that the environmental parameters are created as image sensor data at a different time than the vehicle movement profiles. US 2017 / 372575 describes a method for verifying the correct detection of an alarm system using simulated objects in an environment.

[0017] According to the state of the art, the effectiveness of an image sensor installed in an area, and thus a real one, is tested by simulating a situation such as the movement of an object or person in the area as accurately as possible, which is generally a very complex process. The image sensor data recorded using the real image sensor is forwarded to an alarm unit, which, using state-of-the-art methods, classifies the simulated movement of the object or person as a normal movement or an unusual movement and accordingly evaluates the situation, including the simulated movement of the object or person, as an alarm event.For example, a burglary situation in a building can be simulated and it can be checked whether an alarm system linked to the real image sensor recognizes the burglary situation as such a situation and, if necessary, triggers an alarm signal.

[0018] Every recording of a situation using an image sensor is subject to disruptive influences, such as, but not limited to, different weather scenarios. In addition to the complexity of recreating the situation itself, the aforementioned re-creation of unusual situations also involves the problem that these disruptive influences can only be recreated to a limited extent or not at all. This is a serious problem, especially since these disruptive influences can significantly influence the image sensor data. Essentially, the influence of disruptive influences on the re-created movements can only be determined if the respective disruptive influence actually exists. Testing the influence of snowfall on an outdoor surveillance camera is therefore limited to winter and the actual occurrence of snowfall. This is by no means a satisfactory situation for an expert.

[0019] Recreating the movement of the object or person also depends on the first location where the movement is to be recreated being accessible at the required time. Recreating an escape attempt from a correctional facility during snowfall or rain combined with fog at night is a task that cannot be accomplished by recreating the situation.

[0020] According to the state of the art, the testing of image sensors integrated into alarm systems is not possible at the planning stage, for example, before the construction of a building or an area. As explained above, the state of the art only allows for the creation of a virtual image sensor model based on calculated virtual image data at the planning stage. Using this virtual image data, the testing is limited to the extent of the field of view of the image sensor – hereinafter referred to as the first image sensor. No statement can be made as to whether the image sensor provides sufficiently analyzable image sensor data so that a situation can be classified by the downstream alarm unit as an exceptional situation, such as a vehicle accident in a parking garage in foggy conditions, or as a normal situation.

[0021] The method disclosed below aims to overcome this disadvantage according to the prior art.

[0022] The invention discussed below has the object of determining whether an alarm system recognizes an unusual situation as such a situation and, if necessary, triggers an alarm signal. It is the object of the method according to the invention to provide a possibility for a time-saving and efficient checking of the effectiveness of image sensors.

[0023] In the context of the discussion of the method according to the invention, no distinction is made between a person and an object unless explicitly stated and a specific technical effect can be derived from this distinction. In this sense, no distinction is made between a building, a part of a building, or a defined area.

[0024] The method according to the invention can furthermore have the task of changing or supplementing image sensor data or the image sensor model data by the smallest possible amount, namely, for example and thus not restrictively, by the object movement data described below, and thereby creating simulation data, which simulation data can be introduced into an alarm system for checking this alarm system.

[0025] According to the invention, the problems derivable from the prior art are solved by claim 1 or claim 2. Claim 1 essentially relates to the implementation of the inventive method in a virtual domain. Claim 2 essentially relates to the implementation of the inventive method in a real-world domain.

[0026] The effect of integrity can be achieved by modifying the image sensor data or image sensor model data to include only the object motion data. This change to the image sensor data is easily documented.

[0027] The method according to the invention provides that simulation data are created in the form of image data, which image data are present in a format known according to the prior art and in an equally known form, so that these image data generally comprise information about a generated image.

[0028] The image data can generally be in the form of videos or images in common file formats such as, but not limited to, avi, jpg, etc. or as a stream.

[0029] The image data may include coding of the objects depending on their properties. The objects may be color-coded using state-of-the-art methods. The image data, as video data, may include information about the frame of the video in which an object is described. The image data may include further information about properties of the objects described in the image data. Such descriptions are known to those skilled in the art by the technical term "ground truth."

[0030] The image data may include vector information about objects.

[0031] The image data can be an image of a sensor's effective area.

[0032] The simulation data represents an artificially created image of a situation to be checked by a downstream alarm unit, which does not actually occur during the implementation of the method according to the invention and thus exists virtually, namely that an object is virtually moved at the first location. This image corresponds with sufficient accuracy to a situation that actually occurs later.

[0033] The alarm unit may comprise a classification unit, by means of which the classification unit performs the classification of the virtual situation described by the simulation data. Since the simulation data are created by merging the image sensor data or the image sensor model data and the object movement data, the situation described by the simulation data does not actually exist and is considered a virtual situation within the scope of the disclosure of the method according to the invention.

[0034] The classification of a situation described using at least image data is typically a computer-implemented method. The process of classifying the situation itself is not part of the method according to the invention described here; the method according to the invention is limited to the creation of simulation data as input data for such a classification.

[0035] The above description includes the provision that the alarm unit is connected downstream of the creation of the simulation data. The creation of the simulation data according to the features described in the context of the disclosure of the invention can take place in a computing unit, so that the simulation data can be transmitted via data transmission. The simulation data can also be displayed on a monitor by the computing unit and recorded using an image sensor integrated in the alarm unit. The alarm unit can comprise a smart camera with event detection or situation detection.

[0036] According to the state of the art, the classification of a situation into an exceptional situation (alarm event) or a normal situation (no alarm event) is carried out using methods of object detection and / or analysis of the movement of the detected object. Those skilled in the art will recognize that the simulation data must be sufficiently accurate to ensure that this accuracy does not influence the classification of the virtual situation.

[0037] The method according to the invention can be used to specifically further develop the classification of situations carried out by the alarm unit by changing the simulation data.

[0038] The requirement of sufficient accuracy can be met by ensuring that the area and the sub-areas of the object have proportions to each other and to those occurring in reality when viewed individually.

[0039] The proportions of a person's body parts stored in the object motion data correspond to the proportions of the body parts found in reality. For example, the head of a person described in the object motion data has a head size compared to the other body parts of the person, which proportions can be found in a person in reality.

[0040] Likewise, the proportions of an object such as a vehicle must be maintained, which is described by the object motion data. The vehicle's tires have a tire size compared to other dimensions of the vehicle, so that the proportions of the real object and the virtual vehicle match.

[0041] Furthermore, the size ratio of the person or object to the building and vice versa must be observed.

[0042] Those skilled in the art will recognize that capturing a situation using an image sensor in general, here using the first image sensor, can be subject to geometric distortion depending on the image sensor properties. Accordingly, the simulation data can include a distorted object, with the distortion being determined by the properties of the first image sensor and thus corresponding to reality.

[0043] During object recognition, for example, geometric shapes are recognized as an example of properties of an object, based on which geometric shapes object recognition can be carried out. Sufficient accuracy is given when the alarm unit connected downstream of the image sensor can carry out object classification - here using the example of object recognition based on geometric shapes - using methods according to the prior art and when carrying out an embodiment of the method according to the invention using image data describing a real object. In a similar way, sufficient accuracy is given when an object classification can be carried out by the downstream alarm unit when applying a further embodiment of the method according to the invention using image data describing a virtual object.Those skilled in the art will be aware of further techniques for classifying a situation using an alarm unit, from which specific accuracy requirements can be derived, such as, but not limited to, the object appearance and shape described in the simulation data. The creation of the simulation data may include modifying the image sensor data created by the first image sensor with object movement data describing a real, ongoing object movement or a virtual object movement. This includes supplementing the first image sensor data with object movement data as image data of a moving object. In both cases, the simulation data describe a virtual situation, which situation as such does not actually exist within the sensor's effective range.

[0044] The object movement can either be a normal situation, which normal situation should be classified by an alarm unit as not being an alarm situation, or an exceptional situation, based on the detection of which exceptional situation the alarm unit should trigger an alarm upon classifying the object movement in the area as an exceptional situation. Within the scope of the disclosure of this invention, with reference to the prior art, an exceptional situation can be, for example and thus not restrictively, an unauthorized entry into an area by a person. A situation occurring in the area can be defined as an exceptional situation insofar as this exceptional situation is not desired by the person monitoring the area and the occurrence or further progression of this situation should be prevented.

[0045] In addition to classifying the situation, the alarm unit can also detect a condition within the first sensor range. For example, the alarm unit can detect, but is not limited to, the number of people in the first sensor range or the weather.

[0046] The real object movement data, which object movement data is created as data describing an object movement taking place at the second location using a second sensor, such as real second image data recorded with a second image sensor, can be inserted into the first image sensor data as second image data. In this case, the object movement taking place at a second location is virtually transformed to the first location, which in short has the technical advantage that the object movement at the first location does not have to be recreated at the first time. Furthermore, a plurality of sequential object movements or a plurality of simultaneous object movements can be integrated into the simulation data. The skilled person can design any desired combination, even random combinations.

[0047] In addition to an image sensor, a person skilled in the art may also use other sensors, such as, but not limited to, distance measurement sensors or three-dimensional measurement sensors (laser scanners, time-of-flight sensors), to record a real movement occurring at a second location and integrate it into the first image sensor data. A person skilled in the art may also use these sensors as first image sensors.

[0048] The object movement data can thus be inserted into the first image sensor data as image data from a movement that actually occurred at the second location. The object movement data can thus be referred to as real object movement data.

[0049] According to the prior art, it is possible to represent object movements with sufficient accuracy in the form of a computer simulation. Those skilled in the art are familiar with such simulations, for example, from computer games; such virtual object movements are also known to those skilled in the art as model movements or computer-generated animations. The method according to the invention can also include incorporating object movement data describing virtual object movements into the image sensor data.

[0050] Here, too, a multitude of sequential object movements or a multitude of simultaneous object movements can be integrated into the simulation data. The skilled person can create any combination, even randomly controlled combinations. In particular, object movement data, which describes virtual object movements, allow the integration of a multitude of object movements into the simulation data. In addition to the variance of the virtual object movements, the properties of the objects, such as, for example (and not limited to) color and size, can also be varied.

[0051] The variance of the object movement - regardless of whether it is a recorded real object movement or a virtual object movement - can be given, for example and thus not restrictively, by a variance of the movement speed and / or the movement direction and / or the movement range within the sensor's effective range.

[0052] The variance of the object movements can be selected by the user. The user can define a possible range of object movements. The variance of the object movements can also be randomly controlled.

[0053] The creation of the simulation data can also include merging the image sensor data recorded by the first image sensor and the virtual object movement data to form the simulation data. Those skilled in the art consider the modification of the first image sensor data by the object movement data and the merging of the first image sensor data and the object movement data to be fundamentally equivalent, although the computing processes and / or the required user input may differ when the method according to the invention is implemented on a computer. In this sense, the above description regarding the creation of simulation data by modifying the image sensor data by the object movement also applies to the creation of simulation data by merging the image sensor data and the object movement data.

[0054] The first image sensor data can be very extensive. The image sensor data can also include information about objects, etc., which information is recognizable to a person skilled in the art as being relevant for carrying out the method according to the invention.

[0055] The creation of the simulation data can also be based on creating the first image sensor model from the above-mentioned input data, while maintaining the sufficient accuracy described above. State-of-the-art methods are used to create the image sensor model. The input data can be plans, photographs, and dimensions of the area, from which a model is created using state-of-the-art computer programs.

[0056] A virtual first image sensor can be arranged in this model. Taking into account the properties of the first image sensor, image sensor model data can be created, which image sensor model data describe the model at least within the sensor effective range of the image sensor arranged in the model.

[0057] The image sensor model data can also describe a larger area than the sensor's effective range. This is particularly useful if the functionality of additional image sensors is to be tested using the image sensor model data.

[0058] The image sensor model data can be limited to information about objects, etc., that are relevant to the method according to the invention. The image sensor model data at most fulfills the requirement of sufficient accuracy, which sufficient accuracy has already been discussed above. The method according to the invention can comprise combining the image sensor model data and the virtual object motion data to form simulation data. This step can also comprise integrating the virtual object motion data into the image sensor model data, wherein combining the image sensor model data and the virtual object motion data and changing the image sensor model data by the virtual object motion data are considered to be equivalent to one another.

[0059] The image sensor model data can describe the model two-dimensionally or three-dimensionally, at least within the sensor's effective range. As a rule, a three-dimensional model is better suited for determining the functionality of an image sensor.

[0060] To meet the above-mentioned requirement of sufficient accuracy of the simulation data, the model sensor data can comprise a two-dimensional or three-dimensional image of the area with matching dimensions, taking into account any perspective distortion. Likewise, the dimensions of the object, which are incorporated into the method according to the invention via the object movement data, can be an image of the object or person with matching dimensions, taking into account any perspective distortion. The simulation data can thus comprise an image of the area and the object in the form of a two-dimensional or three-dimensional model, whereby the dimensions of the area and the object represented in the model should correspond to the real dimensions of the area or the object.The expert may choose a scaled representation, whereby the same scale is applied to the model of the area and the model of the object.

[0061] The question of the dimensionality of the image sensor model is also related to the dimensionality of the object motion. In principle, the image sensor model must have the same dimensionality as the object motion.

[0062] The method according to the invention may comprise combining the image sensor model data and the object movement data to describe a real object movement.

[0063] The expert can combine the above-mentioned process steps to create simulation data in any way he likes.

[0064] The image sensor model can—as disclosed above—be created on the basis of the first image sensor data, taking into account the properties of the first image sensor, wherein the image sensor data from a single first image sensor or from multiple first image sensors are used for this purpose. The question of how many first image sensors the image sensor data is obtained from is essentially a question of the accuracy and dimensionality of the image sensor model to be created and which properties the first image sensor has. A movable first image sensor or a first image sensor with a large sensor effective range can, for example, replace multiple first image sensors.

[0065] In addition to or as an alternative to the above description of creating the image sensor model, the image sensor model can be created based on planning data or on the basis of determined dimensions and measurements of the area. The person skilled in the art can use state-of-the-art sensors to capture the measurements and measurements.

[0066] The method according to the invention is characterized in that the first image sensor data are free of the object. The first image sensor data therefore do not include the moving object. The image sensor model possibly created from the first image sensor data also does not include the moving object. The first image sensor model is free of the moving object; the first image sensor model can be limited to static information.

[0067] The initial image sensor data or the image sensor model may include additional objects. These additional objects may not be relevant to the process.

[0068] The object movement data is created at a second point in time.

[0069] The second time point can be before or after the first time point and thus different from the first time point. The method according to the invention can thus be characterized in that the object movement data can be generated independently of the first sensor data.

[0070] The second time may be the same as the first time. The method according to the invention may be characterized in that the virtual object motion data are combined in real time with the first image sensor data or the image sensor model data.

[0071] The first location and the second location can be different. The object movement can therefore be created independently of the location.

[0072] The location independence and the time independence of the method according to the invention with regard to the object movement to the first location allows the virtual integration of the object movement to locations at times at which locations and at which times there is no access.

[0073] The method according to the invention comprises the step of perspective comparison. The viewing direction of the first image sensor to the first location and thus to the sensor's effective range should be similar to the viewing direction of the second image sensor to the second location and thus to the moving object. The method according to the invention can only deliver simulation data with sufficient accuracy if, for example, and not restrictively, the simulation data comprise a side view of the area, which side view of the area comprises at least the sensor's effective range, the requirement of sufficient accuracy can only be met if the simulation data comprise a side view of the moving object. Simulation data which, for example, comprise a side view of a facade of a building cannot comprise a view of a person from above.

[0074] This may also include scaling the proportions. This process step is necessary to achieve sufficient accuracy in the simulation data.

[0075] The virtual object motion data can be a three-dimensional model of the movement of an object or person. The method according to the invention can include the step of observing the moving object from a viewpoint, from which viewpoint the area is also viewed.

[0076] The use of an image sensor model and the use of object motion data, which describe the object movement using a model, can be useful, as the use of models can facilitate the comparison of viewing directions and the variance of viewing directions. By combining the object motion data and image sensor data, or the image sensor model data, as discussed here, a vector problem can essentially be reduced, a problem that is solvable by the expert using standard mathematical theory. A defined area, like a building or a section of a building, can be represented using vectors. CAD programs or other programs for the planning or visual representation of a building or an area are based on such a representation using vectors.

[0077] The image sensor model can be based on such a vector representation of the building. The image data created using the first image sensor can be in the format of a vector representation. The vectors can be two-dimensional or three-dimensional and, if necessary, time-dependent.

[0078] The object motion data may include a vectorial description of the object motion and, if appropriate, a vectorial motion of the object's subregions. The object motion may be described by waypoints and by object motion velocities.

[0079] The need to take into account movements of parts of an object may be a question of the required accuracy.

[0080] The object motion data can define a movement of the object using time-dependent vectors.

[0081] To create simulation data, the vector representation of the area can be supplemented or merged with the vector representation of the movement. The vector representation of the area and the object movement also allows for perspective adjustment of the viewing directions, so that simulation data is created with a movement occurring at the original location.

[0082] The method according to the invention can be characterized in that the image sensor data or the image sensor model data are created taking into account the properties of a first image sensor remaining at a single image sensor location.

[0083] The first image sensor can be an image sensor fixed in a specific area. A conventional surveillance camera can include such an image sensor. The surveillance camera can be pivotable and include a telephoto lens, allowing the sensor's effective range to be changed.

[0084] The use of an image sensor model and object motion data available in the form of a model can allow the selection of a first image sensor from a multitude of possible image sensors. This can be done regardless of whether the first image sensor is stationary or movable.

[0085] A typical drone may include the first image sensor.

[0086] The method according to the invention can be characterized in that the image sensor data are created by means of a first image sensor moved around an image sensor trajectory or the image sensor model data are created taking into account a first image sensor moved around an image sensor trajectory.

[0087] A typical drone or vehicle may include the first image sensor.

[0088] The method according to the invention can be characterized in that the object movement data describe movements of selected sub-areas of the object.

[0089] The method according to the invention creates simulation data by means of which simulation data an object movement of an object at the first location in the sensor effective range is described.

[0090] An object's movement within the area can be characterized by parts of the object moving relative to each other. When classifying a situation, it can be crucial that the simulation data describe the relevant parts of the object. The simulation data can describe the movements of all parts of the object.

[0091] The object movement data can describe the movements of all sub-areas of the object or those sub-areas whose movements influence the object detection and / or pattern recognition performed by the alarm unit. This can meet the requirement of sufficient accuracy.

[0092] The method according to the invention can be characterized in that the simulation data are modified by virtual environmental data or the simulation data are created to include virtual environmental data, which environmental data describe an environmental influence on the sensor's effective range.

[0093] The simulation data can include image data with variable brightness or contrast, so that the method according to the invention allows the simulation of different lighting scenarios. A variable number of light sources with different properties and the resulting different illumination can be described by the simulation data.

[0094] The example of snowfall or rain is mentioned above. The simulation data can include snowflakes as moving objects, so that the environmental impact of snowfall can be simulated using the object movement data.

[0095] The example of fog is also mentioned above. The simulation data may include, but is not limited to, stationary objects to simulate fog.

[0096] The method according to the invention can be characterized in that a single image sensor model is used to create a large number of different simulation data, which simulation data include different object motion data and / or which simulation data include different image sensor model data.

[0097] In the above description, it was already mentioned that the object movements can be varied, thus creating a variety of object movement data. This allows for the creation of a variety of simulation data, which is due to the variety of object movement data.

[0098] In the above description, it was further mentioned that the image sensor model data describe at least the sensor's effective range using a model. A plurality of image sensor model data can be based on a variance of the first image sensor considered, on a variance in the settings and properties of the image sensors considered, and on a variance in the model. This plurality of image sensor model data can give rise to a plurality of simulation data.

[0099] The multitude of simulation data can also be given by the multitude of object motion data and by the multitude of image sensor model data.

[0100] The above description mentions a single first image sensor for generating the image sensor data. However, the method according to the invention can be characterized in that the sensor data about the sensor effective range is generated by a single image sensor or by multiple image sensors. The above description mentions the generation of the image sensor model data describing the sensor effective range of a single first image sensor. However, the method according to the invention can be characterized in that the image sensor model data is generated describing the sensor effective ranges of multiple first image sensors.

[0101] The method according to the invention can be characterized in that the time span of the object movement data included in the method is different from a real time span of the object movement.

[0102] The virtual object movement can last for a period of time that can correspond to a real movement. The method according to the invention can be implemented such that the simulation data comprises a plurality of consecutive object movements or simultaneous object movements. To shorten the simulation time, the object movements can take place within a shorter period of time than the period of time typically required for a real movement.

[0103] The faster occurrence of object movements can affect both environmental influences such as snowfall or rain (for example and not limiting) and moving objects, which objects represent an ordinary situation (no alarm) or an extraordinary situation (alarm event) to be classified for the alarm unit.

[0104] The method according to the invention can also apply to the opposite case, namely, where the duration of the virtual movement is longer than the duration of a real movement. For example, the virtual object movement can be simulated in slow motion.

[0105] This allows the method according to the invention to be optimized with regard to the available computing power, provided the method according to the invention is implemented as a computer-implemented method. Furthermore, the user, who views the simulation data as image data, can view time periods of the simulation with a degree of accuracy that depends on the duration of the time period.

[0106] The method according to the invention can be characterized in that the first location and the second location are different and / or the first image sensor and the second image sensor are different.

[0107] The method according to the invention can be characterized in that the sensor effective area has area properties and the object has object properties, which area properties or object properties are variable.

[0108] The person skilled in the art can virtually change the area properties and / or the object properties and thereby create further simulation data describing the changed area properties and / or the changed object properties. In this way, the person skilled in the art can check the functionality of the first image sensor depending on the changed area properties and / or the changed object properties. The person skilled in the art can advantageously use the image sensor model data or the modeled object motion data for this purpose.

[0109] The area properties can, for example, and not restrictively, describe weather characteristics such as fog or lighting conditions. It is generally known that the quality of the image data created using image sensors depends heavily on these weather characteristics. However, subsequent processing of the image data, such as object recognition, can only be performed if the image data is of sufficient quality. According to the state of the art, limits to the possible processability of image data can be defined.

[0110] One possible application of the method according to the invention is to determine the limits of processability. The user can do this by changing the region properties. The processability of image data can also depend on the object properties. The user can change the object properties and thus determine the limits of the processability of the image data. For example, a person skilled in the art can change the color of the object as an object property. The user can thus check, for example, what color difference an object must have from a background so that the image data allows object recognition.

[0111] The invention is described in the following Figures 1 to 9 explained in more detail. Fig. 1: an image of the area generated using image sensor model data Fig. 2: a modeled object movement Fig. 3: a modeled object movement Fig. 4: a modeled object movement Fig. 5: an image of a situation generated using simulation data Fig. 6: an image of the area generated using image sensor model data Fig. 7: an image of the area generated using image sensor model data Fig. 8: an image of the area generated using image sensor model data Fig. 9: an illustration of the data streams

[0112] The embodiments shown in the figures merely illustrate possible embodiments. It should be noted at this point that the invention is not limited to these specifically illustrated embodiments. Combinations of the individual embodiments with one another and a combination of an embodiment with the general description above are also possible. These further possible combinations do not need to be explicitly mentioned, since these further possible combinations are within the skill of the person skilled in this technical field based on the teaching of technical action based on the present invention.

[0113] The figures show highly schematic images to illustrate the requirements of figures of a

[0114] Patent document. When the method according to the invention is implemented, the images can have greater accuracy and realistic precision.

[0115] The scope of protection is determined by the claims. However, the description and drawings must be used to interpret the claims. Individual features or combinations of features from the various embodiments shown and described may represent independent inventive solutions. The problem underlying the independent inventive solutions can be derived from the description.

[0116] In the figures, the following elements are identified by the preceding reference numerals: 1 Person 2 First location 3 Second location 4 Space for object movement 5 Coordinate system 6 Movement path 7 Free zone 8 Restricted zone 9 Starting point of the movement path 10 End point of the movement path 11 Building 12 Window 13 Bench 14 Car 15 Shadow 16 First image sensor 17 Alarm unit 18 First image data 19 Computer unit for carrying out the method according to the invention 20 Simulation data 21 Legs 22 Feet 23 Arms 24 Hands 25 Head 26 Tree

[0117] The Figure 1 shows a model of a building 11. The model illustrates the exterior of a building 11 to be monitored, which building 11 to be monitored actually exists, or which building to be monitored is in a planning stage or construction phase. At best, a person skilled in the art can create a model of the building 11 by applying their specialist knowledge without inventive activity. The image sensor model depicts the building 11 with sufficient accuracy so that, using a computer-implemented method, the situation can be classified comprehensively, including a model of the building 11 and, if applicable, the object movement.

[0118] The Figure 1 illustrates only the image sensor model data, which image sensor model data in the Figure 1 shown are examples. The Figure 1The view of building 11 shown can be a model view, which model view is created from an image sensor model, wherein the image sensor model is created from planning data or based on image data. The creation of model views from an image sensor model has the advantage that one image sensor model can be used to create multiple model views.

[0119] Building 11 includes windows 12, although only some of the windows in Figure 1 and are provided with a reference symbol in the other figures. Benches 13 are arranged in front of the building 11. Further objects such as lanterns, a tree 26, a sandbox and a trash can are arranged in front of the building 11. The image sensor model data described in Figure 1 The area shown comprises at least the effective sensor range of the first image sensor. The person skilled in the art will recognize that in the case of Figure 1the first image sensor is an external sensor, which external sensor is Figure 1 shown view of building 11.

[0120] The Figure 1 The view of the building 11 shown may also include, in addition to or as an alternative to the model view, a real view, which real view is recorded with the first image sensor.

[0121] The Figure 1 is free of persons and / or objects which, when classifying the situation, may represent a decision criterion for classifying the situation as an ordinary situation or an extraordinary situation or a non-classifiable situation.

[0122] The Figure 2 shows a scene of a possible object movement of a person 1. The person is similar to the representation of the building 11 in the Figure 1schematically shown to meet the requirements for figures in patent documents; a more realistic description of the figure is possible when carrying out the method according to the invention. Figure 2 Person 1, shown as an example, performs a climbing movement. Such a Figure 2 The scene shown as an example is incorporated into the method according to the invention as object movement data. The object movement data is available as image data.

[0123] The Figure 2 shows a person 1 moving to a second location 3. It is in Figure 2 the second location 3 is symbolized by a circle. After the Figure 2 represents a situation of object movement, the second location 3 is a point in the Figure 2 to understand the location of person 1 at the time shown in the space 4 shown.

[0124] The space 4 is defined by a three-dimensional coordinate system 5. The second location 3 in the space 4 can be defined as stationary or variable. If the second location 3 is defined as stationary, the person 1 performs an object movement at the second location 3. If the second location 3 is defined as a variable location, the person 1 performs a movement, here not restricted to a climbing movement, along a movement path 6.

[0125] The Figure 2 shows the movement of person 1 at a given point in time. The object movement data also describes the movement of parts of person 1. In particular, the object movement data describes the individual movements of the arms, legs, head, and other body parts of person 1.

[0126] The Figure 2The person 1 shown comprises legs 21 and arms 23. A detailed description of the movement of the person 1 in the object movement data can take into account a time-varying movement of the feet 22 and the hands 24, which time-varying movement of the feet 22 and the hands 24 can be observed in reality in a climbing movement shown here as an example. Figure 2 shows, for example, a deformation of a forefoot 22 and the grasping position of the hands 24. Furthermore, the head 25 of person 1 is tilted backwards.

[0127] A movement can be classified based on these partial movements or based on the movement of Person 1 as a whole object. The object movement data describe the movement of Person 1, possibly taking partial movements into account, with the sufficient accuracy described above, so that a classification of the movement of Person 1 is possible.

[0128] Person 1 is in Figure 2 Depicted with respect to human proportions; due to the perspective representation, distortions may occur. The object motion data describes the moving object, in this case the moving person 1, taking into account the proportions that occur in reality.

[0129] The Figure 2 shows a description of the object movement using a three-dimensional model, which model can be easily integrated into an image sensor model, which image sensor model describes the area with a two-dimensional or three-dimensional model.

[0130] The Figure 3shows another scene image of a person 1, which person 1 is being moved at the second location 3. The person 1 moves along a movement path 6, which movement path 6 runs exclusively in a free zone 7 of a space 4 describable by a coordinate system 5; the location 3 is therefore a variable location. The space 4 comprises blocked zones 8, in which blocked zones 8 no object movement can occur, and free zones 7, in which free zones 7 an object movement can occur. The blocked zones 8 are defined by objects situated in the space 4.

[0131] The person 1 is located at a variable second location 3; the change of the second location 3 is defined by the movement path 6.

[0132] The above definition of the movement path 6 allows the consideration of a multitude of possible object movements, namely those object movements that occur exclusively in the free zone 7. The person skilled in the art can create a multitude of possible object movements using computer-implemented methods, whereby the Figure 3 a possible object movement along the Figure 3 represents a possible movement path 6 from a starting point 9 to an end point 10.

[0133] The object movement data can comprise, as image data, the multitude of possible object movements of the person 1. The Figure 3 Person 1 depicted has the body proportions corresponding to a human. Figure 3 The object movement data shown may include a description of partial movements of the head 25, the arms 23 and the legs 21; the above description applies accordingly.

[0134] The Figure 4 shows an object movement of a person 1 running along a straight line along a movement path 6. The second location 3 changes with the position of the person 1, who moves from a starting point 9 of the movement path 6 to an end point 10 of the movement path 6 - here, for example, and thus not restrictively, running. The movement path 6 can thus be defined by points in a space 4, such as, for example, and not restrictively, by the starting point 9 and the end point 10, wherein the movement path 6 connects the points via the shortest path or in the form of a curve. The skilled person can also define further points or even free zones (in Figure 4 not registered, see Figure 3 ) define through which further points or free zones the movement path 6 runs.

[0135] The movement data may include the object movement of the person 1 along a movement path 6 extending from a starting point 9 to an end point 10 and possibly passing through further points or free zones.

[0136] The Figures 2 to 4 The object movements shown are virtual object movements, which are described by means of a model and take place at a second location 3. The object movements shown in the Figures 2 to 4 The object movements shown are based on a three-dimensional model, which can easily be combined with an image sensor model to create simulation data, taking into account the size ratios and viewing directions.

[0137] The Figures 2 to 4The persons 1 shown have body proportions that correspond to reality. The person skilled in the art can vary the characteristics of the persons 1 during or before combining object movement data and image sensor data in order to take into account a range of characteristics. For example, and thus by no means restrictively, the person's skin color, the color of the clothing, and the speed of movement of the person 1 can vary as characteristics. To achieve the sufficient accuracy described above, the object movement data can include a more realistic description of the person.

[0138] The skilled person can also record the real object movement of real people instead of the modeled people 1. However, the representation of people 1 in the form of a model has the technical advantage that the properties of the modeled people are changeable.

[0139] Furthermore, a real object movement recorded with a second image sensor is disadvantageously subject to a limitation with regard to the viewing direction specified by the second image sensor and the adjustment of the viewing directions required to achieve sufficient accuracy.

[0140] The person skilled in the art can record a real object movement with several second image sensors in order to create several viewing directions of the moving object specified by the second image sensors.

[0141] The movement data as in the Figures 2 to 4 shown as an example, all describe an object movement of a person 1 - as in the Figures 2 to 4by way of example and thus not restrictively - or equivalently for the inventive method of an object as image data. The object movement is further defined by a three-dimensional coordinate system, whereby the person skilled in the art can also use other coordinate systems or other forms of sufficient definition of the object movement. At most, the person skilled in the art applies a sufficient definition of the object movement so that this object movement can be incorporated into the Figure 1 presented model can be integrated.

[0142] Through the object movement data, which is stored in the Figures 2 to 4 When describing the exemplary object movement in a room 4, it is not defined whether the respective object movement represents an extraordinary situation or an ordinary situation.

[0143] It is known in the art to create image sensor model data that describe a sensor's effective range with sufficient accuracy. It is also known in the art to create object motion data according to the above description, which object motion data describe a person and / or an object as well as a movement of that person or object with sufficient accuracy.

[0144] The method according to the invention offers a technical solution for testing an alarm system, which technical solution is based on merging the screen model data and the object movement data and thereby creating simulation data. The simulation data has sufficient accuracy so that the alarm system, by processing the simulation data, can classify the situation described by the merged data. This implies that the simulation data is forwarded to the alarm unit and received by the alarm unit.

[0145] The sufficient accuracy of the simulation data is due to the fact that the image sensor model data and the object motion data have sufficient accuracy as described above.

[0146] The Figure 5illustrates a situation described by the simulation data, wherein the simulation data includes a sufficiently accurate description of the area with building 11 and the moving objects. As explained above, the image is sufficiently accurate so that an alarm unit receiving the simulation data can classify the situation described by the simulation data using known methods.

[0147] The Figure 5 includes the image of Building 11, as this Building 11 in Figure 1 without persons 1, 1', 1" and in the figure description to Figure 1 is explained.

[0148] The image sensor model data includes a realistic image of the building, taking into account any geometric distortions. The building essentially represents the area to be monitored with the first image sensor.

[0149] The image of the area or building is such that the proportions of the area are correctly represented. Figure 5 The illustrated embodiment of the method according to the invention shows the Figure 5 the area to be monitored with the first image sensor, which area includes building 11 with window 12 and, for example and thus not restrictively, benches 13. The relative sizes of building 11, window 12, and benches 13 correspond to reality, whereby perspective distortion depending on the image sensor properties must be taken into account.

[0150] With the aid of the method according to the invention, a situation is simulated for forwarding to an alarm unit, as described within the scope of the disclosure, wherein a situation virtually recorded by the first image sensor is observed. Since image sensors according to the prior art are also used to record situations in areas with only low illumination and the image sensors can only provide image data with grayscale when used in this way, the simulation data can be limited to grayscale, as can be created by a thermal imaging camera. For this reason, the requirement for sufficient accuracy of the simulation data can be limited to the description of the area and the objects in the correct size proportions.

[0151] The image of the area can further take into account the properties of the building 11 with the windows 12 and the benches 13. The building 11, the windows 12, and the benches 13 can have the colors in the image sensor model data that the aforementioned objects 11, 12, 13 have in reality. The surfaces of the aforementioned objects 11, 12, 13 can have the surface properties that the aforementioned objects 11, 12, 13 have in reality. The reflective properties, the temperature, and the thermal behavior of the respective surfaces are mentioned here as examples and thus not restrictive.

[0152] The technical object of the invention discussed here is to create simulation data from image sensor model data and object movement data. The simulation data describe a situation in the area as image data; when the simulation data is forwarded to an alarm unit as part of the method according to the invention, the situation can be classified as a normal situation (no alarm situation) or as an exceptional situation (alarm situation). For this purpose, methods known from the prior art, such as object detection and / or tracking, can be applied to the simulation data to classify the situation.

[0153] In a manner analogous to how the state-of-the-art object detection and / or tracking methods are applied directly to image sensor data to classify a situation captured by the image sensor, these object detection and / or tracking methods are applied to the simulation data. This may require that the image sensor data describe the virtual situation sufficiently accurately so that these methods can be applied.

[0154] The simulation data can describe a situation at a point in time.

[0155] The simulation data can describe a situation that lasts over a period of time.

[0156] Simulation data is created from the image sensor model data and the object motion data. The simulation data includes image data.

[0157] To achieve the required sufficient accuracy, it is necessary that the object described by the object motion data is embedded in the image sensor model according to the reality to be imaged.

[0158] The object can be embedded into the image sensor model, taking into account its size and possible geometric distortion. This is a task that a person skilled in the art can accomplish without inventive effort and by applying common teachings, in particular the common teachings of descriptive geometry.

[0159] The object can be embedded in the image sensor model, taking the exposure conditions into account. This is also a technical task that a specialist can accomplish using common theory, particularly descriptive geometry.

[0160] In the Figure 5In the example shown, the simulation data includes object movement data that describe the persons 1, 1', 1". The persons 1, 1', 1" are selected from the multitude of Figure 5 The registered persons are selected as examples and therefore not restrictive.

[0161] Person 1 is embedded in the situation at the first location 2 in the building 11; embedding in the situation is performed by merging the image sensor model data and the object movement data describing person 1 and the movement of person 1. By arranging person 1 at the first location 2, a situation is created which can be classified as an ordinary situation or as an extraordinary situation by the downstream alarm situation.

[0162] The arrangement is such that the second location 3 of person 1 (as shown in Figure 2 shown) and the first location 2 of person 1 in the Figure 5illustrated image data are located at the same location.

[0163] Person 1 can follow a movement path 6. The alarm unit can, taking into account the presence of person 1 at the first location 1 and / or the movement of person 1, classify the situation defined by the image sensor model and / or person 1 and / or by the movement path as an extraordinary situation or a usual situation using common theory such as, for example, and thus exclusively, object recognition. The movement of person 1 as an entire object can be defined by the movement path 6. The movement of person 1 can further be defined by partial movements of person 1.

[0164] The Figure 5 shows Person 1 as a person climbing on the facade of Building 11. The alarm unit can classify this situation as an unusual situation, regardless of the other object movements.

[0165] Person 1' is embedded at another first location 2' in the area. The second location 3' of person 1' and the first location 2' of person 1' are at the same location. Embedding person 1' at the first location 2' and, if applicable, the movement of person 1' creates a situation that can be classified by the alarm unit as described above. The movement of person 1' can be defined by the movement path 6'.

[0166] The person 1' and the movement path 6' of the person 1' are defined as described in the figure description. Figure 3 is set out. The Figure 5includes an object in the vicinity of person 1', which object defines a restricted zone 8. The free zone extends around the object defining restricted zone 8. A further restricted zone can be defined, for example, by building 11 or by further persons or by further objects. Within the scope of the disclosure of the method according to the invention, it is assumed that a normal situation is simulated for person 1' by the image sensor model data and the object movement data.

[0167] The person 1" is embedded at a further first location 2" in the area. The second location 3" of the person 1" and the first location 2" of the person 1" are at the same location. Again, by embedding the person 1" at the first location 2" and, if applicable, by the movement of the person 1", a situation is created, which situation can be classified by the alarm unit as described above. The movement of the person 1" can be defined by the movement path 6". Within the scope of the disclosure of the method according to the invention, it is assumed that a normal situation is simulated by the image sensor model data and the object movement data for the person 1" standing near a bench 13.

[0168] The person 1‴ is embedded at a further first location 2‴ in the area. The second location 3‴ of the person 1‴ and the first location 2‴ of the person 1‴ are at the same location. By embedding the person 1‴ at the first location 2‴ and possibly by the movement of the person 1‴, a situation is created which can be classified by the alarm unit as described above. The movement of the person 1‴ can be defined by the movement path 6‴. The embedding of the person 1‴ in the Figure 5 The situation shown corresponds to the character description of Figure 4 . Within the scope of the disclosure of the method according to the invention, it is assumed that a normal situation is simulated by the image sensor model data and the object movement data for the running person 1‴.

[0169] The Figure 5The situation shown therefore includes ordinary situations and extraordinary situations. The simulation data can represent the individual situations combined into one situation (as in Figure 5 shown). The simulation data in this application includes image sensor model data and several object motion data.

[0170] The person skilled in the art will recognize that the simulation data, the image sensor model data and object motion data can describe a single object and a single object motion, which in Figure 5 is not shown. The user of the method according to the invention can thus create situations which the user wishes to classify with the alarm unit.

[0171] The Figure 6 shows the view of the Figure 1 , in which view the ground of the area is covered with a layer of snow. Just like the Figure 1 illustrates the Figure 6an image sensor model, which image sensor model is included in the simulation data in the form of image sensor model data.

[0172] A possible application of the method according to the invention may be that the user wishes to test, by way of example and thus not by way of limitation, the functionality of an alarm system for monitoring an area without snow and an area with snow. The user can generate initial simulation data using a combination of image sensor model data according to Figure 1 with the object motion data and second simulation data combining image sensor model data according to Figure 6 with the object movement data and forward the first simulation data and the second simulation data to the alarm unit.

[0173] The Figures 7 and 8show a different allocation of the building 11 with a shadow 15, whereby the allocation of the building 11 with a shadow 15 is a problem which the person skilled in the art can solve using the current teaching, in particular the teaching of descriptive geometry, and which requires no further explanation within the scope of the disclosure of the invention. Figures 7 and 8 show the shadow of tree 26 on building 11.

[0174] A possible application of the method according to the invention may be that the user wishes to test, by way of example and thus not by way of limitation, the functionality of an alarm system for monitoring an area under different shadow conditions. The user can generate initial simulation data using a combination of image sensor model data according to Figure 7 with the object motion data and second simulation data combining image sensor model data according to Figure 8with the object movement data and forward the first simulation data and the second simulation data to the alarm unit.

[0175] By way of example and thus by no means limiting, the shadow formation on building 11 can be considered an environmental influence. The simulation data can be modified to include virtual environmental data, or the simulation data can be created to include virtual environmental data, which environmental data describe an environmental influence on the sensor's effective range. Figure 9(a)illustrates the functionality of a first image sensor 16 and an alarm unit 17 according to the prior art. The first image sensor 16 is arranged with a first sensor effective range at a first location 3. The first image sensor 16 delivers first image data 18 to the alarm unit 17, which classifies the first image data 18 with regard to the occurrence of an unusual situation or a normal situation. The first image data 18 describe the respective situation sufficiently precisely that prior art methods, such as pattern recognition, can be applied to classify the situation.

[0176] The Figure 9(b)illustrates an embodiment of the method according to the invention. The first image data 18 recorded by the first image sensor 16 are not sent to the alarm unit 17, but to a computer unit 19 for carrying out the method according to the invention. In the computer unit 19, simulation data 20 comprising the first image data 18 created by the first image sensor 16 and object movement data are created. The simulation data 20 are forwarded to the alarm unit 17. The alarm unit 17 classifies the situation described by the simulation data 20 as a normal situation, an exceptional situation, or an unclassifiable situation, wherein prior art methods are applied to the simulation data 20 for classifying the simulation data 20.The alarm unit 17 can, with reference to the above description, assess the situation using pattern recognition or by applying methods based on artificial intelligence. The simulation data 20 is image data for this purpose.

Claims

1. A computer-implemented method of determining whether an alert system recognises an unusual situation as such and, if required, triggers an alert signal, which first image sensor is a virtual sensor and has a virtual sensor range, and which sensor range comprises a first location, wherein image sensor model data is generated from input data such as maps of an area, which image sensor model data describes at least the virtual sensor range of the first image sensor in the shape of a three-dimensional model, wherein simulation data as image data comprising the image sensor model data and object movement data of a person (1) as an object is generated in a calculating unit, wherein the image sensor model data is devoid of the moved person (1), wherein the object movement data continuing for a timespan comprises image data generated at a second time, of a real movement of the person (1) at a second location determined using a real second image sensor or a virtual movement of the person at a virtual second location, characterised in that, the object movement data describes the movement of the person (1) by individual movements of selected subareas of said person (1), such as their arms, legs, head, wherein the simulation data describing either a common event or an unusual event is forwarded to an alert unit processing image data, wherein, in the alert unit, the situation described by the simulation data comprising the image sensor model data and the object movement data and taking place in the sensor range is classified either as a common event or as an unusual event by using pattern recognition methods and / or object recognition methods, wherein the object movement data is classified in consideration of the individual movements of the person's subareas, which alert system issues an alert signal in the case of an unusual event, allowing verification based on the alert signal issued as to whether an alert system triggers an alert signal in the case of simulation data which comprises an unusual event.

2. A computer-implemented method of determining whether an alert system recognises an unusual situation as such and, if required, triggers an alert signal, which first image sensor is a real image sensor and has a real sensor range, and which first image sensor issues image sensor data captured in reality as image data of the sensor range at a first time, which sensor range comprises a first location, wherein simulation data as image data comprising the image sensor data and virtual object movement data of a person (1) as an object is generated in a calculating unit, wherein the image sensor data is devoid of the moved person (1), wherein the object movement data continuing for a timespan comprises image data generated at a second time using a second image sensor, of a real movement of the person (1) at a second location determined using a second image sensor or a virtual movement of the person (1) at a virtual second location, characterised in that, the object movement data describes the movement of the person (1) by individual movements of selected subareas of said person (1), such as their arms, legs or head, wherein the simulation data representing either a common event or an unusual event is forwarded to an alert unit processing image data, wherein, in the alert unit, a situation described by the simulation data comprising the image sensor model data and the object movement data is classified either as a common event or as an unusual event by using pattern recognition methods and / or object recognition methods, wherein the object movement data is classified in consideration of the individual movements of the person's subareas, which alert system issues an alert signal in the case of an unusual event, allowing verification based on the alert signal issued as to whether an alert system triggers an alert signal in the case of simulation data which comprises an unusual event.

3. The method of claim 1, characterised in that a single image sensor model is used to generate a plurality of different simulation data, which simulation data comprises different object movement data and / or which simulation data comprises different image sensor model data.

4. The method of any one of claims 1 to 3, characterised in that the timespan of the object movement data included in the method is different from a real timespan of the object movement.