Automatic monitoring method

Abstract

The invention relates to a method (100) for automatically monitoring a scene, comprising the steps of: a) acquiring a plurality of images by a sensor configured to scan said scene, b) constructing a panoramic strip (108) of the scene from said images, c) determining a monitoring trajectory from a plurality of waypoints in said scene, and d) repeating a step of following the monitoring trajectory by said sensor, and at each waypoint passed, acquiring at least one current image (110) and extracting information from said current image.

Description

[0001] DESCRIPTION

[0002] TITLE: Automatic monitoring process

[0003] Technical field of the invention

[0004] The invention relates to an automatic surveillance method, in particular intended to be embedded in terrestrial sights or for any other system incorporating cameras or sensors capable of monitoring a given scene.

[0005] State of the prior art

[0006] Sensor monitoring methods are known based on panoramic strips of the observed scene. To do this, a sensor scans the scene at a configurable rotation speed and over a configurable angular sector and acquires images associated with sensor line-of-sight information. These acquired images are then processed in real time in two stages.

[0007] First, the images are projected into a panoramic frame. Projecting the images into the panoramic frame is itself a computationally expensive step because the projection must be of high quality to be used by the detection algorithms mentioned later. It thus requires the implementation of many functions, such as: a complex geometric transformation of homography type, a rolling shutter correction, a correction of the sensor orientations, a correction of distortions, an estimation of the movement for the registration of the current image with the other images, etc.

[0008] Second, the images are processed by at least two image processors.

[0009] On the one hand, a first panoramic strip reconstruction image processing is applied to reconstruct a panoramic strip and optimize its final quality. Thus, many additional computationally expensive functions are applied to the projected image so that the final panoramic strip is of good quality. Such functions can be motion blur correction, contrast enhancement and an operation to smooth out the differences in brightness and contrast between the images used to construct the strip.

[0010] On the other hand, a second object detection image processing between the current projected image and a set of reference strips is applied to detect objects of interest and changes occurring over the entire sector observed by the sensor, image by image. This detection adds computational complexity. To function optimally, object detection requires that the projection of the current image into the panoramic reference frame be carried out optimally and that it be of high quality. Existing monitoring methods therefore require a lot of computational resources to operate and are only implementable on powerful electronic processing cards. They can therefore only be difficult to embed in terrestrial sights. The invention aims to overcome these drawbacks.

[0011] Summary of the invention

[0012] For this purpose, a method for automatically monitoring a scene is proposed comprising the steps of: a) acquiring a plurality of images by a sensor configured to scan said scene, b) constructing a panoramic strip of the scene from said images, c) determining a monitoring trajectory from a plurality of waypoints in said scene, and creating a plurality of instances of an information extraction algorithm for the defined waypoints, each of said algorithm instances has reference images, specific to each of the waypoints (202) and centered on said waypoint, and d) repeating a step of traversing the monitoring trajectory by said sensor, and at each waypoint traversed, acquiring at least one current image continuously, and extracting information from said current image, from the instance of the information extraction algorithm,by comparing the current image with said set of reference images associated with said waypoint,

[0013] Thus, the detection of information from the scene is based on images and not on a panoramic strip as in the state of the art. This makes it possible to reduce the resources required to implement the method. In particular, the method makes it possible to do without complex and computationally expensive image processing. The method can be implemented simply in existing sensors or viewfinders. Unlike the state-of-the-art panoramic strip, the method according to the invention makes it possible to construct the panoramic strip only with inexpensive processing.

[0014] The method may comprise creating N instances of an object detection algorithm on the N defined waypoints. Each instance may have reference images, which are specific to each of the N waypoints and centered on said point.

[0015] The method may comprise traversing the monitoring trajectory, stopping at each of the N passage points, by said sensor, and acquiring the current images continuously.

[0016] The method may comprise, for each stop on a point, a call to an instance of the object detection algorithm linked to said point. The method may comprise the initialization of said instance, for example once only during the first passage over the point, then the comparison of the current image with a set of reference images. The method may then comprise the updating of the reference images and finally the highlighting of the detected objects and / or the recovery / transmission of the detection information of these objects.

[0017] The sensor may be a viewfinder comprising one or more cameras and means for actuating the camera(s) of the sensor along a line of sight. The line of sight may correspond to the horizontal and vertical orientation of the sensor.

[0018] The sensor may be any type of optronic sensor. The sensor may include at least one visible spectrum camera and / or at least one infrared camera, or any other camera. Said cameras may be oriented in parallel.

[0019] The sensor may include means for processing the data acquired by said cameras.

[0020] Furthermore, the panoramic strip of the scene can be constructed once at the start of the method and does not need to be updated at each iteration of step d). This makes it possible to further reduce the computer resources for implementing the method. Step b) of constructing the panoramic strip can be carried out by juxtaposing the images acquired in step a). Prior to this juxtaposition, the images acquired in step a) can be processed by at least one operation to zoom out said current image and / or at least one translation operation and / or at least one operation making it possible to smooth out the differences in brightness and contrast between the images used to construct the strip. Thus, the panoramic strip reconstruction function is therefore much less expensive in terms of computer resources compared to the state-of-the-art methods.

[0021] Step d) may comprise, for each waypoint, extracting information from the current image as a function of the reference image.

[0022] The information may relate to the presence or absence of an object in the image, for example in relation to the last image of the previous scan, or the tracking of the movement of an object in the image, the extraction of characteristics of an object in the image such as its position, its dimensions, its speed and possibly its identification, etc.

[0023] Information extraction can be performed by any type of algorithm for detecting objects or elements in the scene, for example, by a moving object detection algorithm, a change detection algorithm, and / or any algorithm that uses temporal learning on the whole image or on a part.

[0024] Step d) may comprise the detection of the appearance or disappearance of objects from the scene between two successive iterations of the path of the monitoring trajectory by the sensor. This detection may comprise the comparison of said at least one current image with the reference image of the waypoint that the sensor travels. In particular, the reference image may comprise data from the image acquired during the previous scanning of said waypoint by the sensor.

[0025] This allows changes in the scene to be detected when the sensor was pointed at a different area of ​​the scene.

[0026] The method may comprise a step of updating, for each passage point, the reference image with the successive current images acquired at each iteration of step d).

[0027] In particular, the reference image associated with a waypoint may comprise an image at one or more resolution scales and statistical data learned about the scene at all resolution scales and from successive current images taken of said waypoint.

[0028] According to one embodiment, step a) can be carried out by scanning the scene by the sensor at a rotation speed following an angular sector. For example, the rotation speed and the angular sector can be defined by a user, or even calculated automatically by the algorithm.

[0029] At least one waypoint in the scene can correspond to a point of interest in the landscape such as a building, a crossroads, etc.

[0030] Waypoints can be selected by the user, in particular by selecting points on the panoramic strip.

[0031] Waypoints can be determined automatically by automatic analysis of the scene by a dedicated algorithm, for example based on artificial intelligence.

[0032] The number N of waypoints can for example be between 2 and 20 but can also be greater than 20.

[0033] The method may include an exposure time of the sensor during the travel of the monitoring trajectory for the acquisition of said at least one current image, at each passage point.

[0034] The method may include displaying the panoramic banner, the at least one current image, the monitoring trajectory and / or any data extracted from the current image. The method may include displaying the line-of-sight data and / or the sensor movement speed.

[0035] The invention further relates to a device for automatically monitoring a scene, comprising a sensor mounted on actuators configured to move the sensor and a processing circuit for implementing the aforementioned method.

[0036] The sensor may be a viewfinder comprising one or more cameras and means for actuating the camera(s) of the sensor along a line of sight. The sensor may comprise at least one visible spectrum camera and / or at least one infrared camera, or any other camera. The invention also relates to a computer program comprising instructions for implementing the aforementioned method, when said instructions are executed by a processor of a processing circuit.

[0037] Brief description of the figures

[0038] [Fig. 1] represents a diagram of an exemplary embodiment of the method according to the invention, [Fig. 2] represents a first example of display resulting from the method of figure 1, [Fig. 3] represents a second example of display resulting from the method of figure 1, [Fig. 4] represents another example of explanatory display resulting from the method of figure 1.

[0039] Detailed description of the invention

[0040] Referring to Figure 1, the method 100 for automatically monitoring a scene by a sensor.

[0041] The sensor is in particular a viewfinder, for example a terrestrial viewfinder, comprising several cameras and actuators capable of moving the cameras of the sensor horizontally and vertically along a line of sight 102. Alternatively, the sensor can be any type of optronic sensor.

[0042] The sensor includes visible spectrum cameras, infrared cameras, or any other cameras arranged in parallel or not.

[0043] The sensor further comprises processing means capable of implementing the method 100.

[0044] The method 100 comprises a first phase of constructing a panoramic strip 108 visible in detail in Figures 2 and 3. This first phase comprises a step of scanning the scene along a predetermined angular sector and at a given speed. During this scanning, the method 100 comprises the acquisition of data relating to the line of sight 102 of the sensor, that is to say its horizontal and vertical orientations, and the acquisition of images 110 of the sensor at a plurality of successive instants.

[0045] The method 100 comprises a step 104 of projection and processing of the acquired images consisting at least of zooming out operations 104a, i.e. changing the size of the image, and translation of the images 104b.

[0046] The method 100 then comprises a step 106 of reconstructing the panoramic strip 108 by applying an operation 106a making it possible to smooth the differences in brightness and contrast between the images, used to construct the strip, to the images projected and processed in the previous step 104 and by inserting 106b the images into the panoramic strip 108.

[0047] This constructed panoramic strip 108 is stored in memory for the rest of the method 100. The method 100 comprises a step of selecting a plurality of waypoints 202, for example N waypoints 202 (where N is an integer), in the panoramic strip 108. The waypoints 202 are defined by the operator. The number N of waypoints can for example be between 2 and 20.

[0048] According to one embodiment, the waypoints 202 can be determined, or suggested for validation to the operator, automatically by processing the scene of the panoramic banner 108, for example by methods based on artificial intelligence.

[0049] Waypoints 202 can be points in the landscape^

[0050] In a first step, N instances of an object detection algorithm 114 are created in connection with the N defined waypoints. Each instance is associated with reference images 112, which are specific to each of the waypoints and centered on said waypoint.

[0051] Waypoints 202 define a surveillance trajectory 204.

[0052] The method 100 then comprises the repetitive traversal, throughout the monitoring duration, of the monitoring trajectory 204 by the sensor. This consists of traversing the monitoring trajectory 204, stopping at each of the N waypoints 202, by said sensor, and acquiring the current images continuously.

[0053] For each stop on a point 202, the instance of the object detection algorithm 114 linked to said point 202 is called. It is initialized, for example only once during the first passage over the waypoint. Then, the current image 110, having said waypoint 202 as its optical center, is compared with a set of reference images 112 and then the reference images 112 of said waypoint are updated. Finally, the detected objects are highlighted and / or the detection information of these objects 1000 is recovered / transmitted.

[0054] For example, a moving object detection process is applied to the current image 110. This makes it possible to detect the objects 206-1, 206-2 and 206-3 which move in the zone 122 corresponding to the visual field of the sensor.

[0055] The moving object detection processing can be replaced by any other type of processing suitable for extracting useful information from images. In particular, by algorithms for detecting objects or elements in the scene, such as for example a change detection algorithm or any algorithm that uses temporal learning on the whole image or on a part.

[0056] The reference images 112, linked to each of the N waypoints, some of which are framed in FIG. 4, are regularly updated as a function of the successive current images 110 acquired. For example, the reference images 112 are updated at each instant t of the course of the monitoring trajectory 204. The reference image 112 associated with a waypoint 202 comprises an image at one or more resolution scales and statistical data learned on the scene at all the resolution scales and from the successive current images 110 taken from said waypoint 202.

[0057] Thus, the method 100 maintains in memory N instances of an object detection algorithm 114 on the N defined waypoints. Each instance will have its reference images 112, which will also be maintained.

[0058] During the traversal of the monitoring trajectory, the optical field of the sensor may be modified. In this case, the N instances of the object detection algorithm 114 will be reset, as well as their associated reference images 112.

[0059] Figures 2 and 3 represent display examples that can be produced during the execution of the method 100. Such images can be displayed on a screen equipping the sensor or on a screen remote and separate from the sensor. The current image 110 displayed in the upper part corresponds to the image captured by the sensor when it is aligned along a line of sight corresponding to the zone 122 of the panoramic strip 108. In this current image 110, the detections 1000 are also displayed / inlaid. Other information can be displayed to the operator such as the data of the line of sight and / or the speed of movement of the sensor.

[0060] The method 100 further comprises the detection of the appearance or disappearance of objects from the scene between two successive iterations of the path of the monitoring trajectory 204 by the sensor, via the processing 114. For example, the current image(s) 110 are compared to the reference image 112 of the waypoint 202 that the sensor displays. In particular, the reference image 112 comprises data from the image acquired during the previous scanning of said waypoint 202 by the sensor.

[0061] This allows changes in the scene to be detected when the sensor was pointed at a different area of ​​the scene.

[0062] According to one embodiment, the panoramic banner 108 is not updated during the steps of the trajectory and the successive steps 114. Alternatively, the panoramic banner 108 can be updated, for example to update its content or to extend or reduce the surveillance zone.

Claims

CLAIMS 1. A method (100) for automatically monitoring a scene comprising the steps of: a) acquiring a plurality of images (110) by a sensor configured to scan said scene, b) constructing a panoramic strip (108) of the scene from said images, c) determining a monitoring trajectory (204) from a plurality of waypoints (202) in said scene, and creating a plurality of instances of an information extraction algorithm for the defined waypoints, each of said algorithm instances has reference images (112), specific to each of the waypoints (202) and centered on said waypoint (202), and d) repeating a step of traversing the monitoring trajectory by said sensor, and at each waypoint (202) traversed, acquiring at least one current image continuously and extracting information from said current image, from said instance of the information extraction algorithm,by comparing the current image with said set of reference images associated with said waypoint (202)., 2. Method (100) according to the preceding claim, in which step d) comprises the application of an object detection algorithm to said at least one current image (110).

3. Method (100) according to one of the preceding claims, in which step d) comprises the detection of the appearance or disappearance of objects from the scene between two successive iterations of the course of the monitoring trajectory (204) by the sensor.

4. Method (100) according to one of the preceding claims, comprising a step consisting of updating, for each passage point (202), the reference image (112) with the successive current images (110) acquired at each iteration of step d).

5. Method (100) according to one of the preceding claims, comprising a step consisting of processing at least one of the current images (110) by at least one operation for zooming out said current image and / or at least one translation operation.

6. Method (100) according to one of the preceding claims, in which step a) is carried out by scanning the scene by the sensor at a rotation speed following an angular sector.

7. Method (100) according to one of the preceding claims, in which at least one passage point (202) of the scene corresponds to a point of the landscape.

8. Device for automatic monitoring of a scene, comprising a sensor mounted on actuators configured to move the sensor and a processing circuit for implementing the method (100) according to one of the preceding claims.

9. Computer program comprising instructions for implementing the method (100) according to one of claims 1 to 8, when said instructions are executed by a processor of a processing circuit.

Images