Automatic monitoring method
The automatic surveillance process optimizes computational efficiency by building panoramic banners with less resource-intensive methods and using instance-based object detection at specific points, enabling deployment on simpler sensors and targets.
Patent Information
- Application Number
- FR2023001954
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-03-03
- Publication Date
- 2025-04-25
- Estimated Expiration
- 2043-03-03
AI Technical Summary
Existing surveillance processes using panoramic headbands require significant computational resources and are limited to being implemented on powerful electronic devices, making them unsuitable for on-board applications.
An automatic surveillance process that acquires images from a sensor sweeping a scene, builds a panoramic banner using less resource-intensive methods, and employs instance-based object detection algorithms at specific crossing points, reducing computational demands by comparing current images with reference images.
This approach reduces the computational resources needed, allowing implementation on simpler sensors and targets, while maintaining effective object detection and change monitoring without the need for continuous panoramic banner updates.
Smart Images
Figure 00000010_0000 
Figure 00000011_0000 
Figure 00000012_0000
Abstract
Description
Title of the invention: Automatic monitoring method Technical field of the invention
[0001] The invention relates to an automatic surveillance method, in particular intended to be embedded in terrestrial sights or for any other system incorporating cameras or sensors capable of monitoring a given scene. State of the prior art
[0002] Sensor monitoring methods are known based on panoramic strips of the observed scene. For this, a sensor scans the scene at a configurable rotation speed and over a configurable angular sector and acquires images associated with line-of-sight information from the sensor. These acquired images are then processed in real time in two stages.
[0003] First, the images are projected into a panoramic reference frame. Projecting the images into the panoramic reference frame is in itself a computationally expensive step because the projection must be of high quality to be used by the detection algorithms mentioned later. It thus requires the implementation of numerous functions, such as: a complex geometric transformation of the homography type, a rolling shutter correction, a correction of the sensor orientations, a correction of the distortions, an estimation of the movement for the registration of the current image with the other images, etc.
[0004] Second, the images are processed by at least two image processors.
[0005] On the one hand, a first processing of panoramic strip reconstruction images is applied to reconstruct a panoramic strip and optimize its final quality. Thus, many additional functions costing computational resources are applied to the projected image so that the final panoramic strip is of good quality. Such functions can be a correction of motion blur, an improvement of the contrast and an operation allowing to smooth the differences in brightness and contrast between the images used to construct the strip.
[0006] On the other hand, a second object detection image processing between the current projected image and a set of reference strips is applied to detect the objects of interest and the changes occurring over the entire sector observed by the sensor, image by image. This detection adds computational complexity. To function optimally, object detection requires that the projection of the current image into the panoramic reference frame be carried out optimally and that it be of high quality.
[0007] Existing monitoring methods therefore require a huge amount of computing resources to operate and can only be implemented on powerful electronic processing cards. They can therefore only be difficult to embed in terrestrial sights. The invention aims to overcome these drawbacks. Summary of the invention
[0008] To this end, a method for automatically monitoring a scene is proposed comprising the steps of:
[0009] a) acquiring a plurality of images by a sensor configured to scan said scene,
[0010] b) constructing a panoramic strip of the scene from said images,
[0011] c) determining a monitoring trajectory from a plurality of points of passage in said scene, and creating a plurality of instances of an information extraction algorithm for the defined passage points, each of said algorithm instances has reference images, specific to each of the passage points (202) and centered on said passage point, and
[0012] d) repeating a step consisting of traversing the monitoring trajectory by said sensor, and at each crossing point traversed, acquiring at least one current image continuously, and extracting information from said current image, from the instance of the information extraction algorithm, by comparing the current image with said set of reference images associated with said crossing point
[0013] Thus, the detection of information from the scene is based on images and not on a panoramic strip as in the state of the art. This makes it possible to reduce the resources required to implement the method. In particular, the method makes it possible to do without complex and computationally expensive image processing. The method can be implemented simply in existing sensors or viewfinders. Unlike the panoramic strip of the state of the art, the method according to the invention makes it possible to construct the panoramic strip only with inexpensive processing.
[0014] The method may comprise creating N instances of an object detection algorithm on the N defined waypoints. Each instance may have reference images, which are specific to each of the N waypoints and centered on said point.
[0015] The method may comprise traversing the monitoring trajectory, by marking a stop at each of the N passage points, by said sensor, and the acquisition of the current images continuously.
[0016] The method may comprise, for each stop on a point, a call to an instance of the object detection algorithm linked to said point. The method may comprise the initialization of said instance, for example only once during the first passage over the point, then the comparison of the current image with a set of images of reference. The method may then include updating the reference images and finally highlighting the detected objects and / or recovering / transmitting the detection information for these objects.
[0017] The sensor may be a viewfinder comprising one or more cameras and means for actuating the camera(s) of the sensor along a line of sight. The line of sight may correspond to the horizontal and vertical orientation of the sensor.
[0018] The sensor may be any type of optronic sensor. The sensor may comprise at least one visible spectrum camera and / or at least one infrared camera, or any other camera. Said cameras may be oriented in parallel.
[0019] The sensor may comprise means for processing the data acquired by said cameras.
[0020] Furthermore, the panoramic strip of the scene can be constructed once at the start of the method and does not need to be updated at each iteration of step d). This makes it possible to further reduce the computational resources for the implementation of the method.
[0021] Step b) of constructing the panoramic strip can be carried out by juxtaposing the images acquired in step a). Prior to this juxtaposition, the images acquired in step a) can be processed by at least one operation to zoom out said current image and / or at least one translation operation and / or at least one operation making it possible to smooth out the differences in brightness and contrast between the images used to construct the strip. Thus, the panoramic strip reconstruction function is therefore much less expensive in terms of computational resources compared to the methods of the state of the art.
[0022] Step d) may comprise, for each passage point, the extraction of information from the current image as a function of the reference image.
[0023] The information may relate to the presence or absence of an object in the image, for example in relation to the last image of the previous scan, or the tracking of the movement of an object in the image, the extraction of characteristics of an object in the image such as its position, its dimensions, its speed and possibly its identification, etc.
[0024] The extraction of information can be carried out by any type of algorithm for detecting objects or elements in the scene, for example by a moving object detection algorithm, a change detection algorithm and / or any algorithm which uses temporal learning on the whole image or on a part.
[0025] Step d) may comprise the detection of the appearance or disappearance of objects from the scene between two successive iterations of the path of the surveillance trajectory by the sensor. This detection may comprise the comparison of said at least one current image with the reference image of the point of passage that the sensor travels. in particular, the reference image may include data from the image acquired during the previous scan of said waypoint by the sensor.
[0026] This makes it possible to detect changes that occurred in the scene when the sensor was pointed towards another area of the scene.
[0027] The method may comprise a step consisting of updating, for each passage point, the reference image with the successive current images acquired at each iteration of step d).
[0028] In particular, the reference image associated with a waypoint may comprise an image at one or more resolution scales and statistical data learned on the scene at all resolution scales and from the successive current images taken of said waypoint.
[0029] According to one embodiment, step a) can be carried out by scanning the scene by the sensor at a rotation speed following an angular sector. For example, the rotation speed and the angular sector can be defined by a user, or even calculated automatically by the algorithm.
[0030] At least one waypoint of the scene may correspond to a point of interest in the landscape such as a building, a crossroads, etc.
[0031] The waypoints can be selected by the user, in particular by selecting points on the panoramic strip.
[0032] The waypoints can be determined automatically by automatic analysis of the scene by a dedicated algorithm, for example based on artificial intelligence.
[0033] The number N of waypoints can for example be between 2 and 20 but can also be greater than 20.
[0034] The method may comprise an exposure time of the sensor during the travel of the monitoring trajectory for the acquisition of said at least one current image, at each passage point.
[0035] The method may comprise displaying the panoramic banner, said at least one current image, the monitoring trajectory and / or any data extracted from the current image. The method may comprise displaying the line of sight data and / or the sensor movement speed.
[0036] The invention further relates to a device for automatically monitoring a scene, comprising a sensor mounted on actuators configured to move the sensor and a processing circuit for implementing the aforementioned method.
[0037] The sensor may be a viewfinder comprising one or more cameras and means for actuating the camera(s) of the sensor along a line of sight. The sensor may comprise at least one visible spectrum camera and / or at least one infrared camera, or any other camera.
[0038] The invention also relates to a computer program comprising instructions for implementing the aforementioned method, when said instructions are executed by a processor of a processing circuit. Brief description of the figures
[0039] [Fig. 1] represents a diagram of an exemplary embodiment of the method according to the invention,
[0040] [Fig.2] represents a first example of display resulting from the method of [Fig. 1],
[0041] [Fig.3] represents a second example of display resulting from the method of [Fig.l],
[0042] [Fig.4] represents another example of an explanatory display resulting from the method of the [Fig.l]. Detailed description of the invention
[0043] With reference to [Fig.l], the method 100 of automatic monitoring of a scene by a sensor.
[0044] The sensor is in particular a viewfinder, for example a terrestrial viewfinder, comprising several cameras and actuators capable of moving the cameras of the sensor horizontally and vertically along a line of sight 102. Alternatively, the sensor can be any type of optronic sensor.
[0045] The sensor comprises visible spectrum cameras, infrared cameras, or any other cameras arranged in parallel or not.
[0046] The sensor further comprises processing means capable of implementing the method 100.
[0047] The method 100 comprises a first phase of constructing a panoramic strip 108 visible in detail in Figures 2 and 3. This first phase comprises a step of scanning the scene along a predetermined angular sector and at a given speed. During this scanning, the method 100 comprises the acquisition of data relating to the line of sight 102 of the sensor, that is to say its horizontal and vertical orientations, and the acquisition of images 110 of the sensor at a plurality of successive instants.
[0048] The method 100 comprises a step 104 of projection and processing of the acquired images consisting at least of zooming out operations 104a, i.e. changing the size of the image, and translation of the images 104b.
[0049] The method 100 then comprises a step 106 of reconstructing the panoramic strip 108 by applying an operation 106a making it possible to smooth the differences in brightness and contrast between the images, used to construct the strip, to the images projected and processed in the previous step 104 and by inserting 106b the images into the panoramic strip 108.
[0050] This constructed panoramic strip 108 is stored in memory for the rest of the method 100.
[0051] The method 100 comprises a step of selecting a plurality of waypoints 202, for example N waypoints 202 (where N is an integer), in the panoramic strip 108. The waypoints 202 are defined by the operator. The number N of waypoints can for example be between 2 and 20.
[0052] According to one embodiment, the waypoints 202 can be determined, or suggested for validation to the operator, automatically by processing the scene of the panoramic strip 108, for example by methods based on artificial intelligence.
[0053] The waypoints 202 can be points in the landscape.
[0054] In a first step, N instances of an object detection algorithm 114 are created in connection with the N defined waypoints. Each instance is associated with reference images 112, which are specific to each of the waypoints and centered on said waypoint.
[0055] The waypoints 202 define a monitoring trajectory 204.
[0056] The method 100 then comprises the repetitive traversal, throughout the monitoring duration, of the monitoring trajectory 204 by the sensor. This consists of traversing the monitoring trajectory 204, stopping at each of the N waypoints 202, by said sensor, and acquiring the current images continuously.
[0057] For each stop on a point 202, the instance of the object detection algorithm 114 linked to said point 202 is called. It is initialized, for example only once during the first passage over the waypoint. Then, the current image 110, having said waypoint 202 as its optical center, is compared with a set of reference images 112 and then the reference images 112 of said waypoint are updated. Finally, the detected objects are highlighted and / or the detection information of these objects 1000 is recovered / transmitted.
[0058] For example, a moving object detection process is applied to the current image 110. This makes it possible to detect the objects 206-1, 206-2 and 206-3 which are moving in the zone 122 corresponding to the visual field of the sensor.
[0059] The processing for detecting moving objects can be replaced by any other type of processing suitable for extracting useful information from the images. And in particular, by algorithms for detecting objects or elements in the scene, such as for example a change detection algorithm or any algorithm which uses temporal learning on the whole image or on a part.
[0060] The reference images 112, linked to each of the N waypoints, some of which are framed in [Fig. 4], are regularly updated as a function of the successive current images 110 acquired. For example, the reference images 112 are updated at each instant t of the course of the monitoring trajectory 204. The reference image 112 associated with a waypoint 202 comprises an image at one or more resolution scales and statistical data learned on the scene at all resolution scales and from the successive current images 110 taken from said waypoint 202.
[0061] Thus, the method 100 maintains in memory N instances of an object detection algorithm 114 on the N defined waypoints. Each instance will have its reference images 112, which will also be maintained.
[0062] During the course of the monitoring trajectory, the optical field of the sensor may be modified. In this case, the N instances of the object detection algorithm 114 will be reset, as well as their associated reference images 112.
[0063] Figures 2 and 3 represent examples of displays that can be produced during the execution of the method 100. Such images can be displayed on a screen equipping the sensor or on a screen remote and separate from the sensor. The current image 110 displayed in the upper part corresponds to the image captured by the sensor when it is aligned along a line of sight corresponding to the zone 122 of the panoramic strip 108. In this current image 110, the detections 1000 are also displayed / inlaid. Other information can be displayed to the operator such as the data of the line of sight and / or the speed of movement of the sensor.
[0064] The method 100 further comprises the detection of the appearance or disappearance of objects from the scene between two successive iterations of the path of the monitoring trajectory 204 by the sensor, via the processing 114. For example, the current image(s) 110 are compared to the reference image 112 of the waypoint 202 that the sensor displays. In particular, the reference image 112 comprises data from the image acquired during the previous scanning of said waypoint 202 by the sensor.
[0065] This makes it possible to detect changes that occurred in the scene when the sensor was pointed towards another area of the scene.
[0066] According to one embodiment, the panoramic banner 108 is not updated during the steps of traveling the trajectory and the successive steps 114. Alternatively, the panoramic banner 108 can be updated, for example to update its content or to extend or reduce the surveillance zone.
Claims
Claims
1. A method (100) for automatically monitoring a scene comprising the steps of: a) acquiring a plurality of images (110) by a sensor configured to scan said scene, b) constructing a panoramic strip (108) of the scene from said images, c) determining a monitoring trajectory (204) from a plurality of waypoints (202) in said scene, and creating a plurality of instances of an information extraction algorithm for the defined waypoints, each of said algorithm instances has reference images (112), specific to each of the waypoints (202) and centered on said waypoint (202), and d) repeating a step of traversing the monitoring trajectory by said sensor, and at each waypoint (202) traversed, acquiring at least one current image continuously and extracting information from said current image, from said instance of the information extraction algorithm,by comparing the current image with said set of reference images associated with said waypoint (202).,
2. Method (100) according to the preceding claim, wherein step d) comprises applying an object detection algorithm to said at least one current image (110).
3. Method (100) according to one of the preceding claims, in which step d) comprises detecting the appearance or disappearance of objects from the scene between two successive iterations of the course of the monitoring trajectory (204) by the sensor.
4. Method (100) according to one of the preceding claims, comprising a step consisting of updating, for each passage point (202), the reference image (112) with the successive current images (110) acquired at each iteration of step d).
5. Method (100) according to one of the preceding claims, comprising a step of processing at least one of the current images (110) by at least one operation for zooming out said current image and / or at least one translation operation.
6. Method (100) according to one of the preceding claims, in which step a) is carried out by scanning the scene by the sensor at a speed of rotation along an angular sector.
7. Method (100) according to one of the preceding claims, in which at least one passage point (202) of the scene corresponds to a point of the landscape.
8. Device for automatic monitoring of a scene, comprising a sensor mounted on actuators configured to move the sensor and a processing circuit for implementing the method (100) according to one of the preceding claims.
9. Computer program comprising instructions for implementing the method (100) according to one of claims 1 to 7, when said instructions are executed by a processor of a processing circuit.