Method for generating at least one generalized detection of a broad source

Abstract

The invention relates to a method for generating at least one generalized detection of a large source present in an image, comprising the successive steps of: - from at least one elementary detection of said large source, generating a segmentation mask associated with said detection, - generating at least one segmentation region such that each generated segmentation region is defined to encompass: either a single segmentation mask, or several overlapping segmentation masks, - determining a generalized detection for each generated segmentation region. FIGURE IN ABRIDGED: Fig. 2

Description

Title of the invention: Method for generating at least one generalized detection of a wide source

[0001] The invention relates to a method for generating a generalized detection of a wide source present on an initial image by processing said image or for generating several generalized detections, each generalized detection relating to a distinct wide source on the initial image by processing said image.

[0002] BACKGROUND OF THE INVENTION

[0003] In the field of military naval operations, it is known to use systems for acquiring and processing at least one image to target objects such as, for example, surface ships.

[0004] [Fig.l] illustrates a possible architecture of an existing system. Such a system comprises different modules: a detection determination module 101, a detection fusion module 120 and a merged detection classification and / or tracking module 103.

[0005] In fact, the detection determination module 101 can comprise two blocks: - one 104 for large sources, also called extended source (that is to say that the source is represented by at least ten pixels on the image), and - the other 105 for so-called point sources (i.e. the source is represented by a few pixels at most on the image).

[0006] Each block 104, 105 can implement an algorithm which assumes that the background of the image is Gaussian and is therefore characterized by at least two parameters which are the “mean” and the “variance”. Thus the algorithm can evaluate the following criterion(s): - an average of the background surrounding an object on the image (the average of the background will be called “ImB” hereinafter) i.e. the average level of the pixels constituting the background surrounding an object; - a variance of the noise of said background (subsequently the standard deviation of the background will be noted “o B” and its variance “oB2”).

[0007] For example, the algorithm determines the average level of the pixels constituting an object (said average level will be called “ImT” hereinafter). Alternatively, the algorithm imposes ImT on the value of the central pixel of the object.

[0008] Then the algorithm estimates the signal-to-noise ratio (hereinafter called “SNR”) by the formula

[0009] SNR = (ImT - ImB) / oB

[0010] In case of management of positive contrasts

[0011] Or as a variant according to the formula (in case of management of negative contrasts):

[0012] SNR = IImT - ImBI / oB

[0013] From the SNR value and a nominal threshold Thl value, predetermined value from detection performance and / or tolerance of false alarms:

[0014] - if SNR is less than Thl then there is no detection,

[0015] - if SNR > Thl then there is detection and block 104, 105 provides the position as output of the detection and / or one or more characteristic information of the detection (morphology, entropy, etc.).

[0016] According to a variant, the algorithm is based on segmentation by Markov fields: the algorithm thus works on the image by successive iterations to obtain homogeneous zones modelable by Gaussians whose contours it then analyzes to obtain the detections.

[0017] Regardless of the algorithm used, a recurring problem is the poor analysis of wide sources. Thus, it often happens that the system ends up deducing from an image that there are several detections for the same wide source.

[0018] This of course hinders the proper counting of sources but also disrupts the processing of the information linked to the deductions, particularly when one wishes to track a particular source, which harms the performance of the whole.

[0019] SUBJECT OF THE INVENTION

[0020] An aim of the invention is to propose a method limiting a risk of multiple detections of the same large source present in an image. Summary of the invention

[0021] With a view to achieving this aim, the invention proposes a method for generating at least one generalized detection of a wide source present in an image, comprising the successive steps of: - from at least one elementary detection of said wide source, generate a segmentation mask associated with said detection, - generate at least one segmentation region such that each generated segmentation region is defined to encompass:

[0022] either a single segmentation mask,

[0023] or several segmentation masks which overlap with each other, - determine for each segmentation region generated a generalized detection.

[0024] In this way, the invention allows a fusion dedicated to wide sources.

[0025] When the invention is used in a system for acquiring and processing at least one image to target objects, the invention thus limits the risk of obtaining two targets for the same wide source.

[0026] The invention therefore makes it possible to improve the counting of targets. The invention also makes it possible to limit errors in the subsequent processing of the image and / or the target, for example at the level of a module for tracking said target.

[0027] More generally, the invention makes it possible to improve the detection of a target, the stability of tracking a target, the stability of counting targets, the stability of the classification and / or identification of targets, etc.

[0028] The invention thus proves to be relatively efficient. In particular, a device implementing the invention proves to be relatively efficient, particularly in terms of false alarms (or False Alarms).

[0029] Advantageously, the invention can be quickly and easily implemented in existing devices.

[0030] The invention is therefore inexpensive to implement.

[0031] Furthermore, the invention does not excessively increase the calculation time when implemented in an existing device.

[0032] For the present application, it is recalled that a wide source (or extended source) is a source represented by at least ten pixels on an image and which is thus opposed to the notion of a point source represented by a maximum of a few pixels on an image.

[0033] Optionally, a cumulative histogram is determined on a portion of the initial image centered on the position of the elementary detection of the wide source considered.

[0034] Optionally, a local detection threshold is determined based on gray levels of the pixels of said portion by thresholding the cumulative histogram by an intermediate threshold of predetermined value.

[0035] Optionally the intermediate threshold has a value between 80 and 90%.

[0036] Optionally the initial image is thresholded in gray levels by the local threshold of detection in order to obtain a first image presenting one or more segmentations associated with the elementary detection of a large source.

[0037] Optionally, the first image is dilated and / or eroded to obtain a second image.

[0038] Optionally, the first image or the second image is labeled to extract at least one segmentation mask.

[0039] Optionally, a new image is created including at least one segmentation mask and said image is labeled to extract the at least one segmentation region.

[0040] Optionally, a position of the at least one generalized detection of is calculated. the broad source associated with at least one segmentation region.

[0041] Optionally, it is considered that the position of the at least one generalized detection of the wide source is the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level.

[0042] The invention also relates to a device for implementing the aforementioned method.

[0043] The invention also relates to an optronic system comprising a device as mentioned above.

[0044] The invention also relates to a marine vehicle comprising an optronic system as mentioned above.

[0045] Other characteristics and advantages of the invention will emerge from reading the following description of a particular implementation of the invention. Brief description of the drawings

[0046] [Fig. 1] The figure has already been described in relation to the prior art of the invention.

[0047] Furthermore, the invention will be better understood in light of the description which follows with reference to the appended figures among which:

[0048] [Fig.2] [Fig.2] schematically illustrates in part a system allowing a setting in particular implementation of the invention, said system comprising an image processing device;

[0049] [Fig.3] [Fig.3] is a schematic representation of a first block of a third module of the image processing device illustrated in [Fig.2];

[0050] [Fig.4] [Fig.4] is a schematic representation of an image processed during a first step by the first block illustrated in [Fig.3];

[0051] [Fig.5] [Fig.5] is a schematic representation of a second block of the third module of the image processing device illustrated in [Fig.2];

[0052] [Fig.6] [Fig.6] is a schematic representation of a binary image generated by the second block shown in [Fig.5];

[0053] [Fig.7] [Fig.7] is a schematic representation of another binary image generated by the second block shown in [Fig.5]. DETAILED DESCRIPTION OF THE INVENTION

[0054] With reference to [Fig. 2], a system allowing a particular implementation of the invention will now be described. Such a system is for example an optronic system. It is recalled that an optronic system relies on both optics and electronics to fulfill its function(s).

[0055] In the present case, the optronic system comprises at least one image-taking element (optical sensor type) as well as a device for processing said images 1 in order to enable the detection of targets. Preferably, the system also comprises one or more modules arranged downstream of the image processing device 1 such as for example a module for classifying and / or identifying said targets, a module for tracking said targets, an electro-optical fire control system, etc. For example, the system comprises a target classification module 2.

[0056] Optionally, the images acquired by the image-taking element are infrared images.

[0057] Furthermore, the images acquired by the image-taking element have a single-mode background. Alternatively, the images acquired by the image-taking element have a multimodal (i.e. textured) background.

[0058] In the present case, the system is arranged on a marine vehicle (such as a surface ship or even a submarine).

[0059] The system is for example an optronic marine surveillance system.

[0060] The image processing device 1 is for example a calculator, a controller, a computer, etc. incorporating different modules which will be described later. Each module can for example be implemented by a reprogrammable logic circuit (such as an FPGA circuit) installed in the image processing device 1.

[0061] The image processing device 1 comprises a first module 3 for detecting point sources and a second module 4 for detecting elementary wide sources. Said sources are in this case surface objects such as surface ships and / or aircraft.

[0062] In a manner known per se, the function of the first module 3 is to segment each image in order to obtain detections of point sources and the function of the second module 4 is to segment each image in order to obtain elementary detections of wide sources. For example, each module 3, 4 detects the points of interest on each image by evaluating the SNR as described in the “Technological background of the invention” section. These points of interest correspond to “contrasting” points or even “salience” points, i.e. “hot” points if the image processing is based on a positive contrast or “cold” points if the image processing is based on a negative contrast.

[0063] The first module 3 and the second module 4 are well known from the prior art and will not be detailed further here.

[0064] It is noted that at the input of the first module 3 and the second module 4 there is an initial image acquired by the optical sensor and at the output of the first module 3 there are recovered detections of point sources and at the output of the second module 4 elementary detections of wide sources.

[0065] The image processing device 1 comprises a third module 5 for merging the elementary detections of wide sources. This third module 5 is therefore arranged in downstream of the second module 4 so as to have as input the elementary detections of large sources determined by the third module 4.

[0066] With reference to [Fig.3], the third module 5 comprises a first block 11 calculating, from said elementary detections of wide sources, segmentation masks associated with each of said detections.

[0067] Preferably, the first block 11 only retains the position of the detections of large sources (positions expressed in row-column) from the information transmitted by the second module 4.

[0068] Optionally the first block 11 performs the following loop.

[0069] Start of loop.

[0070] For each of the elementary wide source detections provided by the second module 4, the following different levels are implemented.

[0071] According to a first stage of a first step, the first block 11 determines a cumulative histogram on a portion of the initial image centered on the position of the elementary detection of the wide source considered. This portion of the initial image is of predetermined shape and dimensions identical to all the elementary detections of wide sources. For example, the portion of the image is a rectangle. [Fig. 4] represents such a portion of image. As for the cumulative histogram, it is a local representation of the distribution of the gray levels (also called "intensities") of the pixels around the position of elementary detection of the wide source considered. A cumulative histogram associated with an elementary detection of the wide source considered can therefore be totally different from that evaluated for another detection of a wide source, however belonging to the same wide source.

[0072] During the first stage, a local analysis of the gray levels is thus carried out around each elementary detection of the wide source considered extracted from the initial image.

[0073] Then, during a second stage of the first step, the first block 11 determines a local detection threshold Thp based on the gray levels of the pixels of the portion of the image by thresholding the cumulative histogram by an intermediate percentage threshold Th_% of predetermined value.

[0074] Optionally, we choose Th_% to have a value between 80 and 90%. For example, we choose Th_% to 85%. This means that we seek to keep only 15% of the pixels present in the portion of the image, independently of their positions in said portion. For each portion of the initial image associated with an elementary detection of a large source, the local detection threshold Thp depends on Th_% and the cumulative histogram: said local detection threshold Thp will therefore depend directly on the local neighborhood of the associated elementary detection of a large source and will be different from one portion to another.

[0075] During a second step, the initial image is in turn thresholded in gray levels (whereas at the level of the first module 3 and the second module 4 the initial image was thresholded in SNR) by the local detection threshold Thp.

[0076] We thus obtain a binary image I2p presenting one or more segmentations associated with the elementary detection of a wide source.

[0077] It is noted that the local detection level threshold Thp is specific to each elementary detection of a wide source so that the different binary images I2p each associated with one of the elementary detections of wide sources are different from each other.

[0078] A local segmentation is thus carried out around each of the elementary detections of wide sources after analysis of the distribution of the gray levels around said elementary detections of wide sources. The segmentation carried out in the third module 5 is therefore more precise than in the second module 4 thanks in particular to the use of the cumulative histogram.

[0079] Optionally, the first block 11 expands and / or erodes the binary image I2p to: - delete isolated segmentations, for example if said segmentations are too small and / or inconsistent with the other segmentations; and / or - group together segmentations that are close to each other.

[0080] For this purpose, the first block relies on predetermined criteria to define a “small” segmentation, an “incoherent” segmentation, an “isolated” segmentation, “close” segmentations, etc. For example, for “close” segmentations, the first block can rely on a proximity criterion expressed in row-column distance.

[0081] We thus obtain a binary image I3p having a main segmentation (and preferably which is the only segmentation of said binary image I3p) associated with the elementary detection of a wide source.

[0082] Said main segmentation preferably includes the elementary detection of a wide source. Furthermore, said main segmentation is chosen so as to be the largest segmentation in terms of surface area which includes the elementary detection of a wide source.

[0083] Preferably, the first block 11 labels the I3p binary image to extract an area in the I3p binary image associated with the elementary wide source detection. Preferably, said area is defined so as to contain the main segmentation without including other segmentations of the I3p binary image. Said area is also called a "segmentation mask" hereinafter.

[0084] Preferably, the segmentation mask follows the contours of the main segmentation and has a surface area greater than but as close as possible to said main segmentation.

[0085] The segmentation mask is therefore advantageously of a larger surface area than the elementary detection of a wide source while being associated with said wide source. This ensures better segmentation with the third module 5 than at the end of the second module 4.

[0086] An image (of the same dimensions as the initial image) comprising said segmentation mask can thus be generated. Such an image is denoted I4p binary image.

[0087] The other areas of the binary image I3p are thus considered to be of no interest, which is logical since the local detection threshold Thp has no meaning for these other areas, the local detection threshold Thp having been evaluated locally around the elementary detection of the wide source.

[0088] End of loop

[0089] The output of the first block 11 thus provides a segmentation mask for each elementary wide source detection transmitted as input to the first block 11.

[0090] Preferably, if the first block 11 cannot determine a segmentation mask for at least one of the elementary detections of wide sources, the first block 11 provides as output a segmentation zone associated with said elementary detection of wide source which has been determined by the second module 4. In this way, no elementary detection of wide source is lost through the first block 11.

[0091] Optionally, if the main segmentation does not include the elementary wide source detection (for example due to erosion and / or dilation), an additional segmentation is sought in the vicinity of said elementary wide source detection, segmentation which would include said detection. The vicinity is for example an area of ​​3 pixels by 3 pixels centered on the elementary wide source detection (or a central pixel of said elementary wide source detection). The corresponding segmentation mask will then be defined with respect to this additional segmentation (and not the main segmentation).If no additional segmentation can be determined then, as indicated above, it is considered that the first block 11 cannot determine a segmentation mask for this elementary detection of a wide source and the first block 11 provides as output a segmentation zone associated with said elementary detection of a wide source which has been determined by the second module 4. The third module 5 also comprises a second block 12 arranged downstream of the first block 11 and receiving from the latter the segmentation masks and / or the I4p images (and possibly segmentation zones associated with one or more elementary detections of wide sources, elementary zones which have been determined by the second module 4 and / or images comprising such zones) to provide as output generalized detections of wide sources.

[0092] For this purpose, and according to a first step, the second block 12 creates a binary image 14 which is in reality the union of all the binary images I4p (and optionally segmentation areas calculated by the second module 4 and / or images comprising such areas).

[0093] For example, binary image 14 is created as follows: - for each pixel belonging to a segmentation mask in an I4p image (and optionally to a segmentation zone calculated by the second module 12), the second block sets to 1 the value of the pixel having the same row-column position in the binary image 14; - if several segmentation masks (and optionally a segmentation zone calculated by the second module 12) include the same pixel, the value of the corresponding pixel in the binary image 14 is maintained at 1; - otherwise the value of the pixel of the binary image 14 is taken equal to 0.

[0094] An example of binary image 14 is visible in [Fig.6].

[0095] According to a second step, the second block 12 labels the binary image 14 to extract segmentation regions in the binary image 14 which are independent of each other and to consequently associate the segmentation masks which overlap, the overlapping segmentation masks thus jointly forming the same segmentation region.

[0096] The second block 12 thus constitutes one (or more) segmentation region(s) grouping (each) the segmentation masks associated with the different elementary detections of the same wide source.

[0097] An example of a labeled binary image 14 is visible in [Fig.7].

[0098] It is noted that each segmentation region is defined so as to encompass: - either a single segmentation mask, - or several segmentation masks which overlap each other.

[0099] Preferably, the segmentation region follows the contours of the associated segmentation mask(s) and has a surface area greater than but as close as possible to said mask (or the intersection of said masks).

[0100] Furthermore, it is thus possible to define a generalized detection of the wide source associated with each segmentation region.

[0101] This definition is made here during a third level.

[0102] Optionally, the second block 12 considers that the position of said general detection is the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level. The second block can for this purpose use the following formula:

[0103] with

[0104] X = the number of the column (or row) considered in the segmentation region,

[0105] i = a pixel in the segmentation region,

[0106] level = the gray level of the pixel considered, and

[0107] Mask = the set of pixels constituting said segmentation region.

[0108] Thus, the second block 12 provides at its output (which also corresponds here to that of the third module 5) the generalized detections of large sources and in particular their positions.

[0109] These generalized detections of wide sources are thus defined so as to group together elementary detections of wide sources associated with the same wide source by the second module.

[0110] Advantageously, these generalized detections of large sources are also more refined than the elementary detections of large sources with regard to their morphological characteristics (position, contour, etc.)

[0111] Referring again to [Fig.2], the image processing device 1 comprises a fourth module 6 for merging the generalized detections of wide sources generated by the third module 5 with the detections of point sources generated by the first module 3. This fourth module 6 is therefore arranged downstream of the first module 3 and the third module 5.

[0112] The fourth module 6 operates like a prior art wide source - point source fusion module (with the difference that the input data of the fourth module 6 will therefore be the generalized detections of wide sources coming from the third module 5 and no longer the elementary detections of wide sources generated by the second module 4) so ​​that it will not be detailed here.

[0113] At the output of the fourth module 6 (which also corresponds here to the output of the image processing device 1) the targets are recovered which will then be processed by the rest of the system.

[0114] An image processing device 1 has thus been proposed comprising two modules 4, 5 dedicated to the detection and segmentation of wide sources. The second module 4 allows the detection and segmentation of the points of interest on which to focus, then the third module 5 performs a local analysis of the cumulative histogram around all the points of interest provided by the second module 4. The third module 5 consequently makes it possible to adapt the local detection threshold for each of the points of interest and to obtain segmentation masks associated with said points of interest (more finely segmented than if one simply used the second module 4), masks which are then merged by proximity criterion, in order to develop a single detection per wide source.

[0115] More generally, an image processing device 1 has been proposed based on a segmentation of large sources controlled by multi-statistical criteria. premises in order to associate a unique detection with them.

[0116] Advantageously, the second module 4 and the third module 5 are relatively independent.

[0117] Thus the second module 4 can be relatively rustic and / or an already existing module of the prior art.

[0118] The use of the second module 4 - third module 5 duplicate therefore advantageously avoids the third module 5 having to resort to a complex segmentation process around the wide source since the third module 5 directly takes as input the elementary detections of wide sources provided by the second module 4 (which is a relatively rustic module) which thus indicates to it the zones to be prioritized for segmentation.

[0119] Similarly, the first module 3 and the fourth module 6 may be already existing modules of the prior art.

[0120] The third module 5 can thus easily be implemented in existing image processing devices of the prior art.

[0121] The third module 5 therefore improves the performance of the image processing device 1 and therefore of the system.

[0122] The image processing device 1 thus described improves the segmentation of large sources and their counting while maintaining the requirement of an acceptable false alarm rate (rate linked to the second module 4). Advantageously, the image processing device 1 thus described can even reduce this false alarm rate thanks to the grouping of the multi-detections on the sources (which can make it possible to discard unwanted detections such as, for example, waves on the sea, edges of clouds, etc.).

[0123] Of course, the invention is not limited to the implementation described and variant embodiments may be made without departing from the scope of the invention.

[0124] In particular, the invention is applicable to many other applications than the military maritime domain. The invention may thus be applied in the civil domain, in the military domain, in the land domain, in the aeronautical domain, in the airborne domain, in the transport domain in general, etc.

[0125] It will also be possible, in addition or as a replacement, to work in other wavelength bands than infrared and for example in the visible.

[0126] In the same way, we can use different types of optical sensors: camera (infrared, visible, etc.); radar (synthetic aperture radar, usually referred to by the English acronym SAR, side-opening radar, etc.); ...

[0127] The third module may be different from what has been indicated and for example perform additional functions. Thus if a segmented region and / or a segmentation mask is inconsistent (for example in terms of morphology), the third module may decide to associate with the elementary detection(s) of large sources the characteristics (in particular the position) provided by the second module. The detection of the inconsistency may for example be based on one or more of the following parameters: number of pixels obtained in the segmented region and / or the segmentation mask on total number of pixels, maximum permitted sizes in width and height predetermined in the segmented region and / or the segmentation mask, width to height ratio in the segmented region and / or the segmentation mask

Claims

Claims

1. Method for generating at least one generalized detection of a wide source present on an initial image comprising the successive steps of: - from at least one elementary detection of said wide source, generating a segmentation mask associated with said detection, the segmentation mask being of a larger surface area than the elementary detection of the wide source while being associated with said wide source, - generating at least one segmentation region such that each generated segmentation region is defined so as to encompass: either a single segmentation mask, or several segmentation masks which overlap with each other, - determining for each generated segmentation region a generalized detection.

2. Method according to claim 1, in which a cumulative histogram is determined on a portion of the initial image centered on the position of the elementary detection of the wide source considered.

3. Method according to claim 2, in which a local detection threshold is determined based on gray levels of the pixels of said portion by thresholding the cumulative histogram by an intermediate threshold of predetermined value.

4. A method according to claim 3 wherein the intermediate threshold has a value between 80 and 90%.

5. Method according to one of claims 3 to 4, in which the initial image is thresholded in gray levels by the local detection threshold in order to obtain a first image having one or more segmentations associated with the elementary detection of a wide source.

6. A method according to claim 5, wherein the first image is expanded and / or eroded to obtain a second image.

7. Method according to claim 5 or claim 6, in which the first image or the second image is labeled to extract the at least one segmentation mask.

8. Method according to one of claims 1 to 7 in which a new image is created including the at least one segmentation mask and said image is labeled to extract the at least one segmentation region.

9. Method according to one of claims 1 to 8 in which one calculates a position of the at least one generalized detection of the wide source associated with the at least one segmentation region.

10. Method according to claim 9 in which it is considered that the position of the at least one generalized detection of the wide source is the barycenter of the set of pixels constituting the associated segmentation region, weighted by their gray level.

11. Image processing device for implementing the method according to one of claims 1 to 10, the device comprising a first module (3) for detecting point sources, a second module (4) for detecting elementary detections of wide sources, a third module (5) for merging elementary detections of wide sources generated by the second module and a fourth module (6) for merging generalized detections of wide sources generated by the third module with the detections of point sources generated by the first module.

12.

13. An optronic system comprising a device according to claim 11. A marine vehicle comprising an optronic system according to claim 12.

Images