Method for displaying a multispectral image

Abstract

One aspect of the invention relates to a computer-implemented method for displaying a multispectral image, comprising: - selecting, within the multispectral image, an analysis window and a plurality of target areas within the analysis window; - segmenting a portion of the multispectral image into a set of classes; - for each target area: determining a respective background area; determining a Fisher projection to obtain an elementary image of the scene portion corresponding to the analysis window; - combining the elementary images to construct a synthetic image and displaying the synthetic image; wherein the background area is determined from the class to which the target area belongs. Figure to be published with the abstract: Figure 3

Description

Title of the invention: Method for displaying a multi-spectral image TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of monitoring an environment.

[0002] In particular, the present invention relates to a method of visualizing a multispectral image. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] Monitoring an environment is a common task, particularly for detecting adverse intrusions. Such monitoring presents particular difficulties when carried out in a terrestrial environment. Indeed, a terrestrial environment such as a countryside landscape may contain a large number of distinct elements with irregular contours such as trees, bushes, rocks, buildings, road signs, etc., which make the interpretation of the image and the search for intruding elements complex. Furthermore, in certain circumstances, an intruding element may be camouflaged so that its detection in the landscape is more difficult. Commonly, such camouflage is effective against observation in visible light, particularly for wavelengths which are between 0.45 pm (micrometer) and 0.65 pm, and particularly around 0.57 pm which corresponds to the maximum sensitivity of the human eye.

[0004] To successfully detect the intruder element, also called "target", despite a complex landscape and possible camouflage of the target, it is known to carry out a multispectral observation of the environment. Such a multispectral observation involves the simultaneous use of several images of the same landscape, each acquired in different spectral bands, so that a target which would not appear distinctly in the images captured according to certain spectral bands is revealed by the images corresponding to other spectral bands. Each spectral band can be narrow, with an interval of wavelength values ​​which extends over a few nanometers or a few tens of nanometers, or which can be wider.It is thus known that observation in the wavelength interval between 0.6 pm and 2.5 pm can be effective in revealing a target in a vegetation environment, while the target is effectively camouflaged against detection by observation in the light interval visible to the human eye.

[0005] Reference 10 in [Fig.l] generally designates such a multispectral image, formed of several individual images 11, 12, 13 which have been acquired simultaneously for the same scene. For example, the individual images 11, 12, 13 may have were acquired by imaging channels arranged in parallel, activated at the same time and having the same optical input field. Alternatively, the individual images 11, 12, 13 may have been acquired from a single imaging channel allowing the acquisition of several images associated with different spectral bands. However, each image 11, 12, 13 has been acquired by selecting only a portion of the radiation from the scene, which is separated from each portion of the radiation used for another of the images 11, 12, 13. This separation is carried out as a function of the wavelength X of the radiation in a known manner, so that each of the images 11, 12, 13, called a spectral image, has been acquired with a respective radiation whose wavelength belongs to a distinct interval, preferably without overlap with each of the intervals of the other spectral images. The number of spectral images 11, 12, 13 can be any. Such a multispectral image can also be called hyperspectral, depending on the number of spectral images that compose it and the width of each of the wavelength intervals thereof. For example, Xb X2, X3 designate central values ​​for the respective wavelength intervals of the spectral images 11, 12, 13.Furthermore, x and y denote two spatial dimensions, which are common to all spectral images 11, 12, 13. .

[0006] Each spectral image 11, 12, 13 may be processed from a file that is read from a storage medium, or from a digital stream that is produced by a camera instrument at a video rate. Depending on the case, the image data may be raw data produced by one or more image sensors, or data already processed for certain operations such as cropping the spectral images relative to each other, over- or under-exposure correction, etc.

[0007] However, such multispectral detection may still be insufficient to enable an operator in charge of surveillance to detect the presence of a target in a terrestrial environment. Indeed, in certain circumstances, none of the images which are separately associated with the spectral bands show the target clearly enough for the surveillance operator to be able to detect the target in these images, taking into account the observation time allocated.

[0008] For such situations, it is known to improve the effectiveness of target detection by presenting to the operator an image that is constructed by Fisher projection. Such a method is known in particular from the article “Some practical issues in anomaly detection and exploitation of regions of interest in hyperspectral images”, by F. Goudail et al., Applied Optics, Vol. 45, No. 21, pp. 5223-5236. According to this method, the image that is presented to the operator is constructed by combining at each point of the image, called a pixel, the intensity values ​​that are acquired separately for several spectral bands, so as to optimize a contrast of the resulting image. Theoretically, This image construction consists of projecting for each pixel the vector of intensities that have been captured for the selected spectral bands, onto an optimal direction in the multidimensional space of spectral intensity values. This optimal projection direction can be determined from the covariance matrix of the spectral intensities, estimated over the entire image field. This in fact amounts to seeking a maximum correlation between the variations in intensities that are present in the different images captured according to the selected spectral bands. The contrast of the image that is presented to the operator is thus at least equal to that of each separate spectral image, so that target detection by the operator is both more efficient and more reliable.Alternatively, the optimal projection direction can be sought directly using a common optimization algorithm, to maximize image contrast by varying the projection direction in the multidimensional space of spectral intensities.

[0009] It is common to restrict the contrast enhancement by Fisher projection to a spatial window within the multispectral image, called the analysis window, which is smaller than the entire image. The analysis window, denoted FA in [Fig.l], is then applied identically to all the spectral images 11, 12, 13 which constitute the multispectral image 10. This restriction to the analysis window FA concerns both the part of the multispectral image 10 from which the Fisher projection is determined, and the part of the multispectral image 10 to which this Fisher projection is applied. This results in a significant reduction in the computation time required to obtain improved contrast, which is advantageous for real-time execution during an environmental monitoring mission.A viewing image is then presented to the monitoring operator, in which the contents of the multispectral image within the analysis window are displayed as they result from the Fisher projection.

[0010] To determine the Fisher projection, it is necessary to define a target area inside the analysis window, and a background area associated with it. [Fig.2] shows a common arrangement of the target area in the FA analysis window: the target area, denoted T, is located in the center of the FA analysis window, and has a shape that can be square with a side of a few pixels, for example 3x3 or 7x7 pixels. The background area, denoted B, surrounds the target area T, being separated from it by an intermediate area, or guard area, denoted G. The background area B can be externally limited by the FA analysis window itself.

[0011] The contrast that is intended to be maximized by the Fisher projection is calculated between the content of the multispectral image in the target area, and the content of this same multispectral image in the background area. Thus, to determine the Fisher projection, the intensities of the spectral images are spatially averaged over the area target on the one hand, and on the background area on the other.

[0012] Therefore, the effectiveness of Fisher projection in enhancing contrast may be reduced in the following situations in particular:

[0013] / i / case of a variegated target, that is to say which presents spectral characteristics different between neighboring parts of the target that are simultaneously contained within the target area; or

[0014] / ii / case of several targets each of small dimensions which would be contained simultaneously in the target zone, or some of which would be in the target zone and others in the background zone.

[0015] To improve the effectiveness of the Fisher projection in enhancing contrast, particularly in such adverse situations, patent FR3037428B1 proposes defining a plurality of target areas within the FA analysis window. The content of the multispectral image 10 in the FA analysis window is then processed to improve the contrast between each target area and a background area associated with it, independently of the other target areas. This processing produces an elementary image for the content of the multispectral image 10 in the FA analysis window, which is distinct for each target area. These elementary images are then combined or merged into a synthetic image, which is an improved representation of the content of the multispectral image 10 within the FA analysis window.This computer-generated image can then be presented to the monitoring user, for example displayed on a screen, so that the user can make a decision or trigger an action.

[0016] While the method described in patent FR3037428B1 significantly improves contrast in situations such as those mentioned above, it nevertheless struggles to detect certain targets, particularly when these are located in complex and highly heterogeneous background areas, such as forest landscapes. In such cases, the method does not always succeed in identifying the backgrounds useful for "decamouflage" in relation to the targets that are sought to be detected.

[0017] There is therefore a need for a more robust multispectral image visualization method, which makes it possible to reinforce the contrast around the target, even when the latter is located in a region of the image whose background is highly heterogeneous. Summary of the invention

[0018] The invention provides a solution to the problems mentioned above, by segmenting the multispectral image into a set of K classes, with K greater than or equal to 1, and by determining, for each target area, the background area associated with it as a function of the class to which the target area belongs. Thus, the background area corresponds to the same class of the image, and therefore comprises “similar” pixels. This allows to improve the contrast between a potential anomaly present in the analysis window and the background, and therefore to provide the operator with an image in which a potential anomaly appears more distinctly.

[0019] One aspect of the invention thus relates to a method, implemented by computer, for displaying a multispectral image on a display unit, the multispectral image comprising several spectral images of the same scene acquired simultaneously, the method comprising:

[0020] - select, within the multispectral image, an analysis window cor corresponding to a portion of the scene and a plurality of target areas in the analysis window;

[0021] - for each target area in the analysis window: • determine a background area associated with each target area; • determine a Fisher projection from image intensity values spectral, said Fisher projection being a linear combination of the spectral images which optimizes a contrast between said each target area and the background area which is associated with said each target area; • apply the determined Fisher projection to the intensity values ​​of the spectral images included in the analysis window, so as to obtain an elementary image of the part of the scene corresponding to the analysis window;

[0022] - combine the elementary images obtained to construct a synthetic image relative to the part of the scene corresponding to the analysis window;

[0023] - display an image of the scene on the display unit, using in the window analysis of the constructed synthetic image;

[0024] the method being characterized in that it further comprises:

[0025] - segment the multispectral image into a set of classes;

[0026] and in that, for each target area in the analysis window, determining the background area associated with said each target area comprises:

[0027] - select a primary background area associated with said each target area and separated from said target area by an intermediate area such that the target area is not contiguous with the background area which is associated with said each target area;

[0028] - determine, among the set of classes, the class comprising said each zone of target;

[0029] - determining the background area associated with said each target area as an in intersection between the primary background area associated with said each target area and the selected class.

[0030] In a known manner, a multispectral image is formed from several spectral images of the same scene which were acquired at the same time respectively in separate spectral intervals.

[0031] By analysis window is meant an area corresponding to a part of the scene. Typically, if the multispectral image comprises a grid of pixels, the analysis window defines a subset of pixels in the grid of pixels. For example, the analysis window is a rectangle inside the multispectral image. It is noted that the analysis window is generally of dimensions strictly smaller than those of the multispectral image, but it can also have the same dimensions.

[0032] By "target area" is meant an area within the analysis window, of smaller dimensions than the analysis window (for example a 3x3, 5x5 or 7x7 pixel square). The target area corresponds to an area whose contrast is optimized with respect to a background, to highlight the possible presence of an anomaly.

[0033] Each target area is associated with a respective background area relative to which the contrast is analyzed. According to the present invention, the background area is determined so as to be homogeneous (and to correspond to the background in front of which the target area is positioned). For this, a so-called "primary" background area is first determined. The terminology "primary background area" is used to facilitate distinction with the (final) "background area" on the basis of which the Fisher projection is determined.

[0034] This primary background area is not contiguous with the target area. By "is not contiguous", it is understood that no pixel of the primary background area is adjacent to a pixel of the target area. By construction, the background area (i.e. the "final" background area) is also non-contiguous with the target area.

[0035] Thus, each elementary image optimizes the contrast with respect to one of the target areas, separately from the other target areas, by using a homogeneous background area. A target or a portion of a target can thus be revealed in the intermediate image, even when the image has different background areas that are very heterogeneous with each other.

[0036] For example, the segmentation of the multispectral image into a set of classes can be carried out by a k-means type method.

[0037] In embodiments, the method further comprises:

[0038] - determine, among the set of classes, a class called background class, in function of the classes determined for the plurality of target areas;

[0039] - apply a contrast enhancement method to a portion of the image of synthesis corresponding to the determined background class;

[0040] wherein the image of the scene displayed on the display unit is constructed using in the analysis window the synthetic image after the application of the contrast enhancement method.

[0041] In these embodiments, a background class is determined to be the class corresponding to the background in front of which the potential target is located. For example, the background class may correspond to the class associated with the greatest number of target areas among the plurality of target areas. Thus, the contrast is enhanced in this area, which makes a potential target stand out more compared to the background in front of which it is placed.

[0042] For example, the contrast enhancement method may be a histogram stretching method.

[0043] In addition, the method may comprise:

[0044] - decrease intensity values ​​in the computer-generated image deprived of said portion cor corresponding to the determined background class;

[0045] wherein the image of the scene displayed on the display unit is constructed using in the analysis window the synthetic image after the application of the contrast enhancement method and the reduction of the intensity values.

[0046] By "synthetic image deprived of said portion corresponding to the background class", is meant the portion(s) of the synthetic image whose pixels are associated with a class other than the background class.

[0047] Thus, the portion of the image that does not include the potential target is “darkened”, so that the potential target appears more distinctly.

[0048] In embodiments, at least some of the plurality of target areas have overlaps with each other in the analysis window.

[0049] Such overlaps may be advantageous in the case of variegated targets, so that certain target area boundaries are closer to the boundaries between parts of the target that have different spectral characteristics.

[0050] In embodiments, the target areas of the plurality of target areas are aligned in the analysis window.

[0051] In particular, the scene may be a portion of a terrestrial landscape, and the target areas of the plurality of target areas may be aligned parallel to a direction that is vertical or horizontal relative to the terrestrial landscape.

[0052] Indeed, a terrestrial landscape is naturally traversed by an observer by moving his direction of gaze vertically or horizontally.

[0053] In these embodiments, each class may correspond to a respective landscape pattern. For example, the set of classes may include: forest, field, road.

[0054] In other embodiments, the target areas of the plurality of target areas are arranged in the analysis window to form a cross or a box.

[0055] In embodiments, a common primary background area is associated with all target areas in the analysis window.

[0056] Alternatively, distinct primary background areas are respectively associated with the target areas of the analysis window, and for all target areas, a relative arrangement of the target area and its associated primary background area is constant. In other words, all target areas are arranged in the same way relative to their respective primary background areas (e.g., each target area may be a small square centered inside its respective primary background area, which is, for example, a larger square).

[0057] According to another alternative, distinct primary background areas are respectively associated with the target areas of the analysis window, with an arrangement of said primary background areas such that no primary background area has an overlap with any of the target areas.

[0058] In particular, the primary background zone which is associated with each target zone results from a selection of at least one primary background zone segment from among several background zone segments which are likely to be associated with said target zone, said selection being carried out to reduce differences existing between respective spectral signatures of said primary background zone segments selected for the same target zone, and the primary background zone associated with each target zone is a union of the primary background zone segments selected for said target zone.

[0059] In embodiments, for each target area of ​​the analysis window, the Fisher projection is determined by weighting intensity values ​​of the spectral images, in the background area which is associated with said target area, with coefficients whose values ​​are decreasing functions of a distance between the target area and a location in the background area to which the intensity values ​​of the spectral images correspond.

[0060] Thus, an element of the scene which is further away from the target area is less likely to alter a spectral homogeneity of the background area, for the calculation of the contrast, than an element of the scene which is closer to this target area.

[0061] In embodiments, the synthetic image is constructed by applying an intensity scale correction to each elementary image, then by assigning to points of the synthetic image which are in the analysis window:

[0062] - for each of said points, a maximum value of the intensity values ​​at the same point among the corrected elementary images; or

[0063] - for each of said points, a weighted average of said intensity values ​​of the corrected elementary images, relative to said point and raised to the power n, n being a non-zero and fixed positive integer.

[0064] In embodiments, several analysis windows are selected successively in the multispectral image, and, for each analysis window among the several analysis windows selected: a synthetic image relating to the part of scene contained in said analysis window is constructed and an image of the scene is displayed on the display unit using in said analysis window the constructed synthetic image.

[0065] Another aspect of the invention relates to a device for displaying a multispectral image on a display unit, the multispectral image comprising several spectral images of the same scene acquired simultaneously, the device being configured to implement a method as defined above.

[0066] A computer program, implementing all or part of the method described above, installed on pre-existing equipment, is in itself advantageous.

[0067] Thus, the present invention also relates to a computer program product comprising instructions for implementing the method described above, when this program is executed by a processor.

[0068] This program may use any programming language (for example, an object-oriented language or other), and be in the form of interpretable source code, partially compiled code or fully compiled code.

[0069] The [Fig.3] described in detail below can form the flowchart of the general algorithm of such a computer program.

[0070] The invention and its various applications will be better understood upon reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES

[0071] Other characteristics and advantages of the invention will appear on reading the description, which can be read in conjunction with the figures. These figures are presented for information purposes only and in no way limit the invention.

[0072] [Fig.l] [Fig.l] represents a schematic representation of a multispectral image to which the invention can be applied;

[0073] [Fig.2] [Fig.2] represents an example of the division of the analysis window used to determine a Fisher projection of the multispectral image according to the state of the art;

[0074] [Fig.3] [Fig.3] represents a flowchart of a method for displaying a multispectral image according to an embodiment of the invention;

[0075] [Fig.4a], [Fig.4b], [Fig.4c], [Fig.4d], [Fig.4e], [Fig.4f] Figures 4a to 4f re present examples of arrangements of target areas and primary background areas;

[0076] [Fig.5a], [Fig.5b], [Fig.5c], [Fig.5d] Figures 5a to 5d represent an example of determination of a background zone Bi associated with a target zone Ti;

[0077] [Fig.6a], [Fig.6b], [Fig.6c] Figures 6a to 6c represent an example of processing of the obtained synthetic image;

[0078] [Fig.7] [Fig.7] represents a device for displaying a multispectral image according to one embodiment of the invention.

[0079] For the sake of clarity, the dimensions of the elements shown in these figures do not correspond to actual dimensions or to actual dimensional ratios. Furthermore, identical references indicated in different figures designate identical elements or those having identical functions. DETAILED DESCRIPTION

[0080] [Fig.3] [Fig.3] represents a flowchart of a method of displaying an image multispectral according to one embodiment of the invention.

[0081] The method described below with reference to [Fig. 3] can be applied to a multispectral image as shown in [Fig. 1]. The multispectral image 10 can be read from a data storage medium, on which it was recorded after being acquired. This is then a deferred processing of the multispectral image. Alternatively, the multispectral image 10 can belong to a video stream which is acquired and viewed in real time by a surveillance operator.

[0082] The multispectral image acquisition device(s), an image processing unit capable of implementing the method of the invention, and an image display unit may be carried on board a vehicle, for example a land vehicle with a surveillance function, or may be integrated into individual viewing equipment such as a pair of binoculars. In other contexts of use of the invention, the multispectral image acquisition device(s) 10 may be carried on board an aircraft, for example an aerial drone, while the image processing and display units may be located in a control and observation station which is static and remote. In such a context, the multispectral images which are acquired are transmitted to the processing unit, by electromagnetic signals in a manner which is known per se.

[0083] In the following, each image 11, 12, 13 which corresponds separately to one of the spectral bands of the multispectral image 10, is called a spectral image.

[0084] With reference to [Fig. 3], the display method comprises a step 110 in which the FA analysis window is selected in the multispectral image 10. The FA analysis window may be located at a position which may be fixed relative to an external spatial boundary frame of the image 10, or at a position which may be selected by the surveillance operator. For example, the FA analysis window may thus be placed at the center of the multispectral image 10. The dimensions of the FA analysis window may vary, in particular depending on the number and size of the target areas, described below. For example, the FA analysis window may be a square of 101 x 101 pixels, but other sizes may be used.

[0085] Several target zones, denoted T1, T2, T3, T4 are then selected within the analysis window FA during a step 120. It is noted that in the context of the invention, several target zones are selected for the same analysis window.

[0086] The target areas T1, T2, T3, T4 may be aligned vertically within the FA analysis window, as illustrated in Figures 4a-4c and 4f. Such vertical alignment is particularly suitable for monitoring a terrestrial landscape by subsequently moving the FA analysis window horizontally within the landscape.

[0087] Alternatively, the target areas T1, T2, T3, T4 may be aligned horizontally in the analysis window FA, for example when this analysis window is then moved in the landscape in a direction which is contained in a vertical plane. Such a horizontal arrangement of the target areas in the analysis window may be more suitable for multispectral images which are captured from an aerial drone.

[0088] Alternatively, the target areas may be grouped into a block, as shown in [Fig.4d], or to form a cross, as shown in [Fig.4e]. Such configurations are suitable for tracking targets that are likely to move relatively slowly in the landscape, between two multispectral images acquired and then processed successively according to the invention.

[0089] Finally, for each general arrangement of the target areas T1, T2, T3, T4 in the analysis window FA, two target areas which are neighbors can be spaced from each other, as in Figures 4a, 4b and 4f, or have partial overlaps, as in Figures 4c-4e, or even be contiguous without overlap or intermediate space.

[0090] For example, each target area T1, T2, T3, T4 can be a square of 3x3, 5x5 or 7x7 pixels.

[0091] In a step 130, a segmentation of the multispectral image 10 into a set of K classes Ak is implemented, where K is a natural integer between 1 and the number N of pixels in the image. For example, each pixel of the multispectral image 10 may be associated with a respective class. Such a segmentation may be implemented by an image segmentation method of the state of the art. For example, when the image represents a terrestrial landscape, the number of classes K may be equal to 3, and the different classes may be: “forest”, “field”, “road”. When the image is homogeneous (a single pattern), the number of classes K may be equal to 1. Of course, these examples are in no way limiting of the invention.

[0092] It is noted that in step 130, the segmentation can be performed only on a portion of the multispectral image (and not the entire multispectral image), said portion comprising the FA analysis window (i.e. the portion represents at least the part of the scene covered by the FA analysis window).

[0093] In step 130, any known image segmentation method can be used, for example a data partitioning method such as a k-means type method, a method based on the use of neural networks, a method based on histogram thresholding such as the Otsu method, etc.

[0094] Thus, at the end of the segmentation step 130, the multispectral image 10 is subdivided into K classes Ak, which means that each pixel of the multispectral image 10 is associated with a respective class Ak, where k E [[ 1, K ]].

[0095] It is noted that step 130 may be implemented before steps 110 and 120, or after step 142 described below.

[0096] In steps 140, 150 and 160, the content of the multispectral image 10 in the analysis window FA is processed to improve the contrast between each target area and a background area associated with it, independently of the other target areas. This processing produces an elementary image for the content of the multispectral image 10 in the analysis window FA, which is distinct for each target area T1, T2, T3, T4. Thus, for each target area selected in step 120 within the analysis window FA, a sequence of steps 140, 150 and 160 is implemented independently of the other selected target areas. The different sequences of steps 140, 150 and 160 for the different target areas can be implemented successively or in parallel.

[0097] In step 140, a background area associated with the target area Ti for which the sequence of steps 140, 150 and 160 is being executed is determined. Generally, this background area is separated from the target area Ti by an intermediate guard area, so that certain small targets which are partially contained in the target area Ti do not overflow into the background area. For example, the guard area may have a thickness of at least 15 pixels between the target area Ti and the background area.

[0098] Step 140 of determining the background zone associated with the target zone Ti considered comprises a first step 142 of selecting a background zone called primary PBi (see [Fig.5c]) associated with the target zone Ti considered.

[0099] Generally, the primary background area PBi associated with the target area Ti is separated from the target area Ti by an intermediate area Gi such that the target area Ti is not contiguous with the primary background area PBi. In other words, no pixel of the target area Ti is a direct neighbor of a pixel of the primary background area PBi.

[0100] According to a first example illustrated in [Fig.4a], a different primary background zone PBI, PB2, PB3, PB4 can be associated with each target zone T1, T2, T3, T4 according to relative arrangements which are identical for all target areas. For example, each target area T1, T2, T3, T4 may be surrounded by the corresponding primary background area PB1, PB2, PB3, PB4, the latter having inner and outer boundaries which may be square, with an intermediate guard zone, denoted Gl, G2, G3, G4 for each pair of target and background areas {Ti,PBi}.

[0101] According to a second example illustrated in Figures 4b to 4e, a single primary background zone, denoted PB, can be associated with all the target zones T1, T2, T3, T4. Preferably, the primary background zone PB can be arranged around all the target zones T1, T2, T3, T4 with an intermediate guard zone which is denoted G. An advantage of such a second arrangement is to prevent one of the target zones from covering the primary background zone associated with another of the target zones. Thus, for a position of the FA analysis window, a scene element located in the FA analysis window is processed to calculate the contrast in the multispectral image, either as a potential target within at least one of the target zones, or as a scene background, in a way that is consistent for all the target zones.

[0102] Other possibilities for associating a primary background zone PBi with each target zone T1, T2, T3, T4 are illustrated in 4f, and described below. Background zone segments which are respectively denoted SB1-SB14, are defined around the target zones T1, T2, T3, T4 with an intermediate guard zone G. Then, the primary background zone associated respectively with each target zone can be constituted by the union of several of the segments SB1-SB14 which are assigned to this target zone.

[0103] According to a third example for defining the primary background area associated with each target area, each target area is initially assigned those of the primary background area segments that are spatially closest to it. For example, for the target area configuration of [Fig.4f], the primary background area PBI associated with the target area T1 may be the union of segments SB1-SB5 and SB13-SB14, the primary background area PB2 which is associated with the target area T2 may be the union of segments SB4-SB6 and SB12-SB14, the primary background area PB3 associated with the target area T3 may be the union of segments SB5-SB7 and SB11-SB13, and the primary background area PB4 associated with the target area T4 may be the union of segments SB6-SB12.

[0104] According to a fourth example, the selection of the primary background area which is associated with each target area makes it possible to ensure that the content of the multispectral image in this background area is sufficiently homogeneous spectrally. For this, a spectral signature can be determined for each of the primary background area segments SB1-SB14. Such a signature can be for example a vector which is constituted by the average values ​​of the intensities of each spectral image 11, 12, 13 calculated on all the pixels of this primary background area segment. Such a vector has therefore a separate coordinate for each spectral image. The signatures of all primary background area segments are then compared with each other, and the segment(s) whose signature(s) is (are) the furthest from those of the other segments is (are) discarded. Only the primary background area segments that remain after such discrimination can be selected to constitute the primary background area associated with the target area.

[0105] According to this example, the spectral signatures of the primary background zone segments are given by the following formula, expressed for the primary background zone segment SBj:

[0106] R (image point) JF Nj ^image points of the segment SBj A. \ t' & /

[0107] where j is an integer that counts the predefined primary background area segments, denoted SBj (j varies from 1 to 14 in the example of Figure 4f), Sj is the spectral signature of the primary background area segment SBj, k is an integer that counts the spectral images 11, 12, 13 that constitute the multispectral image 10, Ik is the intensity of the spectral image k at the pixel considered, N j is the number of pixels in the primary background area segment SBj, and 5' is the k-th coordinate of the signature vector Sj. Then, a separation angle between two of the signatures can be calculated according to the formula:

[0108] .. [ (M) ' 3; v = arccos ——r " l NN

[0109] where (S jS^ denotes the scalar product of the signature vectors Sj and gj, |Sj| denotes the norm of the signature vector Sj, arccos is the inverse of the cosine function, and 0jj' is the separation angle between the two vectors of the signatures of the primary background zone segments SBj and SB;>. The separation angles which are thus obtained for all pairs of primary background zone segments (SBj, SB,) are compared with each other, and those of the primary background zone segments SBj which form only separation angles &jj' which are small, with the majority of the other primary background zone segments SBj', are kept to participate in the primary background zones which are respectively associated with the target zones Ti.

[0110] Generally speaking, for the invention, it is not necessary for each primary background zone to be contained in the FA analysis window, but some of the primary background zones or the primary background zone which is common to all the target zones for the implementations concerned, may (may) overflow outside the FA analysis window. Furthermore, it is specified that the primary background zone which is associated with one of the target zones does not necessarily surround the latter completely, but may only be arranged on some of its sides. For example, each zone primary background or primary background area segment may have a dimension or thickness in the multispectral image that is greater than or equal to 3 pixels, in a spatial direction that is vertical or horizontal relative to the scene.

[0111] Then, at a step 144, the class Ak, among the set of classes defined at step 130, to which the target zone Ti considered belongs is determined.

[0112] In step 146, the background area Bi (see [Fig.5d]) associated with the target area Ti considered is determined as being the intersection between the primary background area selected in step 142 and the selected class Ak. In other words, the background area Bi comprises the pixels belonging to the primary background area which are associated with the selected class Ak.

[0113] Figures 5a to 5d illustrate steps 130 and 140 of the method of [Fig. 1].

[0114] [Fig.5a] represents an example of a multispectral image 10 of a terrestrial landscape comprising two types of patterns 10a, 10b. For example, image 10 represents a forest edge landscape comprising a forest pattern 10a and a field pattern 10b.

[0115] [Fig.5b] represents the result of a segmentation obtained at the end of step 130. The image thus segmented comprises two zones corresponding to respective classes A1, A2. Here, class A1 is the “forest” class and class A2 is the “field” class. In this example, the number of classes is equal to 2, but it is understood that the number of classes used for the segmentation may be equal to 1 or greater than 2 (for example between 3 and 10), or even much greater than 2 (for example, several dozen classes). Furthermore, in this example each segmented zone is presented in a single “block”, that is to say comprises a set of pixels in which each pixel is at least close to another pixel of the zone (in other words, in this example no zone comprises separate groups of pixels). This configuration is of course not obligatory. There may be several separate zones associated with the same class.Generally speaking, at the end of the segmentation implemented in step 130, each pixel of the image is associated with a respective class (here, class A1 or class A2).

[0116] [Fig.5c] shows a target area Ti and a primary background area PBi associated with the target area Ti (as selected in step 142). In this example, the primary background area PBi comprises pixels between two squares of different sizes centered on the target area Ti, such that the smaller of the two squares has a size chosen such that no pixel of the target area Ti is in contact with a pixel of the smaller of the two squares.

[0117] Then, it is determined to which class Al, A2 the target area Ti belongs (step 144). This is done by looking at the class Al, A2 associated with the pixels of the target area Ti. In this example, all the pixels of the target area Ti belong to the class AL. Thus, the class determined in step 144 is the class AL.

[0118] It is noted that, when the target area Ti comprises several pixels, it may happen that some pixels of the target area Ti belong to one class and other pixels belong to another class. In this case, the class A1, A2 to which the target area Ti belongs can be determined from a predefined rule. An example of such a predefined rule is to select the class to which the pixel located in the center of the target area Ti belongs. Another example is to select the class to which the largest number of pixels of the target area Ti belongs.

[0119] [Fig.5d] represents the “final” background area Bi, determined in step 146. The background area Bi is the intersection between the primary background area PBi and the region(s) of the image associated with the selected class (here, Al). In other words, the background area Bi comprises the pixels of the primary background area PBi associated with the selected class AL

[0120] Referring again to [Fig.3], in a step 150, the Fisher projection which maximizes the contrast between each target area Ti and the background area Bi associated with it is determined. Such a Fisher projection is known and is not detailed further here.

[0121] According to one embodiment of the invention, the Fisher projection can be determined by weighting the intensity of each pixel of a background area Bi associated with a respective target area Ti as a function of the distance between this pixel and the target area Ti (for example, from its center). For example, if I^x, y) denotes the intensity value of the pixel with coordinates (x, y) in the spectral image k, the average weighted intensity of the background TT to be used for this spectral image k during the 1 k\XTi> yTi) determination of the Fisher projection for the target area Ti and the associated background area Bi can be:

[0122] 7*3 y / \ T / \ . xLe / AUT) X / *U» >4

[0123] with , \ / f 3 7” 3 (xé) = 1 / V (- xTi) + (yi - yT.)

[0124] where xTi and y^ are the coordinates of the center of the target area Ti and jy ) are the coordinates of the pixels in the background area Bi. Thus, parasitic scene elements that are contained in the background area disturb less the contrast enhancement that is produced by the Fisher projection, when they are further away from the target area. In general, the values ​​of the weighting coefficients can vary according to any decreasing function of the distance between the pixel in the background area and the target area. Such weighting can also be used to calculate the signatures of the background area segments in the fourth example that was presented higher.

[0125] In step 160, for the target area Ti considered, the Fisher projection which was obtained for this target area in step 150 is then applied to the multispectral image 10 inside the window FA. An elementary image is thus obtained, which represents the content of the multispectral image 10 in the analysis window FA with maximum contrast.

[0126] In a test 165, it is checked whether a respective elementary image has been determined for all target areas Ti. If this is not the case, the method continues at step 140 for a target area Ti for which an elementary image has not yet been determined.

[0127] If all the elementary images have been determined, the method continues to step 170.

[0128] In step 170, all the elementary images obtained from all the target areas Ti (one elementary image per target area Ti) in steps 160 are combined to construct a single synthetic image. Such a combination can be carried out by retaining, for each pixel in the analysis window FA, the maximum intensity at this pixel among all the elementary images (i.e., all the intensities of this pixel in the elementary images are looked at and the largest is selected). Alternatively, the intensities of all the elementary images at this point can be raised to a fixed, common and positive power p, and the intensity of the synthetic image at the pixel considered can be equal to the average of these elementary image intensities raised to the power p.

[0129] Then, the synthetic image obtained in step 170 in the FA analysis window is displayed to the monitoring operator in step 180. For this, it can be displayed on a screen, or more generally on a display unit which is provided for visual observation. Optionally, to facilitate the location of the operator in the scene, the displayed image can be supplemented outside the FA analysis window by one of the spectral images, for example by a spectral image which corresponds to a thermal radiation acquisition channel. Alternatively, the image which is displayed can be supplemented outside the FA analysis window by a combination of several spectral images which correspond to different colors in the visible range, to reconstruct a daytime vision in colors outside the FA analysis window.Finally, it is also possible to supplement the image that is presented to the operator outside the FA analysis window, with an image of the scene that is optically formed and transmitted directly to the display unit.

[0130] In certain embodiments, an optional step 175 can be implemented after the step 170 of determining the synthetic image and before the step of displaying this synthetic image.

[0131] During this optional step 175, a contrast enhancement method is applied to the synthetic image obtained in step 170. More precisely, a contrast enhancement method is applied to the portion of the synthetic image associated with the class A1, A2 determined during step 144 for the target areas Ti.

[0132] It is noted that the classes determined during step 144 may differ depending on the target areas considered. In this case, step 175 comprises a prior determination of a class from among the set of classes of the image. For example, the determined class may be the class associated with the largest number of target areas Ti from among all the target areas Ti.

[0133] Thus, the contrast is increased in the region of interest of the image, i.e. in the region containing the potential anomaly (target). If an anomaly is actually present, its contrast with respect to the background is accentuated (it appears lighter than the background), which makes it easier to visualize for the operator. The contrast enhancement method can be any method known from the state of the art, for example a histogram stretching.

[0134] Furthermore, during step 175, it is possible to reduce the intensity values ​​of the pixels of the elementary image which correspond to classes different from that in which the contrast enhancement is applied (the regions of the images outside the region of interest). This makes it possible to “darken” the part of the background which does not contain the potential anomaly, and therefore to make the potential target stand out more, which appears lighter.

[0135] An example of implementation of step 175 is shown in Figures 6a to 6c. [Fig.6a] corresponds to the synthetic image obtained at the end of step 160. In this example, the image has been segmented into two classes A1, A2, and the target area Ti considered belongs to class A1. [Fig.6b] represents the synthetic image after applying a contrast enhancement in the area corresponding to class A1. It can be seen in [Fig.6b] that the target stands out more compared to the background pattern located behind said target. [Fig.6c] represents the image of [Fig.6b] after reducing the intensity values ​​in the area corresponding to class A2. The target stands out clearly from the two areas associated with classes A1, A2, which allows the operator to detect it (and confirm that it is indeed an anomaly) more easily.

[0136] The synthetic image thus processed can then be displayed in step 180 as described above.

[0137] It is understood that the dimensions, shapes and positions of the target and background areas which have been given as examples in the detailed description above, may be modified depending on the application and the implementation context. In particular, they may be determined from additional data, such as the distance and / or the size of a target which is sought in the imaged scene.

[0138] [Fig.7] represents a device for displaying a multispectral image according to a embodiment of the invention.

[0139] In this embodiment, the device comprises a computer 700, comprising a memory 701 for storing instructions allowing the implementation of the method, the images received and temporary data for carrying out different steps of the methods described previously.

[0140] The computer 700 further comprises a circuit 702. This circuit may be, for example, a processor capable of interpreting instructions in the form of a computer program, an electronic card whose steps of the method of the invention are described in the silicon, or even a programmable electronic chip such as an FPGA chip (for “Field-Programmable Gate Array” in English).

[0141] The computer 700 comprises an input interface 703 for receiving in particular a multispectral image, an analysis window and / or a target area, and an output interface 704 for providing the synthetic image to a display unit. The display unit may be external to the computer 700, or may belong to the computer 700. Thus, the computer may comprise a display unit such as a screen 705. The computer 700 may also comprise, to allow easy interaction with a user, a keyboard 706 and / or a mouse. Of course, the keyboard and the mouse are optional, in particular in the context of a computer having the form of a touch pad, for example.

[0142] Furthermore, the functional diagram presented in [Fig. 3] is a typical example of a program of which certain instructions can be carried out using the device described. In this respect, [Fig. 3] can correspond to the flowchart of the general algorithm of a computer program within the meaning of the invention.

[0143] Of course, the present invention is not limited to the embodiments described above as examples.

Claims

Claims

1. A computer-implemented method of displaying a multispectral image (10) on a display unit, the multispectral image (10) comprising several spectral images (11, 12, 13) of the same scene acquired simultaneously, the method comprising: - selecting (110, 120), within the multispectral image (10), an analysis window (FA) corresponding to a part of the scene and a plurality of target zones (Ti) in the analysis window (FA); - for each target area (Ti) in the analysis window (FA): • determine (140) a background area (Bi) associated with said each target area (Ti); • determining (150) a Fisher projection from intensity values of the spectral images (11, 12, 13), said Fisher projection being a linear combination of the spectral images (11, 12, 13) which optimizes a contrast between said each target area (Ti) and the background area (Bi) which is associated with said each target area (Ti); • apply (160) the determined Fisher projection to the intensity values of the spectral images (11, 12, 13) included in the analysis window (FA), so as to obtain an elementary image of the part of the scene corresponding to the analysis window; - combine (170) the elementary images obtained to construct a synthetic image relating to the part of the scene corresponding to the analysis window (FA); - displaying (180) an image of the scene on the display unit, using in the analysis window (FA) the constructed synthetic image; the method being characterized in that it further comprises: segmenting (130) at least a portion of the multispectral image (10) into a set of classes (A1, A2), said portion comprising the analysis window; and in that, for each target area (Ti) in the analysis window (FA), determining the background area (Bi) associated with said each target area (Ti) comprises: - selecting (142) a primary background area (PBi) associated with said each target area (Ti) and separated from said target area by an intermediate area such that the target area (Ti) is not contiguous with the background area (Bi) which is associated with said each target area (Ti); - determining (144), from the set of classes (Al, A2), the class (Al) comprising said each target area (Ti); - determining the background area (Bi) associated with said each target area (Ti) as an intersection between the primary background area (PBi) associated with said each target area and the selected class (Al).

2. Method according to the preceding claim, further comprising: - determining, from among the set of classes, a class called background class, as a function of the classes determined for the plurality of target zones (Ti); - applying (175) a contrast enhancement method to a portion of the synthetic image corresponding to the determined background class; in which the image of the scene displayed (180) on the display unit is constructed using in the analysis window the synthetic image after the application of the contrast enhancement method.

3. Method according to the preceding claim, further comprising: - decreasing (175) intensity values in the synthetic image deprived of said portion corresponding to the determined background class; wherein the image of the scene displayed (180) on the display unit is constructed using in the analysis window the synthetic image after the application of the contrast enhancement method and the reduction of the intensity values.

4. Method according to one of the preceding claims, in which at at least some target areas (Ti) among the plurality of target areas (Ti) have overlaps between them in the analysis window (FA).

5. Method according to one of the preceding claims, wherein the target areas (Ti) of the plurality of target areas (Ti) are aligned in the analysis window (FA).

6. A method according to the preceding claim, wherein the scene is a portion of a terrestrial landscape, and wherein the target areas (Ti) of the plurality of target areas (Ti) are aligned parallel to a direction which is vertical or horizontal relative to the terrestrial landscape.

7. A method according to any one of claims 1 to 6, wherein distinct primary background areas (PB1, PB2, PB3, PB4) are respectively associated with the target areas (T1, T2, T3, T4) of the analysis window (FA), with an arrangement of said primary background areas such that no primary background area has an overlap with any of the target areas.

8. Method according to the preceding claim, in which the primary background area (PB1, PB2, PB3, PB4) which is associated with each target area (T1, T2, T3, T4) results from a selection of at least one primary background area segment from among several background area segments (SB1, SB2, ..., SB14) which are likely to be associated with said target area, said selection being carried out to reduce differences existing between respective spectral signatures of said primary background area segments selected for the same target area, and the primary background area associated with each target area is a union of the primary background area segments selected for said target area.

9. Method according to one of the preceding claims, in which, for each target area (Ti) of the analysis window (FA), the Fisher projection is determined by weighting intensity values of the spectral images (11, 12, 13), in the background area which is associated with said target area, with coefficients whose values are decreasing functions of a distance between the target area and a location in the background area to which the intensity values of the spectral images correspond.

10. Method according to one of the preceding claims, in which the synthetic image is constructed by applying an intensity scale correction to each elementary image, then by attributing to points of the synthetic image which are in the analysis window (FA): - for each of said points, a maximum value of the intensity values at the same point among the corrected elementary images; or - for each of said points, a weighted average of said intensity values of the corrected elementary images, relative to said point and raised to the power n, n being a non-zero and fixed positive integer.

11. Device for displaying a multispectral image (10) on a display unit, the multispectral image (10) comprising several spectral images (11, 12, 13) of the same scene acquired simultaneously, the device being configured to implement the method according to one of claims 1 to 10.

12. Computer program product comprising instructions for implementing the method according to one of claims 1 to 10 when this program is executed by a processor.

Images