Method for displaying a multispectral image

Abstract

One aspect of the invention relates to a computer-implemented method for displaying a multispectral image, comprising: - selecting, within the multispectral image, an analysis window and a plurality of target areas within the analysis window; - segmenting a portion of the multispectral image into a set of classes; - for each target area: determining a respective background area; determining a Fisher projection to obtain an elementary image of the scene portion corresponding to the analysis window; - combining the elementary images to construct a synthetic image and displaying the synthetic image; wherein the background area is determined from the class to which the target area belongs. Figure to be published with the abstract: Figure 3

Description

Title of the invention: Method for displaying a multispectral image. TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of monitoring an environment.

[0002] In particular, the present invention relates to a method for visualizing a multispectral image. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] Monitoring an environment is a common task, particularly for detecting adverse intrusions. Such monitoring presents particular difficulties when carried out in a terrestrial environment. Indeed, a terrestrial environment such as a rural landscape can contain a large number of distinct elements with irregular outlines, such as trees, bushes, rocks, buildings, road signs, etc., which complicate image interpretation and the search for intruders. Furthermore, in certain circumstances, an intruder may be camouflaged to make its detection in the landscape more difficult. Typically, such camouflage is effective against observation in visible light, particularly for wavelengths between 0.45 pm (micrometers) and 0.65 pm, and especially around 0.57 pm, which corresponds to the maximum sensitivity of the human eye.

[0004] To successfully detect the intruder, also called the "target," despite a complex landscape and possible camouflage, it is known to perform multispectral observation of the environment. Such multispectral observation involves the simultaneous use of several images of the same landscape, each acquired in different spectral bands, so that a target that would not appear distinctly in images captured in certain spectral bands is revealed by images corresponding to other spectral bands. Each spectral band can be narrow, with a wavelength range extending over a few nanometers or tens of nanometers, or it can be wider.It is thus known that observation in the wavelength range between 0.6 pm and 2.5 pm can be effective in revealing a target in a vegetation environment, while the target is effectively camouflaged against detection by observation in the range of light visible to the human eye.

[0005] Reference numeral 10 in [Fig. 1] designates globally such a multispectral image, formed from several individual images 11, 12, 13 which were acquired simultaneously for the same scene. For example, the individual images 11, 12, 13 may have were acquired by imaging channels arranged in parallel, activated at the same time and having the same optical input field. Alternatively, the individual images 11, 12, 13 may have been acquired from a single imaging channel allowing the acquisition of several images associated with different spectral bands. However, each image 11, 12, 13 was acquired by selecting only a portion of the radiation from the scene, which is separated from each portion of the radiation used for another of the images 11, 12, 13. This separation is performed based on the known wavelength X of the radiation, so that each of the images 11, 12, 13, called a spectral image, was acquired with respective radiation whose wavelength belongs to a distinct interval, preferably without overlap with any of the intervals of the other spectral images. The number of spectral images 11, 12, 13 can be arbitrary. Such a multispectral image can also be called hyperspectral, depending on the number of spectral images that compose it and the width of each of their wavelength intervals. For example, Xb, X2, X3 denote central values ​​for the respective wavelength intervals of the spectral images 11, 12, 13.Furthermore, x and y denote two spatial dimensions, which are common to all spectral images 11, 12, 13.

[0006] Each spectral image 11, 12, 13 can be processed from a file that is read from a storage medium, or from a digital stream that is produced by a camera at a video frame rate. Depending on the case, the image data can be raw data produced by one or more image sensors, or data already processed for certain operations such as cropping the spectral images relative to each other, correcting over- or underexposure, etc.

[0007] However, such multispectral detection may still be insufficient to allow a monitoring operator to detect the presence of a target in a terrestrial environment. Indeed, in certain circumstances, none of the images associated separately with the spectral bands shows the target sufficiently distinctly for the monitoring operator to detect the target in these images, given the available observation time.

[0008] For such situations, it is known to improve the efficiency of target detection by presenting the operator with an image constructed by Fisher projection. Such a method is described in particular in the article "Some practical issues in anomaly detection and exploitation of regions of interest in hyperspectral images," by F. Goudail et al., Applied Optics, Vol. 45, No. 21, pp. 5223-5236. According to this method, the image presented to the operator is constructed by combining, at each point of the image, called a pixel, the intensity values ​​acquired separately for several spectral bands, in order to optimize the contrast of the resulting image. Theoretically, this image construction involves projecting, for each pixel, the vector of intensities captured for the selected spectral bands onto an optimal direction in the multidimensional space of spectral intensity values. This optimal projection direction can be determined from the covariance matrix of spectral intensities, estimated over the entire image field. This essentially amounts to finding the maximum correlation between the intensity variations present in the different images captured for the selected spectral bands. The contrast of the image presented to the operator is thus at least equal to that of each separate spectral image, making target detection by the operator both more efficient and more reliable.Alternatively, the optimal projection direction can be sought directly using a standard optimization algorithm, to maximize image contrast by varying the projection direction in the multidimensional space of spectral intensities.

[0009] It is common practice to restrict contrast enhancement by Fisher projection to a spatial window within the multispectral image, called the analysis window, which is smaller than the entire image. The analysis window, denoted FA in [Fig. 1], is then applied identically to all the spectral images 11, 12, 13 that constitute the multispectral image 10. This restriction to the analysis window FA applies both to the portion of the multispectral image 10 from which the Fisher projection is determined and to the portion of the multispectral image 10 to which this Fisher projection is applied. This results in a significant reduction in the computation time required to obtain enhanced contrast, which is advantageous for real-time execution during an environmental monitoring mission.A visualization image is then presented to the monitoring operator, in which the content of the multispectral image inside the analysis window is displayed as it results from the Fisher projection.

[0010] To determine the Fisher projection, it is necessary to define a target area within the analysis window and an associated background area. Figure 2 shows a common arrangement of the target area in the FA analysis window: the target area, denoted T, is located in the center of the FA analysis window and has a shape that can be a square a few pixels on each side, for example, 3x3 or 7x7 pixels. The background area, denoted B, surrounds the target area T, separated from it by an intermediate area, or guard area, denoted G. The background area B can be externally bounded by the FA analysis window itself.

[0011] The contrast intended to be maximized by the Fisher projection is calculated between the content of the multispectral image in the target area and the content of the same multispectral image in the background area. Thus, to determine the projection In Fisher's method, the intensities of the spectral images are spatially averaged over the target area on the one hand, and over the background area on the other.

[0012] Consequently, the effectiveness of the Fisher projection in enhancing contrast may be reduced in the following situations, in particular:

[0013] / i / case of a variegated target, that is to say, one which has spectral characteristics different between neighboring parts of the target that are simultaneously contained within the target area; or

[0014] / ii / case of several targets each of small dimensions which would be contained simultaneously in the target zone, or of which some would be in the target zone and others in the background zone.

[0015] To improve the effectiveness of the Fisher projection in enhancing contrast, particularly in such unfavorable situations, patent FR3037428B1 proposes defining a plurality of target areas within the FA analysis window. The content of the multispectral image 10 within the FA analysis window is then processed to enhance the contrast between each target area and its associated background area, independently of the other target areas. This processing produces an elementary image for the content of the multispectral image 10 within the FA analysis window, which is distinct for each target area. These elementary images are then combined or merged into a synthetic image, which is an enhanced representation of the content of the multispectral image 10 within the FA analysis window.This synthetic image can then be presented to the monitoring user, for example displayed on a screen, so that they can make a decision or trigger an action.

[0016] While the method described in patent FR3037428B1 significantly improves contrast in situations such as those mentioned above, it nevertheless struggles to detect certain targets, particularly when these are located in complex and highly heterogeneous background areas, such as forest landscapes. In such cases, the method does not always succeed in identifying the backgrounds useful for "decamouflage" in relation to the targets that one seeks to detect.

[0017] There is therefore a need for a more robust method of multispectral image visualization, which makes it possible to enhance the contrast around the target, even when the latter is located in a region of the image with a highly heterogeneous background. Summary of the invention

[0018] The invention provides a solution to the problems mentioned above by segmenting the multispectral image into a set of K classes, with K greater than or equal to 1, and by determining, for each target area, the background area associated with it as a function of the class to which the target area belongs. Thus, the background area corresponds to the same class of the image, and therefore includes "similar" pixels. This improves the contrast between a potential anomaly present in the analysis window and the background, thus providing the operator with an image in which a potential anomaly appears more distinctly.

[0019] One aspect of the invention thus relates to a computer-implemented method for displaying a multispectral image on a display unit, the multispectral image comprising several spectral images of the same scene acquired simultaneously, the method comprising:

[0020] - select, within the multispectral image, an analysis window corresponding to a part of the scene and a plurality of target areas in the analysis window;

[0021] - for each target area in the analysis window: • determine a background area associated with each target area; • determine a Fisher projection from intensity values ​​of spectral images, said Fisher projection being a linear combination of spectral images that optimizes a contrast between said each target area and the background area that is associated with said each target area; • apply the determined Fisher projection to the intensity values ​​of the spectral images within the analysis window, so as to obtain an elementary image of the scene portion corresponding to the analysis window;

[0022] - combine the elementary images obtained to construct a synthetic image relating to the part of the scene corresponding to the analysis window;

[0023] - display an image of the scene on the display unit, using in the window analysis of the constructed synthetic image;

[0024] the process being characterized in that it further comprises:

[0025] - segment the multispectral image into a set of classes;

[0026] and in that, for each target zone in the analysis window, the determination of the background zone associated with said each target zone comprises:

[0027] - select a primary background area associated with said target area and separated from said target area by an intermediate area so that the target area is not contiguous with the background area which is associated with said target area;

[0028] - determine, from the set of classes, the class comprising said zone target;

[0029] - determine the background area associated with said each target area as a intersection between the primary background area associated with each target area and the selected class.

[0030] In a known manner, a multispectral image is formed from several spectral images of the same scene which were acquired at the same instant respectively in separate spectral intervals.

[0031] By analysis window, we mean an area corresponding to a portion of the scene. Typically, if the multispectral image comprises a grid of pixels, the analysis window defines a subset of pixels within the pixel grid. For example, the analysis window is a rectangle within the multispectral image. It should be noted that the analysis window is generally of strictly smaller dimensions than the multispectral image, but it can also have the same dimensions.

[0032] The term "target area" means an area within the analysis window, smaller than the analysis window itself (for example, a 3x3, 5x5, or 7x7 pixel square). The target area corresponds to an area whose contrast is optimized relative to a background, in order to highlight the possible presence of an anomaly.

[0033] Each target area is associated with a respective background area against which the contrast is analyzed. According to the present invention, the background area is determined to be homogeneous (and to correspond to the background against which the target area is positioned). To this end, a so-called "primary" background area is first determined. The term "primary background area" is used to facilitate distinction from the (final) "background area" on the basis of which the Fisher projection is determined.

[0034] This primary background area is not contiguous with the target area. By "is not contiguous," it is understood that no pixel of the primary background area is adjacent to a pixel of the target area. By construction, the background area (i.e., the "final" background area) is also not contiguous with the target area.

[0035] Thus, each elementary image optimizes the contrast with respect to one of the target areas, separately from the other target areas, by using a homogeneous background area. A target or a portion of a target can therefore be revealed in the intermediate image, even when the image presents different background areas that are very heterogeneous with each other.

[0036] For example, the segmentation of the multispectral image into a set of classes can be carried out by a k-means type method.

[0037] In some embodiments, the method further comprises:

[0038] - determine, from the set of classes, a class called the background class, in function of the classes determined for the plurality of target areas;

[0039] - apply a contrast enhancement method to a portion of the image of synthesis corresponding to the determined background class;

[0040] in which the image of the scene displayed on the display unit is constructed using in the analysis window the synthetic image after the application of the contrast enhancement method.

[0041] In these embodiments, a background class is defined as the class corresponding to the background against which the potential target is located. For example, the background class may correspond to the class associated with the largest number of target areas among the plurality of target areas. Thus, the contrast is enhanced in this area, making a potential target stand out more against the background against which it is placed.

[0042] For example, the contrast enhancement method may be a histogram stretching method.

[0043] In addition, the method may include:

[0044] - decrease intensity values ​​in the synthetic image deprived of said portion corresponding to the determined class of background;

[0045] wherein the image of the scene displayed on the display unit is constructed using in the analysis window the synthetic image after the application of the contrast enhancement method and the reduction of intensity values.

[0046] By "synthetic image deprived of said portion corresponding to the background class", it is understood that the portion or portions of the synthetic image whose pixels are associated with a class other than the background class.

[0047] Thus, the portion of the image that does not include the potential target is "darkened", so that the potential target appears more distinctly.

[0048] In embodiments, at least some target areas among the plurality of target areas have overlaps with each other in the analysis window.

[0049] Such overlaps may be advantageous in the case of variegated targets, so that certain target zone boundaries are closer to the boundaries between parts of the target that have different spectral characteristics.

[0050] In embodiments, the target areas of the plurality of target areas are aligned in the analysis window.

[0051] In particular, the scene may be a portion of a terrestrial landscape, and the target areas of the plurality of target areas may be aligned parallel to a direction that is vertical or horizontal with respect to the terrestrial landscape.

[0052] Indeed, a terrestrial landscape is naturally traversed by an observer by moving his direction of gaze vertically or horizontally.

[0053] In these embodiments, each class can correspond to a respective landscape motif. For example, the set of classes can include: forest, field, road.

[0054] In other embodiments, the target areas of the plurality of target areas are arranged in the analysis window so as to form a cross or a cuboid.

[0055] In embodiments, a common primary background area is associated with all target areas in the analysis window.

[0056] Alternatively, distinct primary background areas are respectively associated with the target areas of the analysis window, and for all target areas, a relative arrangement of the target area and its associated primary background area is constant. In other words, all target areas are arranged in the same way with respect to their respective primary background areas (for example, each target area may be a small square centered inside its respective primary background area, which is, for example, a larger square).

[0057] According to another alternative, distinct primary background areas are respectively associated with the target areas of the analysis window, with an arrangement of said primary background areas such that no primary background area has an overlap with any of the target areas.

[0058] In particular, the primary background zone associated with each target zone results from a selection of at least one primary background zone segment from among several background zone segments that are likely to be associated with said target zone, said selection being made to reduce discrepancies existing between respective spectral signatures of said primary background zone segments selected for the same target zone, and the primary background zone associated with each target zone is a union of the primary background zone segments selected for said target zone.

[0059] In embodiments, for each target area of ​​the analysis window, the Fisher projection is determined by weighting intensity values ​​of the spectral images, in the background area which is associated with said target area, with coefficients whose values ​​are decreasing functions of a distance between the target area and a location in the background area to which the intensity values ​​of the spectral images correspond.

[0060] Thus, an element of the scene that is further away from the target area is less likely to alter a spectral homogeneity of the background area, for the calculation of contrast, than an element of the scene that is closer to this target area.

[0061] In some embodiments, the synthetic image is constructed by applying an intensity scaling correction to each elementary image, and then assigning to points of the synthetic image that are in the analysis window:

[0062] - for each of said points, a maximum value of the intensity values ​​at the same point among the corrected elementary images; or

[0063] - for each of said points, a weighted average of said intensity values corrected elementary images, relative to said point and raised to the power of n, n being a fixed non-zero positive integer.

[0064] In embodiments, several analysis windows are selected successively in the multispectral image, and, for each analysis window among the several selected analysis windows: a synthetic image relating to the part of the scene contained in said analysis window is constructed and an image of the scene is displayed on the display unit using in said analysis window the synthetic image constructed.

[0065] Another aspect of the invention relates to a device for displaying a multispectral image on a display unit, the multispectral image comprising several spectral images of the same scene acquired simultaneously, the device being configured to implement a method as defined above.

[0066] A computer program, implementing all or part of the process described above, installed on pre-existing equipment, is in itself advantageous.

[0067] Thus, the present invention also relates to a computer program product comprising instructions to implement the process described above, when this program is executed by a processor.

[0068] This program may use any programming language (for example, an object-oriented language or other), and may be in the form of interpretable source code, partially compiled code or fully compiled code.

[0069] The [Fig.3] described in detail below can form the flowchart of the general algorithm of such a computer program.

[0070] The invention and its various applications will be better understood by reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES

[0071] Other features and advantages of the invention will become apparent from the description, which can be read in conjunction with the figures. These figures are provided for illustrative purposes only and are not intended to limit the scope of the invention.

[0072] [Fig.1] Fig.1 represents a schematic representation of a multispectral image to which the invention can be applied;

[0073] [Fig.2] The [Fig.2] represents an example of the segmentation of the analysis window used to determine a Fisher projection of the multispectral image according to the prior art;

[0074] [Fig.3] The [Fig.3] represents a flowchart of a method for displaying a multispectral image according to an embodiment of the invention;

[0075] [Fig.4a], [Fig.4b], [Fig.4c], [Fig.4d], [Fig.4e], [Fig.4f] Figures 4a to 4f represent examples of arrangements of primary target zones and background zones;

[0076] [Fig.5a], [Fig.5b], [Fig.5c], [Fig.5d] Figures 5a to 5d represent an example of determining a background zone Bi associated with a target zone Ti;

[0077] [Fig.6a], [Fig.6b], [Fig.6c] Figures 6a to 6c represent an example of processing the synthetic image obtained;

[0078] [Fig.7] Fig.7 represents a multispectral image display device according to an embodiment of the invention.

[0079] For clarity, the dimensions of the elements shown in these figures do not correspond to actual dimensions or actual dimension ratios. Furthermore, identical reference numerals shown in different figures designate identical elements or elements having identical functions. DETAILED DESCRIPTION

[0080] [Fig.3] The [Fig.3] represents a flowchart of a method for displaying a multispectral image according to an embodiment of the invention.

[0081] The method described below with reference to [Fig. 3] can be applied to a multispectral image such as that shown in [Fig. 1]. The multispectral image 10 can be read from a data storage medium on which it was recorded after being acquired. This constitutes deferred processing of the multispectral image. Alternatively, the multispectral image 10 can be part of a video stream that is acquired and viewed in real time by a monitoring operator.

[0082] The multispectral image acquisition device(s), an image processing unit capable of implementing the method of the invention, and an image display unit may be mounted on board a vehicle, for example, a ground vehicle used for surveillance, or integrated into individual viewing equipment such as a pair of binoculars. In other contexts of use of the invention, the multispectral image acquisition device(s) 10 may be mounted on board an aircraft, for example, an aerial drone, while the image processing and display units may be located in a static and remote control and observation post. In such a context, the acquired multispectral images are transmitted to the processing unit by electromagnetic signals in a manner that is known per se.

[0083] In the following, each image 11, 12, 13 which corresponds separately to one of the spectral bands of the multispectral image 10, is called a spectral image.

[0084] With reference to [Fig. 3], the display method includes a step 110 in which the FA analysis window is selected in the multispectral image 10. The The FA analysis window can be located at a fixed position relative to an external spatial boundary frame of the image 10, or at a position selected by the monitoring operator. For example, the FA analysis window can be placed at the center of the multispectral image 10. The dimensions of the FA analysis window can vary, depending in particular on the number and size of the target areas, described below. For example, the FA analysis window can be a 101 x 101 pixel square, but other sizes are possible.

[0085] Several target areas, denoted T1, T2, T3, T4, are then selected within the FA analysis window during a step 120. It is noted that in the context of the invention, several target areas are selected for the same analysis window.

[0086] The target areas T1, T2, T3, T4 can be aligned vertically within the FA analysis window, as illustrated in Figures 4a-4c and 4f. Such vertical alignment is particularly suitable for monitoring a terrestrial landscape by subsequently performing a horizontal displacement of the FA analysis window within the landscape.

[0087] Alternatively, the target areas T1, T2, T3, T4 can be aligned horizontally in the FA analysis window, for example, when this analysis window is subsequently moved across the landscape in a direction contained within a vertical plane. Such a horizontal arrangement of the target areas in the analysis window may be more suitable for multispectral images captured from an aerial drone.

[0088] Alternatively, the target areas can be grouped into a block, as shown in [Fig. 4d], or to form a cross, as shown in [Fig. 4e]. Such configurations are suitable for tracking targets that are likely to move relatively slowly across the landscape, between two multispectral images acquired and then processed successively according to the invention.

[0089] Finally, for each general arrangement of target areas T1, T2, T3, T4 in the FA analysis window, two target areas that are close together may be spaced apart from each other, as in Figures 4a, 4b and 4f, or have partial overlaps, as in Figures 4c-4e, or be contiguous without overlap or intermediate space.

[0090] For example, each target area T1, T2, T3, T4 can be a 3x3, 5x5 or 7x7 pixel square.

[0091] In a step 130, a segmentation of the multispectral image 10 into a set of K classes Ak is implemented, where K is a natural number between 1 and the number N of pixels in the image. For example, each pixel of the multispectral image 10 can be associated with a respective class. Such a segmentation can be implemented by a prior art image segmentation method. For example, when The image represents a terrestrial landscape; the number of K classes can be equal to 3, and the different classes can be: "forest", "field", "road". When the image is homogeneous (a single motif), the number of K classes can be equal to 1. Of course, these examples are in no way limiting to the invention.

[0092] It is noted that in step 130, the segmentation can be performed only on a portion of the multispectral image (and not the entire multispectral image), said portion including the FA analysis window (i.e. the portion represents at least the part of the scene covered by the FA analysis window).

[0093] At step 130, any known image segmentation method can be used, for example a data partitioning method such as a k-means method, a method based on the use of neural networks, a method based on histogram thresholding such as the Otsu method, etc.

[0094] Thus, at the end of the segmentation step 130, the multispectral image 10 is subdivided into K classes Ak, which means that each pixel of the multispectral image 10 is associated with a respective class Ak, where k G [[ 1, ÆJ.

[0095] It is noted that step 130 can be implemented before steps 110 and 120, or after step 142 described below.

[0096] During steps 140, 150, and 160, the content of the multispectral image 10 in the FA analysis window is processed to improve the contrast between each target area and its associated background area, independently of the other target areas. This processing produces an elementary image for the content of the multispectral image 10 in the FA analysis window, which is distinct for each target area T1, T2, T3, T4. Thus, for each target area selected in step 120 within the FA analysis window, a sequence of steps 140, 150, and 160 is implemented independently of the other selected target areas. The different sequences of steps 140, 150, and 160 for the different target areas can be implemented successively or in parallel.

[0097] In step 140, a background area associated with the target area Ti, for which the sequence of steps 140, 150, and 160 is being executed, is determined. Generally, this background area is separated from the target area Ti by an intermediate guard area, so that certain small targets that are partially contained within the target area Ti do not extend into the background area. For example, the guard area may have a thickness of at least 15 pixels between the target area Ti and the background area.

[0098] Step 140 of determining the background zone associated with the target zone Ti considered includes a first step 142 of selecting a background zone called primary PBi (cf. [Fig.5c]) associated with the target zone Ti considered.

[0099] In general, the primary background area PBi associated with the target area Ti is separated from the target area Ti by an intermediate area Gi such that the target area Ti is not contiguous with the primary background area PBi. In other words, no pixel of the target area Ti is directly adjacent to a pixel of the primary background area PBi.

[0100] According to a first example illustrated in [Fig. 4a], a different primary background zone PBI, PB2, PB3, PB4 can be associated with each target zone T1, T2, T3, T4 according to relative arrangements that are identical for all target zones. For example, each target zone T1, T2, T3, T4 can be surrounded by the corresponding primary background zone PBI, PB2, PB3, PB4, the latter having inner and outer boundaries that can be square, with an intermediate guard zone, denoted Gl, G2, G3, G4 for each pair of target and background zones {Ti,PBi}.

[0101] According to a second example illustrated in Figures 4b to 4e, a single primary background area, denoted PB, can be associated with all target areas T1, T2, T3, T4. Preferably, the primary background area PB can be arranged around all target areas T1, T2, T3, T4 with an intermediate guard area, denoted G. An advantage of such a second arrangement is to prevent one of the target areas from overlapping the primary background area associated with another of the target areas. Thus, for a given position of the analysis window FA, a scene element located within the analysis window FA is processed to calculate the contrast in the multispectral image, either as a potential target within at least one of the target areas, or as a scene background, in a way that is consistent for all target areas.

[0102] Other possibilities for associating a primary background zone PBi with each target zone T1, T2, T3, T4 are illustrated in Figure 4f and described below. Background zone segments, denoted SB1-SB14 respectively, are defined around the target zones T1, T2, T3, T4 with an intermediate guard zone G. Then, the primary background zone associated with each target zone can be formed by the union of several of the SB1-SB14 segments assigned to that target zone.

[0103] According to a third example for defining the primary background zone associated with each target zone, each target zone is initially assigned those primary background zone segments that are spatially closest to it. For example, for the target zone configuration of [Fig. 4f], the primary background zone PBI associated with the target zone T1 can be the union of segments SB1-SB5 and SB13-SB14, the primary background zone PB2 associated with the target zone T2 can be the union of segments SB4-SB6 and SB12-SB14, the primary background zone PB3 associated with the target zone T3 can be the union of segments SB5-SB7 and SB11-SB13, and the primary background zone PB4 associated with the target zone T4 can be the union of segments SB6-SB12.

[0104] According to a fourth example, selecting the primary background area associated with each target area ensures that the content of the multispectral image in that background area is sufficiently spectrally homogeneous. To achieve this, a spectral signature can be determined for each of the primary background area segments SB1-SB14. Such a signature can, for example, be a vector consisting of the average intensity values ​​of each spectral image 11, 12, 13 calculated over all the pixels of that primary background area segment. Such a vector therefore has a distinct coordinate for each spectral image. The signatures of all the primary background area segments are then compared, and the segment(s) whose signature(s) is / are furthest from those of the other segments is / are discarded.Only the primary background zone segments that remain after such discrimination can be selected to constitute the primary background zone associated with the target zone.

[0105] According to this example, the spectral signatures of the primary background zone segments are given by the following formula, expressed for the primary background zone segment SBj:

[0106] 5, -JV R (image point) Jf< j "image points of the segment Slif bf

[0107] where j is an integer that counts the predefined primary background zone segments, denoted SBj (j varies from 1 to 14 in the example in Figure 4f), Sj is the spectral signature of the primary background zone segment SBj, k is an integer that counts the spectral images 11, 12, 13 that constitute the multispectral image 10, Ik is the intensity of the spectral image k at the pixel considered, Nj is the number of pixels in the primary background zone segment SBj, and S^ is the k-th coordinate of the signature vector Sj. Then, a separation angle between two of the signatures can be calculated according to the formula:

[0108] (h ;■ = arccos

[0109] where ($ '1 denotes the dot product of the signature vectors Sj and 5'., | Sj | denotes V j J) the norm of the signature vector Sj, arccos is the inverse of the cosine function, and 6 jj' is the separation angle between the two signature vectors of the primary background zone segments SBj and SBj'. The separation angles thus obtained for all pairs of primary bottom zone segments j are compared between them, and those of the primary bottom zone segments SBj which form only separation angles Ojj' which are small, with the majority of the other primary bottom zone segments SBj', are retained to participate in the primary bottom zones which are respectively associated with the target areas Ti.

[0110] Generally, for the invention, it is not necessary for each primary background area to be contained within the FA analysis window, but some of the primary background areas, or the primary background area common to all target areas for the relevant implementations, may extend beyond the FA analysis window. Furthermore, it is specified that the primary background area associated with one of the target areas does not necessarily completely surround the target area, but may be located only on some of its sides. By way of example, each primary background area or primary background area segment may have a dimension or thickness in the multispectral image that is greater than or equal to 3 pixels, in a spatial direction that is vertical or horizontal with respect to the scene.

[0111] Then, at a step 144, the class Ak, among the set of classes defined in step 130, to which the considered target area Ti belongs is determined.

[0112] In step 146, the background area Bi (see [Fig. 5d]) associated with the considered target area Ti is determined to be the intersection between the primary background area selected in step 142 and the selected class Ak. In other words, the background area Bi comprises the pixels belonging to the primary background area that are associated with the selected class Ak.

[0113] Figures 5a to 5d illustrate steps 130 and 140 of the process of [Fig. 1].

[0114] Fig. 5a represents an example of a multispectral image 10 of a terrestrial landscape comprising two types of patterns 10a, 10b. For example, image 10 represents a forest edge landscape comprising a forest pattern 10a and a field pattern 10b.

[0115] Figure 5b represents the result of a segmentation obtained at the end of step 130. The segmented image comprises two areas corresponding to respective classes A1 and A2. Here, class A1 is the "forest" class and class A2 is the "field" class. In this example, the number of classes is 2, but it is understood that the number of classes used for the segmentation can be equal to 1 or greater than 2 (for example, between 3 and 10), or even much greater than 2 (for example, several dozen classes). Furthermore, in this example, each segmented area is a single "block," that is, it comprises a set of pixels in which each pixel is at least adjacent to another pixel in the area (in other words, in this example, no area comprises separate groups of pixels). This configuration is, of course, not mandatory. There can be several separate areas associated with the same class.In general, following the segmentation implemented in step 130, each pixel of the image is associated with a respective class (here, class Al or class A2).

[0116] Figure 5c represents a target area Ti and a primary background area PBi associated with the target area Ti (as selected in step 142). In this example, the primary background area PBi comprises pixels between two squares of different sizes. centered on the target area Ti, such that the smaller of the two squares has a chosen size such that no pixel of the target area Ti is in contact with a pixel of the smaller of the two squares.

[0117] Next, it is determined to which class Al, A2 the target area Ti belongs (step 144). This is done by looking at the class Al, A2 associated with the pixels of the target area Ti. In this example, all the pixels of the target area Ti belong to class Al. Thus, the class determined in step 144 is class Al.

[0118] It is noted that, when the target area Ti comprises several pixels, it may happen that some pixels of the target area Ti belong to one class and other pixels belong to another class. In this case, the class Al, A2 to which the target area Ti belongs can be determined from a predefined rule. An example of such a predefined rule is to select the class to which the pixel located at the center of the target area Ti belongs. Another example is to select the class to which the largest number of pixels in the target area Ti belong.

[0119] Figure 5d represents the "final" background area Bi, determined in step 146. The background area Bi is the intersection between the primary background area PBi and the image region(s) associated with the selected class (here, Al). In other words, the background area Bi comprises the pixels of the primary background area PBi associated with the selected class AL.

[0120] Referring again to [Fig. 3], in step 150, the Fisher projection that maximizes the contrast between each target area Ti and its associated background area Bi is determined. Such a Fisher projection is known and is not described in further detail here.

[0121] According to one embodiment of the invention, the Fisher projection can be determined by weighting the intensity of each pixel in a background area Bi associated with a respective target area Ti as a function of the distance between that pixel and the target area Ti (for example, from its center). For example, if / &(*, j) denotes the intensity value of the pixel with coordinates (x, y) in the spectral image k, the average weighted background intensity 7 to be used for this spectral image k during Vn yTi] The determination of the Fisher projection for the target area Ti and the associated background area Bi can be:

[0122] —j —x !

[0123] with

[0124] where xt and yn are the coordinates of the center of the target area Ti and (y) are the coordinates of the background area pixels Bi. Thus, extraneous scene elements contained in the background area interfere less with the contrast enhancement produced by the Fisher projection when they are further from the target area. In general, the weighting coefficient values ​​can vary according to any decreasing function of the distance between the pixel in the background area and the target area. Such weighting can also be used to calculate the signatures of the background area segments in the fourth example presented above.

[0125] In step 160, for the target area Ti considered, the Fisher projection obtained for this target area in step 150 is then applied to the multispectral image 10 inside the FA window. An elementary image is thus obtained, which represents the content of the multispectral image 10 in the FA analysis window with maximum contrast.

[0126] During a test 165, it is checked whether a respective elementary image has been determined for all target areas Ti. If not, the process resumes at step 140 for a target area Ti for which an elementary image has not yet been determined.

[0127] If all the elementary images have been determined, the process continues at step 170.

[0128] In step 170, all the elementary images obtained from all the target areas Ti (one elementary image per target area Ti) in steps 160 are combined to construct a single synthetic image. Such a combination can be performed by retaining, for each pixel in the FA analysis window, the maximum intensity at that pixel among all the elementary images (i.e., all the intensities of that pixel in the elementary images are examined, and the highest is selected). Alternatively, the intensities of all the elementary images at that point can be raised to a fixed, common, and positive power p, and the intensity of the synthetic image at the pixel in question can be equal to the average of these elementary image intensities raised to the power p.

[0129] Then, the synthetic image obtained in step 170 in the FA analysis window is displayed to the monitoring operator in step 180. For this purpose, it can be displayed on a screen, or more generally on a display unit intended for visual observation. Optionally, to facilitate the operator's location within the scene, the displayed image can be supplemented outside the FA analysis window by one of the spectral images, for example, a spectral image corresponding to a thermal radiation acquisition channel. Alternatively, the image that is displayed The image can be supplemented outside the FA analysis window by a combination of several spectral images that correspond to different colors in the visible range, to reconstruct a color daytime view outside the FA analysis window. Finally, it is also possible to supplement the image presented to the operator outside the FA analysis window with an optically formed scene image transmitted directly to the display unit.

[0130] In some embodiments, an optional step 175 can be implemented after step 170 of determining the synthetic image and before the step of displaying this synthetic image.

[0131] In this optional step 175, a contrast enhancement method is applied to the synthetic image obtained in step 170. More specifically, a contrast enhancement method is applied to the portion of the synthetic image associated with class Al, A2 determined in step 144 for target areas Ti.

[0132] It is noted that the classes determined in step 144 may differ depending on the target areas considered. In this case, step 175 includes a preliminary determination of a class from among the set of classes in the image. For example, the determined class may be the class associated with the largest number of target areas Ti among all target areas Ti.

[0133] Thus, the contrast is increased in the region of interest of the image, i.e., in the region containing the potential anomaly (the target). If an anomaly is indeed present, its contrast with the background is enhanced (it appears brighter than the background), making it easier for the operator to visualize. The contrast enhancement method can be any method known in the prior art, for example, histogram stretching.

[0134] Furthermore, in step 175, it is possible to decrease the intensity values ​​of the pixels in the elementary image that correspond to classes different from that in which the contrast enhancement is applied (the image regions outside the region of interest). This makes it possible to "darken" the part of the background that does not contain the potential anomaly, and thus to make the potential target stand out more, appearing brighter.

[0135] An example of implementing step 175 is shown in Figures 6a to 6c. Figure 6a corresponds to the synthetic image obtained at the end of step 160. In this example, the image has been segmented into two classes A1 and A2, and the target area Ti under consideration belongs to class A1. Figure 6b represents the synthetic image after applying contrast enhancement to the area corresponding to class A1. Figure 6b shows that the target stands out more against the background pattern behind it. Figure 6c represents the image of Figure 6b after reducing the intensity values ​​in the area corresponding to class A2. The target stands out. clearly distinguishes between the two zones associated with classes Al, A2, which allows the operator to detect it (and confirm that it is indeed an anomaly) more easily.

[0136] The synthetic image thus processed can then be displayed in step 180 as described above.

[0137] It is understood that the dimensions, shapes, and positions of the target and background areas given as examples in the detailed description above can be modified depending on the application and implementation context. In particular, they can be determined from additional data, such as the distance and / or size of a target being sought in the imaged scene.

[0138] Figure 7 represents a multispectral image display device according to a method of embodiment of the invention.

[0139] In this embodiment, the device includes a computer 700, comprising a memory 701 for storing instructions enabling the implementation of the process, the received images and temporary data for carrying out different steps of the processes described above.

[0140] The computer 700 further comprises a circuit 702. This circuit can be, for example, a processor capable of interpreting instructions in the form of a computer program, an electronic card whose steps of the process of the invention are described in silicon, or a programmable electronic chip such as an FPGA chip (for "Field-Programmable Gate Array" in English).

[0141] The computer 700 includes an input interface 703 for receiving, in particular, a multispectral image, an analysis window, and / or a target area, and an output interface 704 for providing the synthesized image to a display unit. The display unit may be external to the computer 700, or it may be part of the computer 700. Thus, the computer may include a display unit such as a screen 705. The computer 700 may also include, to allow easy interaction with a user, a keyboard 706 and / or a mouse. Of course, the keyboard and mouse are optional, particularly in the case of a computer in the form of a touch tablet, for example.

[0142] Furthermore, the functional diagram shown in [Fig. 3] is a typical example of a program in which certain instructions can be executed using the described device. As such, [Fig. 3] can be considered the flowchart of the general algorithm of a computer program within the meaning of the invention.

[0143] Of course, the present invention is not limited to the embodiments described above by way of example.

Claims

Demands

1. A computer-implemented method for displaying a multispectral image (10) on a display unit, the multispectral image (10) comprising several spectral images (11, 12, 13) of the same scene acquired simultaneously, the method comprising: - select (110, 120), within the multispectral image (10), an analysis window (FA) corresponding to a part of the scene and a plurality of target areas (Ti) in the analysis window (FA); - for each target zone (Ti) in the analysis window (FA): • determine (140) a background zone (Bi) associated with each target zone (Ti); • determine (150) a Fisher projection from intensity values ​​of spectral images (11, 12, 13), said Fisher projection being a linear combination of spectral images (11, 12, 13) that optimizes a contrast between said each target area (Ti) and the background area (Bi) that is associated with said each target area (Ti); • apply (160) the Fisher projection determined to the intensity values ​​of the spectral images (11, 12, 13) included in the analysis window (FA), so as to obtain an elementary image of the part of the scene corresponding to the analysis window; - combine (170) the elementary images obtained to construct a synthetic image relating to the part of the scene corresponding to the analysis window (FA); - display (180) an image of the scene on the display unit, using in the analysis window (FA) the constructed synthetic image; the process being characterized in that it further comprises:

2.

3. - segment (130) at least a portion of the multispectral image (10) into a set of classes (Al, A2), said portion including the analysis window; and in that, for each target zone (Ti) in the analysis window (FA), the determination of the background zone (Bi) associated with said each target zone (Ti) comprises: - select (142) a primary background zone (PBi) associated with said each target zone (Ti) and separated from said target zone by an intermediate zone so that the target zone (Ti) is not contiguous with the primary background zone (PBi) which is associated with said each target zone (Ti); - determine (144), among the set of classes (Al, A2), the class (Al) comprising said each target zone (Ti); - determine the background zone (Bi) associated with said each target zone (Ti) as an intersection between the primary background zone (PBi) associated with said each target zone and the selected class (Al). A method according to the preceding claim, further comprising: - determine, from the set of classes, a class called background class, based on the classes determined for the plurality of target areas (Ti); - apply (175) a contrast enhancement method to a portion of the synthetic image corresponding to the determined background class; in which the image of the scene displayed (180) on the display unit is constructed using in the analysis window the synthetic image after the application of the contrast enhancement method. A method according to the preceding claim, further comprising: - decrease (175) the intensity values ​​in the synthetic image deprived of said portion corresponding to the determined background class; in which the image of the scene displayed (180) on the display unit is constructed using in the analysis window the image of synthesis after application of the contrast enhancement method and reduction of intensity values.

4. A method according to any one of the preceding claims, wherein at least some target areas (Ti) among the plurality of target areas (Ti) have overlaps with each other in the analysis window (FA).

5. A method according to any one of the preceding claims, wherein the target areas (Ti) of the plurality of target areas (Ti) are aligned in the analysis window (FA).

6. A method according to the preceding claim, wherein the scene is a portion of a terrestrial landscape, and wherein the target areas (Ti) of the plurality of target areas (Ti) are aligned parallel to a direction that is vertical or horizontal with respect to the terrestrial landscape.

7. A method according to any one of claims 1 to 6, wherein distinct primary background zones (PB1, PB2, PB3, PB4) are respectively associated with the target zones (T1, T2, T3, T4) of the analysis window (FA), with an arrangement of said primary background zones such that no primary background zone has an overlap with any of the target zones.

8. A method according to the preceding claim, wherein the primary background zone (PB1, PB2, PB3, PB4) which is associated with each target zone (T1, T2, T3, T4) results from a selection of at least one primary background zone segment from among several background zone segments (SB1, SB2, ..., SB14) which are likely to be associated with said target zone, said selection being made to reduce discrepancies existing between respective spectral signatures of said primary background zone segments selected for the same target zone, and the primary background zone associated with each target zone is a union of the primary background zone segments selected for said target zone.

9. A method according to any one of the preceding claims, wherein, for each target zone (Ti) of the analysis window (FA), the Fisher projection is determined by weighting intensity values ​​of the spectral images (11, 12, 13) in the background zone associated with said target zone with coefficients whose values ​​are decreasing functions of a distance between the target zone and a location in the background area to which the intensity values ​​of the spectral images correspond.

10. A method according to any one of the preceding claims, wherein the synthetic image is constructed by applying an intensity scaling correction to each elementary image, and then assigning to points of the synthetic image that are in the analysis window (FA): - for each of said points, a maximum value of the intensity values ​​at the same point among the corrected elementary images; or - for each of said points, a weighted average of said intensity values ​​of the corrected elementary images, relative to said point and raised to the power n, n being a fixed non-zero positive integer.

11. A device for displaying a multispectral image (10) on a display unit, the multispectral image (10) comprising several spectral images (11, 12, 13) of the same scene acquired simultaneously, the device being configured to implement the method according to any one of claims 1 to 10.

12. Product computer program comprising instructions to implement the method according to any one of claims 1 to 10 when this program is executed by a processor.

Images