METHOD FOR PRODUCEING AN IMAGE BY MEANS OF A SATELLITE BY ONBOARD FUSION OF IMAGES TAKED BY THE SATELLITE

Description

Technical field

[0001] The present invention belongs to the field of imaging by satellites in Earth orbit, and relates more particularly to a method for generating an image of a geographical area by a satellite, as well as a satellite for implementing such a generation method. State of the art

[0002] Earth observation missions carried out by a satellite in Earth orbit consist of acquiring images of parts of the Earth's surface, that is, taking pictures of it.

[0003] Conventionally, such a satellite follows, for example, a scrolling (i.e. non-geostationary) orbit around the Earth in order to make acquisitions during its overflight of the Earth's surface. For this purpose, it includes an observation instrument associated with a predetermined spatial resolution. The spatial resolution corresponds to the size, for example in meters, of the smallest object that can be detected in a scene represented by an image resulting from an acquisition carried out by the observation instrument.

[0004] Several acquisition methods can be used. Among these, scanning methods are often used. In a more specific example, a "pushbroom" scan is used. Generally, in such a case, the observation instrument includes at least one linear sensor (or bar) arranged transversely to a direction of movement of the satellite on the ground, and the movement of the satellite makes it possible to acquire successive line images that represent adjacent portions of the Earth's surface.

[0005] However, the satellite's scrolling has an impact on the observation instrument's ability to make clear and precise acquisitions. In particular, when one wishes to make acquisitions with high spatial resolution (for example less than 1 meter or 0.5 meters), the flyover time of the portion represented by a line image acquired by the linear sensor becomes too low so that the integration time does not allow a satisfactory signal-to-noise ratio (also known as "SNR") to be achieved.

[0006] A linear sensor known as a time delay and integration or TDI sensor (acronym for the English expression "Time Delay and Integration", see for example patent applications EP 3217649 and FR 2976754) can then be used. Typically, such a TDI sensor comprises several stages of lines of acquisition cells which carry out a charge transfer from one line of acquisition cells to another in a manner synchronized with the movement of the satellite, each line of acquisition cells adding to the charges received from the previous stage the charge measured for the observed portion. Such a TDI sensor therefore makes it possible to obtain a line image benefiting from a longer integration time, proportional to the number of stages of said TDI sensor. However, synchronization errors between the charge transfer and the movement of the satellite (due to errors in the knowledge of the movement of the satellite, vibrations, etc.) lead to a loss of contrast, i.e. a degradation of the modulation transfer function (MTF).

[0007] Other acquisition methods allow the acquisition of several images of the same geographical area during the same overflight of said geographical area by the satellite. This is the case, for example, of so-called "pushframe" techniques (see for example patent US 8487996), or even so-called infinite slow motion techniques in which the ground footprint of a field of vision of the observation instrument is immobilized during acquisition, by attitude maneuvers of the satellite making it possible to compensate for the scrolling of the satellite (see for example patent FR 3107953), which rely on matrix sensors rather than linear sensors. Therefore, one possibility for improving the SNR and / or the contrast could be to carry out post-processing on the ground of the images representing the same geographical area.However, in practice such acquisitions generate a very large volume of data which proves difficult to repatriate to the ground given the current speeds on downlinks from satellites in Earth orbit. Exposition of the invention

[0008] The present invention aims to remedy all or part of the drawbacks of the prior art, in particular those set out above, by proposing a solution which makes it possible to improve on board a satellite the quality (in terms of SNR and / or FTM) of the images acquired by an observation instrument of this satellite.

[0009] To this end, and according to a first aspect, a method is proposed for generating an image by a satellite in Earth orbit, said satellite carrying an observation instrument and a processing circuit, said image representing a geographical area and being obtained by grouping partial images representing different partial areas within the geographical area, in which the partial areas are processed successively and said method comprises, for the processing of each partial area: a reception, by the processing circuit, of at least two elementary partial images acquired by the observation instrument at different respective times and representing said partial area being processed, an estimation, by the processing circuit, of offsets between pixels of said at least two elementary partial images, a fusion, by the processing circuit, of said at least two elementary partial images as a function of the estimated offsets, so as to form the partial image of said partial area being processed.

[0010] The offsets are estimated by means of a Kalman filter carrying out a prediction phase and an update phase, in which the offsets for the partial area being processed are estimated as a function of offsets predicted for said partial area being processed from the offsets updated for the partial area processed at the previous iteration of the Kalman filter, said offsets predicted for said partial area being processed being updated by comparison of the at least two elementary partial images representing the partial area being processed.

[0011] Thus, the proposed image generation method is based on the acquisition of several images representing the same geographical area. However, these images are advantageously merged, to obtain a better quality image, on board the satellite so that it is not necessary to transmit to a ground station, for post-processing, all the acquired images representing the same geographical area.

[0012] However, the capabilities of a satellite in terms of available computing power and memory are generally limited. In order to enable such fusion on board the satellite, the processing to produce the final image of better quality is broken down into several pieces, by successively merging elementary partial images representing partial areas, the union of which makes it possible to obtain an image representing the geographical area considered. The dimensions (in number of pixels) of the elementary partial images are therefore greatly reduced compared to the dimensions of the image to be generated, so that the quantity of data to be processed simultaneously during the fusion of elementary partial images representing the same partial area is limited, the different partial areas being processed successively, one after the other.

[0013] For each partial zone, the observation instrument provides at least two elementary partial images, representing the same partial zone, acquired at different times during the same overflight of said partial zone by the satellite, for different respective positions of said satellite in its orbit. In a known manner, offsets may be present between the pixels of the two elementary partial images. Indeed, due to different sources of errors (lack of knowledge of the line of sight of the observation instrument, geometric deformation of a footprint on the ground of a field of vision of the observation instrument due to the fact that the partial zone is observed from two different positions of the satellite relative to said partial zone, geometric distortion introduced by an optic of the observation instrument, etc.), pixels of different elementary partial images, which should represent the same portion of the partial area, may in practice represent slightly different respective portions of said partial area. The difference between these different respective portions results in a shift between the corresponding pixels (in quantity pixels). In order to correctly merge the elementary partial images, these shifts must be previously estimated and compensated for to merge pixels representing the same portion of the partial area. However, given that the dimensions of the elementary partial images are reduced (compared to those of the final image to be generated), the number of shifts to be estimated and compensated is also reduced. This number of shifts to be estimated can be further reduced, optionally, by considering shifts between groups of pixels ("binning" in the Anglo-Saxon literature).

[0014] Since the two elementary partial images are assumed to represent the same partial area, comparing the content of said elementary partial images allows estimating the offsets between these pixels. Since elementary partial images are of reduced dimensions, they can be used to estimate the offsets between their pixels with limited computational complexity. Furthermore, it can generally be assumed that there is a non-zero correlation between, on the one hand, the offsets between pixels of elementary partial images representing one partial area and, on the other hand, the offsets between pixels of elementary partial images representing a different partial area, for example adjacent to the previous one. Therefore, the offsets between pixels estimated for one partial area can be used to estimate the offsets between pixels for the next partial area.

[0015] The operating principle of a Kalman filter, in that it is based on an iterative estimation of a state vector, according to which the state vector for the current iteration is first predicted from the state vector updated for the previous iteration, then updated to take into account measurements representative of said state vector for the current iteration, is therefore particularly suitable for estimating the offsets between pixels for the fusion of elementary partial images. Indeed, by considering that the successive iterations correspond to the successive processing of the different partial zones and that the state vector corresponds to the offsets between pixels, then the correlation between offsets of successive partial zones makes it possible to predict the offsets for a partial zone currently being processed from the offsets updated for the partial zone processed in the previous iteration.Furthermore, the comparison of the elementary partial images of the partial area being processed makes it possible to observe the offsets between pixels for the partial area being processed, and therefore makes it possible to update the predicted offsets for said partial area being processed.

[0016] The elementary partial images can then be merged on board the satellite based on the estimated offsets for the partial area being processed (either the predicted offsets or the updated offsets).

[0017] In particular embodiments, the generation method may further comprise, optionally, one or more of the following characteristics, taken individually or in all technically possible combinations.

[0018] In particular embodiments, the comparison of said at least two elementary partial images comprises a registration between them of said at least two elementary partial images according to the predicted offsets for the partial zone being processed. Such registration aims to compensate for the offsets between elementary partial images to obtain registered elementary partial images whose pixels represent substantially the same portions of the partial zone. The registration comprises for example a resampling of at least one of said elementary partial images according to the predicted offsets.

[0019] In particular modes of implementation, the updating by the Kalman filter of the predicted offsets implements an optical flow calculation algorithm applied to said at least two elementary partial images recalibrated according to the predicted offsets.

[0020] In special implementations, the Kalman filter estimates offsets between groups of pixels. Such arrangements reduce computing and memory requirements.

[0021] In particular implementations, the estimated offsets consist of a vector of row offsets and a vector of column offsets. Such arrangements reduce computing and memory requirements.

[0022] In particular implementations, the Kalman filter update of predicted offsets implements constant Kalman gains for processing all partial areas. Such arrangements reduce computing and memory requirements.

[0023] In particular implementation modes, the Kalman filter's updating of the predicted offsets implements variable gains.

[0024] In particular implementations, the Kalman filter has no control input.

[0025] In particular embodiments, the Kalman filter comprises a control input for taking into account external information on which the shifts to be estimated depend. Such arrangements are advantageous, in particular, when the time difference between the respective acquisition times of the at least two elementary partial images is relatively large. In such a case, certain sources of shifts (for example projection effects) produce at least partly deterministic effects which can be estimated a priori and injected into the Kalman filter via the control input. It is also possible to inject into the Kalman filter, via the control input, external information representative of the movement of the satellite between these acquisition times, for example obtained via exogenous measurements carried out by gyrometers of the satellite.

[0026] In particular modes of implementation, the fusion of the at least two elementary partial images of a partial zone comprises a registration of said at least two elementary partial images according to the estimated offsets, and a combination of said at least two registered elementary partial images.

[0027] In particular modes of implementation, the registration of said at least two elementary partial images comprises a resampling of at least one of said elementary partial images as a function of the estimated offsets.

[0028] In particular embodiments, the estimated offsets used to register and combine said at least two elementary partial images, during the fusion, correspond to the predicted offsets for the partial area being processed. Such arrangements make it possible to reduce the amount of registrations to be carried out for the partial area being processed, and therefore make it possible to reduce the requirements in terms of computing capacity and memory.

[0029] In particular embodiments, the combination of the at least two registered elementary partial images corresponds to a pixel-by-pixel sum of said at least two registered elementary partial images and / or to a pixel-by-pixel difference between said at least two registered elementary partial images.

[0030] In particular embodiments, the generation method comprises compression / decompression of the elementary partial images after their acquisition by the observation instrument and before their processing by the processing circuit. Such arrangements make it possible to reduce the requirements in terms of data transfer rate and memory on board the satellite.

[0031] According to a second aspect, there is provided a computer program product comprising instructions which, when executed by a processing circuit of a satellite further comprising an observation instrument, configure said processing circuit to implement an image generation method according to any one of the embodiments of the present invention.

[0032] According to a third aspect, there is provided a satellite intended to be placed in Earth orbit, comprising an observation instrument and a processing circuit, said processing circuit being configured to implement an image generation method according to any one of the embodiments of the present invention.

[0033] In particular embodiments, the satellite may further comprise, optionally, one or more of the following characteristics, taken individually or in any technically possible combination.

[0034] In particular embodiments, the observation instrument comprises at least two linear sensors, each of the at least two linear sensors being arranged transversely to a direction of travel of said satellite and being adapted to produce a line of pixels, each elementary partial image being a line image corresponding to one or more lines of pixels acquired successively by one of the at least two linear sensors.

[0035] In particular embodiments, each of the at least two linear sensors is a time-delay and integration sensor.

[0036] In particular embodiments, the observation instrument comprises at least one matrix sensor comprising a plurality of acquisition cells arranged in a plurality of rows and a plurality of columns, said rows of acquisition cells extending transversely to a direction of movement of said satellite, each acquisition cell being adapted to produce a pixel such that said at least one matrix sensor is adapted to produce a matrix image comprising a plurality of rows of pixels and a plurality of columns of pixels, each elementary partial image being a row image corresponding to one or more adjacent rows of pixels of the same matrix image.

[0037] Thus, the proposed solution, which is based on the fusion of several images on board the satellite, is applicable to different configurations for the observation instrument and to different acquisition modes. In the case of a pushbroom acquisition mode, the successively acquired pixel lines can be processed on the fly, before having acquired elementary partial images for all the partial areas, as pixel lines are received from the different linear sensors. In the case of an acquisition mode using a matrix sensor, the elementary partial images correspond to sub-images successively extracted from the images acquired by the matrix sensor. Presentation of figures

[0038] The invention will be better understood by reading the following description, given as a non-limiting example, and made with reference to the figures which represent: Figure 1: a schematic representation of a satellite orbiting the Earth, Figure 2 : schematic representations of examples of sensors of a satellite observation instrument, Figure 3 : a diagram illustrating the main steps of an example of implementation of an acquisition process, Figure 4 : a diagram illustrating the main steps of an example of implementation of the estimation of shifts between elementary partial images.

[0039] In these figures, like references from one figure to another designate identical or similar elements. For reasons of clarity, the elements shown are not to scale, unless otherwise indicated.

[0040] Furthermore, the order of steps shown in these figures is given only as a non-limiting example of the present disclosure which can be applied with the same steps performed in a different order. Description of the modes of realization

[0041] There figure 1 schematically represents a satellite 10 in Earth orbit around the Earth 30. In general, the present disclosure is applicable to any type of Earth orbit. The present disclosure finds a particularly advantageous application in the case of circular orbits (i.e., non-geostationary). However, the present disclosure is also applicable in the case of a satellite 10 in geostationary orbit (GEO) or in an inclined or elliptical orbit. In preferred embodiments of the present disclosure, the satellite 10 is in a low-Earth Orbit (LEO) or medium-Earth Orbit (MEO) altitude, preferably circular. In the remainder of the description, the case of a satellite 10 in a orbit is considered in a non-limiting manner.

[0042] The satellite 10 carries an observation instrument 20. The observation instrument 20 is preferably sensitive in visible and / or infrared wavelengths. The observation instrument 20 comprises, in a manner known per se, acquisition means configured to capture the optical flow coming from the scene observed on the surface of the Earth 30. In particular, the observation instrument 20 comprises at least one sensor comprising a plurality of acquisition cells 21 organized in one or more rows and several columns, each acquisition cell 21 making it possible to acquire the optical flow coming from a portion of the observed scene.

[0043] Different arrangements are possible for the at least one sensor of the observation instrument 20. The figure 2 schematically represents non-limiting examples of sensor arrangements.

[0044] Part (a) of the figure 2schematically represents a first example in which the observation instrument 20 comprises two independent linear sensors 22. In the case of a satellite 10 in a traveling orbit, each of the two linear sensors 22 is for example arranged transversely to a direction of travel of said satellite 10 and is adapted to produce at least one line of pixels. The acquisition cells 21 of each linear sensor 22 are therefore organized in one or more lines arranged transversely to the direction of travel of the satellite 10. In preferred embodiments of the observation instrument 20, each of the two linear sensors 22 is a TDI sensor comprising for example several stages of lines of acquisition cells 21 which carry out a charge transfer from one line of acquisition cells 21 to another in a manner synchronized with the travel of the satellite 10. For example, the linear sensors 22 are panchromatic or infrared sensors.

[0045] Each linear sensor 22 has a significantly greater number of columns than the number of rows. For example, each linear sensor 22 has one or a few (typically of the order of ten or less) rows of acquisition cells 21 and several thousand columns of acquisition cells 21. It should be noted that the linear sensors 22 preferably have identical characteristics in terms of number of rows / number of columns, in terms of spatial resolution and in terms of wavelength band(s) measured.

[0046] It should be noted that, according to other examples, the observation instrument 20 may comprise more than two linear sensors 22 for the implementation of the present disclosure. It should also be noted that the acquisition means of the observation instrument 20 may also comprise other sensors in addition to the two linear sensors 22, for example sensitive in different wavelength bands. For example, if the linear sensors 22 are panchromatic sensors, the acquisition means may also comprise multispectral sensors.

[0047] Part b) of the figure 2represents a second example in which the observation instrument 20 comprises a matrix sensor 23. In such a case, the matrix sensor 23 comprises a plurality of acquisition cells 21 arranged in a plurality of rows and a plurality of columns. In the case of a satellite 10 in a traveling orbit, the lines of acquisition cells 21 extend for example transversely to the direction of travel of the satellite 10, and each acquisition cell 21 is adapted to produce a pixel so that the matrix sensor 23 produces a matrix image comprising a plurality of rows of pixels and a plurality of columns of pixels. For example, the matrix sensor 23 is a panchromatic or infrared sensor. The matrix sensor 23 comprises for example several thousand lines of acquisition cells 21 and several thousand columns of acquisition cells 21.

[0048] It should be noted that, according to other examples, the observation instrument 20 may comprise more than a single matrix sensor 23 for the implementation of the present disclosure. In such a case, the matrix sensors 23 used in the present disclosure preferably have identical characteristics in terms of number of rows / number of columns, in terms of spatial resolution and in terms of measured wavelength band(s).

[0049] It should also be noted that the acquisition means of the observation instrument 20 may also comprise other sensors in addition to the matrix sensor 23, for example sensitive in different wavelength bands.

[0050] The observation instrument 20 may also comprise an optic comprising one or more mirrors arranged to reflect the optical flow coming from the observed scene towards the sensor(s). The sensor(s) are for example arranged at a focal plane of the optics of said observation instrument 20.

[0051] The satellite 10 further comprises a processing circuit (not shown in the figures), which is connected to the observation instrument 20 and processes the images acquired by the observation instrument 20. In particular, the processing circuit implements an image generation method 50 which will be described below.

[0052] The processing circuit comprises, for example, one or more processors and storage means (magnetic hard disk, electronic memory, optical disk, etc.) in which a computer program product is stored, in the form of a set of program code instructions to be executed to implement the different steps of the image generation method 50. Alternatively or in addition, the processing circuit comprises one or more programmable logic circuits (FPGA, PLD, etc.), and / or one or more specialized integrated circuits (ASIC, etc.), and / or a set of discrete electronic components, etc., adapted to carry out all or part of said steps of the image generation method 50.

[0053] It should be noted that the satellite 10 may also comprise, in a conventional manner, other elements such as attitude measurement means (star sensor, gyrometer, etc.), or attitude control means (reaction wheels, flywheels, gyroscopic actuators, magneto-couplers, etc.), in particular, which may also be connected to the processing circuit.

[0054] There figure 3 schematically represents the main steps of a method 50 for generating an image representing a geographical area on the surface of the Earth 30.

[0055] As will be described in more detail below, the generated image comprises a plurality of rows of pixels and a plurality of columns of pixels, and corresponds to the grouping of a plurality of partial images representing different partial areas within the geographic area.

[0056] Furthermore, each partial image is obtained by merging, on board the satellite 10, at least two so-called elementary partial images theoretically representing the same partial area. The elementary partial images are acquired by the observation instrument 20 at different respective times. Thus, each partial image is obtained by merging several elementary partial images representing the same partial area.

[0057] The nature of the elementary partial images depends for example on the type of sensor(s) of the observation instrument 20.

[0058] For example, if the observation instrument 20 comprises at least two linear sensors 22, as illustrated by part a) of the figure 2, then each elementary partial image corresponds to a line image corresponding to one or more lines of pixels acquired successively by one of the at least two linear sensors 22. In the present disclosure, the term "line image" means an image comprising a number of lines significantly lower than the number of columns, for example 100 times fewer lines than columns, preferably 1000 times fewer lines than columns. Due to their particular arrangement, the two linear sensors 22 of part a) of the figure 2 successively observe the same partial area, by scrolling the satellite 10, so that the elementary partial images representing the same partial area are obtained at different respective times.

[0059] If the observation instrument 20 comprises at least one matrix sensor 23, as illustrated by part b) of the figure 2, then each elementary partial image is for example a line image corresponding to one or more adjacent lines of pixels extracted from the same matrix image. In other words, an elementary matrix image represents several partial zones, and the elementary partial image corresponds to a part of an elementary matrix image acquired by the matrix sensor 23. The elementary partial images are therefore extracted from different respective elementary matrix images acquired at different respective times by the observation instrument 20. The elementary matrix images representing the same partial zone can be acquired at different respective times by the same matrix sensor 23, or by different matrix sensors 23 if the observation instrument 20 comprises at least two matrix sensors 23.

[0060] According to the generation method 50, all partial areas of a geographical area are processed successively.

[0061] As illustrated by the figure 3 , the processing of a partial zone comprises a step S51 of reception, by the processing circuit, of at least two elementary partial images acquired by the observation instrument 20 at different respective times and representing said partial zone being processed. Considering a satellite 10 in a moving orbit, said elementary partial images of said partial zone being processed are therefore acquired from different initial positions of said satellite 10 in its orbit, during the same flyby of said partial zone (during the same orbital period).

[0062] In the remainder of the description, the case in which exactly two elementary partial images are considered in order to generate the partial image of the partial area being processed is considered in a non-limiting manner. However, nothing precludes, following other examples, the consideration of more elementary partial images for the generation of the partial image of the partial area being processed.

[0063] As indicated above, the partial areas are processed successively. In practice, the partial area currently being processed is not necessarily known. a priori.In practice, it is sufficient to know that two elementary partial images represent in principle substantially the same partial area. The successive processing of partial areas is carried out by successively processing successive pairs of elementary partial images, each pair of elementary partial images representing substantially the same partial area without having to know the coordinates of the partial area being processed.

[0064] For example, in the case where the observation instrument 20 comprises two linear sensors 22, then the successive processing of the partial zones amounts to successively processing line images acquired successively by each of said linear sensors 22, as each of the linear sensors scans the geographical zone. The line images representing the same partial zone (elementary partial images) are acquired at different respective times since the linear sensors 22 do not point simultaneously towards the same partial zone. Preferably, the line images are processed successively as they are acquired by the observation instrument and received by the processing circuit, so that the partial zones processed successively are partial zones adjacent to the surface of the Earth 30.

[0065] For example, in the case where the observation instrument 20 comprises a matrix sensor 23, then the successive processing of the partial zones amounts to successively processing different line images extracted from different matrix images acquired successively by the matrix sensor 23. For each matrix image, the line images extracted and processed successively are preferably adjacent line images, so that the partial zones processed successively are partial zones adjacent to the surface of the Earth 30.

[0066] As illustrated by the figure 3, the processing of a partial zone then comprises a step S52 of estimation, by the processing circuit, of offsets between pixels of said two elementary partial images. Indeed, due to different sources of errors (lack of knowledge of the line of sight of the observation instrument 20 for example due to vibrations, geometric deformation of a footprint on the ground of a field of vision of the observation instrument 20 due to the fact that the partial zone is observed from two different positions of the satellite 10 relative to said partial zone, geometric distortion introduced by an optic of the observation instrument 20, etc.), pixels of different elementary partial images, which should represent the same portion of the partial zone, may in practice represent slightly different respective portions of said partial zone.The difference between these respective different portions results in a shift between the corresponding pixels (in quantity pixels).

[0067] Since the two elementary partial images are assumed to represent the same partial area, the offsets between pixels can for example be estimated by comparing the content of said elementary partial images. Since elementary partial images are of reduced dimensions (insofar as they represent only a part of the geographical area to be imaged), they can be used to estimate the offsets between their pixels with limited computational complexity.

[0068] The offsets are estimated using a Kalman filter. The Kalman filter may be any type of Kalman filter known to the person skilled in the art, including its nonlinear variants (extended Kalman filter, unscented Kalman filter, etc.), and the choice of a particular type of Kalman filter corresponds to a particular mode of implementation of the generation method 50. Whatever its type, a Kalman filter is an iterative filter which estimates a state vector and which, at each iteration, carries out a prediction phase and a phase of updating said state vector. The prediction phase provides a so-called predicted state vector and the update phase provides a so-called updated state vector. In practice, the predicted state vector for an iteration is calculated from the updated state vector for the previous iteration. The updated state vector is calculated as a function of the predicted state vector and as a function of measurements representative of said state vector.

[0069] In the present case, the state vector of the Kalman filter comprises at least the offsets between pixels of the elementary partial images of the partial area being processed, and the measurements are determined at least from said elementary partial images. The successive iterations of the Kalman filter correspond here to the succession of pairs of elementary partial images processed, i.e. to the succession of partial areas processed. Consequently, the partial area represented by the pair of elementary partial images changes from one iteration of the Kalman filter to another. In the remainder of the description, we therefore refer to iteration interchangeably k or partial rank area processing k .

[0070] There figure 4schematically represents the main steps of a preferred mode of implementation of step S52 of estimating the shifts between the elementary partial images by means of a Kalman filter.

[0071] As illustrated by the figure 4 , step S52 of estimating the offsets for the partial area being processed, of rank k (iteration k of the Kalman filter), takes as input the predicted shifts for said partial rank area k from the updated offsets for the partial rank area ( k - 1), that is to say treated previously at the iteration ( k - 1) of the Kalman filter. As illustrated by the figure 4 , step S52 of estimating the shifts comprises a step S521 of comparing the elementary partial images representing the partial zone of rank k , received by the processing circuit. The comparison step S521 provides a rank innovation kallowing to update the predicted shifts for the partial rank area k . Step S52 of estimating the shifts then includes a step S522 of updating the predicted shifts for the partial zone of rank k depending in particular on the innovation of rank k . Update step S522 outputs the updated offsets for the partial rank area k . Step S52 of estimating the shifts then includes a prediction step S523 which, from the shifts updated for the partial zone of rank k , calculates the predicted shifts for the partial area that will be processed next, i.e. the partial area of ​​rank ( k + 1).

[0072] As illustrated by the figure 4 , the estimated shifts for the partial area being processed in rank k, used during a merging step S53 described below, can be indifferently the shifts predicted for said partial rank zone k or the updated offsets for said rank processing area k .

[0073] In particular modes of implementation, the comparison step S521 comprises a re-alignment between them of said at least two elementary partial images according to the predicted offsets for the partial zone currently being processed, of rank k . By "recalibration" we mean that the predicted shifts for the partial rank area kare corrected in the elementary partial images so that the pixels of the re-registered elementary partial images theoretically represent the same portions of the partial area (within the estimation errors of the predicted offsets). During this re-registering, it is possible to correct the two elementary partial images, or to correct only one of them by considering the other as a reference. The re-registering comprises, for example, a resampling of at least one of said elementary partial images as a function of the predicted offsets. For example, the resampling of an elementary partial image implements a bicubic interpolation of this elementary partial image. However, any type of resampling method known to the person skilled in the art can be implemented and the choice of a particular type of resampling method corresponds only to a non-limiting example of implementation.

[0074] In particular implementations, the Kalman filter update of the predicted shifts for the partial rank area k implements an optical flow calculation algorithm applied to said two elementary partial images readjusted according to said predicted shifts for the partial zone of rank k . However, any type of image matching method (or "matching" in the English literature) known to the person skilled in the art can be implemented and the choice of a particular type of matching method corresponds only to a non-limiting example of implementation. In particular, it is possible, according to other examples of implementation, to match the elementary partial images by a resemblance calculation, such as a correlation calculation, between said elementary partial images.

[0075] As illustrated by the figure 3, the processing of a partial zone then comprises step S53 of merging, by the processing circuit, said two elementary partial images representing the partial zone being processed as a function of the offsets estimated during the estimation step S52.

[0076] For example, the estimated offsets are used to re-register the elementary partial images with each other, and then the re-registered elementary partial images can be combined, for example by calculating a sum (possibly weighted) of said re-registered elementary partial images or by calculating the difference between said re-registered elementary partial images. It should be noted that it is also possible to generate several partial images during the merging step S53 (to generate several images of the geographical area), for example by generating a first partial image representative of the sum of said re-registered elementary partial images and a second partial image representative of the difference between said re-registered elementary partial images.

[0077] As indicated above with reference to the figure 4 , the estimated shifts for the partial area being processed, of rank k, used during the merging step S53, can be indifferently the predicted shifts for the partial rank zone k or the updated offsets for said rank processing area k .

[0078] In principle, the updated offsets for the partial rank area k take into account the elementary partial images representing said partial zone of rank k , so that the updated shifts are in principle more accurate than the predicted shifts for the partial rank area k .

[0079] However, and as indicated previously, the estimation step S52 already comprises in certain implementation modes a recalibration of said elementary partial images representing the partial zone of rank k based on the predicted shifts for said partial rank area k. If, during the merging step S53, the elementary partial images are realigned according to the offsets updated for said partial zone of rank k, this implies that the elementary partial images representing the partial zone of rank k are rejected twice during the processing of said partial rank zone k , which increases the computing and memory requirements for each iteration of the Kalman filter. Therefore, during the fusion step S53, it is advantageous to use the predicted offsets for the partial area being processed if one wishes to reduce the computing and memory requirements. If one wishes to prioritize the accuracy of the estimated offsets, it is advantageous to use the updated offsets for the partial area being processed.

[0080] The fusion of said two elementary partial images, during step S53, provides the partial image of the partial area being processed, which forms a part of the image to be generated of the geographical area. By repeating the different steps illustrated by the figure 3 for all the partial areas, a plurality of partial images representing different partial areas are therefore obtained, which can be grouped in a conventional manner to form an image of the geographical area. Typically, the partial images can simply be juxtaposed to form the image of the geographical area. The image thus generated of the geographical area is then transmitted to the ground, possibly after application, on board the satellite 10, of other optional processing operations on said generated image which are outside the scope of the present disclosure.

[0081] A detailed example of implementation of the image generation method 50 is now described, in the case of an observation instrument comprising two linear sensors 22 (part a) of the figure 2 ). This non-limiting example is based on a simplified Kalman filter, optimized to reduce the requirements in terms of computing capacity and memory to facilitate its implementation on board a satellite 10.

[0082] In order to describe this detailed example, we recall below the implementations in a Kalman filter according to the usual notations, to which we will then refer by analogy to describe the detailed implementation.

[0083] Generally speaking, in a Kalman filter, the equations governing the prediction phase are as follows: x ^ k k − 1 = F k x ^ k − 1 k − 1 + B k u k + w k P k k − 1 = F k P k − 1 k − 1 F k T + Q k expressions in which: x̂ k | k -1 corresponds to the predicted state vector for the iteration k ,P k | k -1 corresponds to a covariance matrix of the predicted error for the iteration k , F k corresponds to an evolution matrix that connects the updated state vector for the iteration ( k - 1) ( x̂ k -1| k -1 ) to x̂ k | k -1 , u k corresponds to a command entry, B k corresponds to a matrix that relates the control input to the state vector for the iteration k , Q k corresponds to a process noise covariance matrix w k .

[0084] The equations governing the update phase are as follows: y ˜ k = z k − H k x ^ k k − 1 S k = H k P k k − 1 H k T + R k K k = P k k − 1 H k T S k − 1 x ^ k k = x ^ k k − 1 + K k y ˜ k P k k = I − K k H k P k k − 1 expressions in which: ỹ k corresponds to the innovation calculated for the iteration k ,z k corresponds to an observation of the process at the iteration k , H k corresponds to a matrix that connects the state vector x̂ k | k -1 to observation z k , K k corresponds to a Kalman gain matrix for the iteration k , x̂ k | k corresponds to the updated state vector for the iteration k , P k | k corresponds to the updated error covariance matrix for the iteration k , R k corresponds to a covariance matrix of a measurement noise, I corresponds to the identity matrix.

[0085] In the non-limiting example considered here, the Kalman filter is simplified by considering that there is no process noise, that said Kalman filter is devoid of control input and that the Kalman gain matrix is ​​constant from one iteration to another. In other words, the main equations governing this simplified Kalman filter are as follows: y ˜ k = z k − H k x ^ k k − 1 x ^ k k = x ^ k k − 1 + K y ˜ k x ^ k k − 1 = F k x ^ k − 1 k − 1

[0086] We now describe the example of implementing the simplified Kalman filter by adopting other notations.

[0087] In this example, the elementary partial images correspond to elementary line images, each elementary line image has a number of lines equal to NL and a number of columns equal to NC, NL being much lower than NC (factor at least 100 or even at least 1000). For example, 2 ≤ NL ≤ 10 and NC > 1000 or even NC > 5000.

[0088] To reduce the amount of computation, offsets are estimated in this example for groups (also known as "patches") of adjacent pixels comprising I lines and J columns. Thus, an offset is estimated for a group of I × J adjacent pixels. Preferably, I = NL so that each group extends over all lines of the elementary partial image and so that each elementary partial image consists of n C groups of adjacent pixels with n C = NC / J. For example, I × J = 4 × 4 or I × J = 2 × 8.

[0089] Considering I = NL , the offsets to be estimated can advantageously consist of a vector of offsets between lines, designated below gridL, and a vector of shifts between columns, hereinafter designated gridC. So each shift in the vector gridL corresponds to the shift between the lines of corresponding groups of pixels (theoretically representing the same portion of the observed scene) in the two elementary partial images. Each shift in the vector gridC corresponds to the offset between the columns of corresponding pixel groups (theoretically representing the same portion of the observed scene) in the two elementary partial images. The two vectors gridL And gridC to be estimated have the same dimensions, namely 1 × n C . The vector gridL (resp. gridC ) predicted for the iteration k is designated by gridL k | k -1 (resp. gridC k | k -1 ) and the vector gridL (resp. gridC ) updated for iteration k is designated by gridL k | k (resp. gridC k | k ).

[0090] In the present example, two elementary partial images are used to form the partial image, denoted respectively by in 1 k And in 2 k for iteration k. In this example, during registration, only the elementary partial image in 2 k is resampled according to the estimated offsets, and the resampled elementary partial image is denoted by in 2 res k . In relation to the groups of pixels defined above, we define a grouping function (or "binning" in Anglo-Saxon literature) as follows: x m , n = 1 I ⋅ J ∑ i = 1 I ∑ j = 1 J x mI + i , nJ + j expression in which x corresponds to an image to which the grouping function └ ┘ is applied, and ( x ) p,r corresponds to the pixel value of the image x located at the line p and to column r of said image x.

[0091] The equations used to calculate innovation for the iteration are described below. k, designated by dL k for the vector gridL and by dC k for the vector gridC (stage ỹ k = z k - H k x̂ k | k -1 according to the usual notations of the Kalman filter). In this example, the calculation of innovations dL k And dC k for updating the offsets implements an optical flow calculation algorithm.

[0092] First, the elementary partial images in 1 k And in 2 k of iteration k are aligned with each other, by resampling the elementary partial image in 2 k based on the predicted offsets for the iteration k, namely vectors gridL k | k -1 and gridC k | k -1. Considering in a non-limiting manner a resampling based on a bicubic interpolation: im 2 res k = bicubic _ interp im 2 k , gridL k k − 1 , gridC k k − 1 expression in which bicubic_interp corresponds to the bicubic interpolation function. For example, the function bicubic_interp can be implemented by means of predetermined filter banks, namely a filter bank p L and a bank of filters p C in which the filters are indexed by the offsets between the rows and columns respectively: im 2 res k = im 2 k ∗ p L gridL k k − 1 ∗ p C gridC k k − 1 expression in which * corresponds to the convolution.

[0093] As part of the optical flow calculation algorithm, we also calculate the sum between the registered elementary partial images (i.e. between in 1 k And in 2 res k ) and the difference between said recalibrated elementary partial images: imSUM k = im 1 k + im 2 res k imDIFF k = im 1 k − im 2 res k

[0094] Then, within the framework of the optical flow calculation algorithm, we calculate the gradients of the elementary partial image in 1 k . The gradients between lines, denoted by gL k , and the gradients between columns, denoted by g c k , are for example calculated as follows: gL k = im 1 k ∗ − 1 0 1 gC k = im 1 k ∗ − 1 0 1

[0095] We also define the square and product functions for images as follows: x 2 p , r = x p , r 2 x ⋅ y p , r = x p , r ⋅ y p , r

[0096] In other words, these are pixel-by-pixel square and product operations.

[0097] Residual shifts between images in 1 k And in 2 res k that is to say, innovations dL k And dC k , are for example calculated as follows: a = gL k 2 + λ b = gL k ⋅ gC k c = gC k 2 + λ u = gL k ⋅ imDIFF k v = gC k ⋅ imDIFF k d = a ⋅ c − b 2 dL k = 1 d c ⋅ u − b ⋅ v dC k = 1 d a ⋅ v − b ⋅ u expressions in which λcorresponds to a Tikhonov regularization term to account for the presence of estimation noise in the optical flow calculation algorithm. The regularization term λ sets the level of consideration of noise present in the elementary partial image in 1 k , which impacts the estimation of gradients in the optical flow calculation algorithm. The order of magnitude of the regularization term λ is preferably that of the variance of the noise in the elementary partial image in 1 k . For example, the regularization term λ is a matrix in which all coefficients are equal. The regularization term λ can for example be predetermined, and kept constant and identical for all the partial zones to be treated. Nothing excludes, however, following other examples, varying the regularization term λduring iterations, for example based on an estimate of the noise in each elementary partial image in 1 k .

[0098] Then the offsets for the iteration k can be updated as follows (step x̂ k | k = x̂ k | k -1 + This k according to the usual notations of the Kalman filter): gridL k k = gridL k k + K ⋅ dL k gridC k k = gridC k k − 1 + K ⋅ dC k expressions in which K corresponds to the Kalman gain matrix. As previously indicated, the Kalman gains are in this example constant during the iterations, i.e. identical for all the partial areas to be processed. Preferably, all the Kalman gains in the matrix K are equal to the same value K 0 which can be predetermined. Generally speaking, the higher the value K 0 is low and the less trust we have in the information inferred from the images in 1 k And in 2 res k .

[0099] However, nothing excludes, following other examples, varying the Kalman gains during successive iterations, for example by using the covariance matrix of the measurement noise at each iteration, or even as a function of a digital surface model, DSM, or as a function of an indicator of reliability of the information deduced from the images. in 1 k And in 2 res k etc. For example, it is possible to increase the Kalman gains when the DSM indicates that the partial area has a high relief, compared to the case where the DSM indicates that the partial area has a low relief (e.g., substantially flat). As another example, an indication of reliability can be obtained by calculating the correlation between the images in 1 k And in 2 res kand the value of the correlation peak can be used as an indicator of the reliability of the information deduced from these images. Thus, it is possible to increase the Kalman gains when the value of the correlation peak is large, compared to the case where the value of the correlation peak is small. In general, the use of variable Kalman gains can in some cases improve the performance of the Kalman filter compared to the use of constant Kalman gains.

[0100] Then, the updated offsets for iteration k are used to calculate the predicted offsets for iteration (k + 1) (step x̂ k | k -1 = F k x̂ k -1| k -1 according to the usual notations of the Kalman filter), for example as follows: gridL k + 1 k = gridL k k ∗ kern gridC k + 1 k = gridC k k ∗ kern expressions in which corecorresponds to a prediction filter which in this example is a predetermined row vector applied both to calculate gridL k +1| k And gridC k +1| k . Generally speaking, the prediction filter core can take any suitable form. For example, it can be a smoothing filter such as a Gaussian filter, a triangular filter, etc. In some cases, it is also possible, for example to further reduce the calculations to be performed, to have a prediction filter core which corresponds to the identity, in which case the predicted shifts for the iteration ( k + 1) are equal to the updated offsets for the iteration k . In addition, it is also possible to have different prediction filters to calculate respectively gridL k +1| k And gridC k +1| k .

[0101] The partial image for iteration k is then obtained by merging the elementary partial images in 1 k And in 2 k based on the estimated shifts for iteration k. As indicated above, it is advantageous to use the predicted shifts for iteration k , since the recalibration (images in 1 k And in 2 res k ) has already been done to calculate the residual shifts dL k And dC k . Furthermore, if the fusion corresponds to the difference between the registered elementary partial images, then it is possible to reuse as a partial image (i.e. as a result of the fusion), the image imDIFF kalready calculated as part of the optical flow calculation algorithm. Such arrangements therefore make it possible to reduce the requirements in terms of computing capacity and memory. More generally, the partial image can correspond to the image imSUM k and / or to the image imDIFF k . Other combinations of the registered elementary partial images are also possible, for example by applying different respective weighting coefficients to the different registered elementary partial images.

[0102] In the detailed example described above, it is understood that it is possible to process on the fly, and systematically, the different elementary partial images acquired successively by the linear sensors 22, which makes it possible to reduce the memory requirements. Indeed, the processing of a partial zone can start as soon as it has been acquired by the linear sensor 22 which last observes this partial zone (synchronization of the linear sensors 22). During iteration k, as soon as the partial image (for example imSUM k and / or imDIFF k ) was calculated and stored, the elementary partial images in 1 k And in 2 k no longer need to be stored in memory.

[0103] In particular modes of implementation, to reduce the requirements in terms of data transfer rate and memory, it is also possible to carry out on-the-fly compression / decompression of the elementary partial images after their acquisition by the observation instrument 20 and before their processing by the processing circuit.

[0104] More generally, it should be noted that the methods of implementation and embodiment considered above have been described as non-limiting examples, and that other variants are consequently conceivable.

[0105] In particular, the above examples have been given by considering mainly the case of linear sensors 22. However, and as indicated previously, the present disclosure is also applicable to other types of sensors, in particular matrix sensors 23. In such a case, the elementary partial images are for example extracted from elementary matrix images acquired at different times, representing the same geographical area, or else distinct geographical areas but having a non-zero intersection in which several partial areas can be processed according to the generation method 50.

[0106] Furthermore, the above examples were given mainly considering a Kalman filter without a control input. Indeed, in the case in particular where the elementary partial images are acquired by linear sensors 22, the variations in the offsets from one pair of elementary partial images to the next are in principle quite small. However, it is also possible, in other examples, to use a control input to take into account external information (exogenous to the images, derived from measurements carried out by means independent of the observation instrument 20) on which the offsets to be estimated depend. In particular, if the variations in the offsets of a pair of elementary partial images cannot be considered small, then the control input can be used to assist the Kalman filter.The variations in the offsets may be greater, in particular, when the time difference between the respective acquisition times of the elementary partial images is relatively large, for example of the order of a few tens or a few hundred milliseconds. This may be the case, in particular, in the case where the elementary partial images are acquired by the same matrix sensor 23. In such a case, certain sources of offsets (for example projection effects) produce at least partly deterministic effects which may be estimated a priori and injected into the Kalman filter via the control input. It is also possible to inject into the Kalman filter, via the control input, external information representative of the movement of the satellite between these acquisition times, for example obtained via measurements carried out by one or more gyrometers of the satellite, etc.More generally, any external information representative of the offsets to be estimated can be injected into the Kalman filter via the control input.

[0107] Furthermore, the above examples were given by considering mainly the case of a fusion of two elementary partial images for each partial zone. Nothing excludes, however, following other examples, from considering a larger number of elementary partial images to be fused. For example, everything that has been described above is applicable by considering an elementary partial image as reference, and by processing at each iteration all the different pairs of elementary partial images which comprise said reference elementary partial image.

[0108] The above description clearly illustrates that, through its various characteristics, the present disclosure achieves the set objectives. It should also be noted that the detailed example described above has been implemented, and its performance confirmed, on an FPGA qualified for space flight.

[0109] The present disclosure has numerous advantages. In particular, the on-board fusion of the images acquired by the observation instrument 20 makes it possible to improve the spatial resolution of the images transmitted to the ground.

[0110] Furthermore, thanks to the on-board fusion of the satellite 10 of the images acquired by the observation instrument 20, it is possible, for a given satellite 10 / observation instrument 20 configuration (i.e. with constant configuration), to increase the performance in terms of figure of merit (FdM, corresponding to the product FTM x SNR). Alternatively, thanks to the on-board fusion of the images, it is possible to achieve a given figure of merit (i.e. with constant figure of merit) with a simpler (and therefore less expensive) configuration of the satellite 10 / observation instrument 20.

Claims

1. Method (50) for generating an image using a satellite (10) in earth orbit, said satellite carrying on board an observation instrument (20) and a processing circuit, said image representing a geographical area and being obtained by grouping partial images representing different partial areas within the geographical area, wherein the partial areas are processed successively and said method comprises, for the processing of each partial area: - receiving (S51), by the processing circuit, at least two basic partial images captured by the observation instrument at different respective times and representing said partial area being processed, - estimating (S52), by the processing circuit, shifts between pixels of said at least two basic partial images, - fusing (S53), by the processing circuit, said at least two basic partial images based on the estimated shifts, so as to form the partial image of said partial area being processed, and in that the shifts are estimated by means of a Kalman filter performing a prediction phase and an update phase, wherein the shifts for the partial area being processed are estimated based on shifts predicted for said partial area being processed from shifts updated for the partial area being processed at the previous iteration of the Kalman filter, said shifts predicted for said partial area being processed being updated by comparing the at least two basic partial images representing said partial area being processed.

2. Generation method (50) according to claim 1, wherein the comparison of said at least two basic partial images comprises adjusting between them said at least two basic partial images based on the shifts predicted for the partial area being processed.

3. Generation method (50) according to any one of claims 1 to 2, wherein the update by the Kalman filter of the predicted shifts implements constant gains for the processing of all of the partial areas.

4. Generation method (50) according to any one of claims 1 to 3, wherein the Kalman filter is devoid of a control input.

5. Generation method (50) according to any one of claims 1 to 3, wherein the Kalman filter comprises a control input for taking into account external information on which the shifts to be estimated depend.

6. Generation method (50) according to any one of claims 1 to 5, wherein the fusion of the at least two basic partial images of a partial area comprises adjusting between them said at least two basic partial images based on the estimated shifts, and combining said at least two adjusted basic partial images.

7. Generation method (50) according to claim 6, wherein the adjusting of said at least two basic partial images comprises resampling at least one of said basic partial images based on the estimated shifts.

8. Method (50) according to any one of claims 6 to 7, wherein the estimated shifts used to adjust and combine said at least two basic partial images correspond to the shifts predicted for the partial area being processed.

9. Generation method (50) according to any one of claims 6 to 8, wherein the combination of the at least two adjusted basic partial images corresponds to a pixelby-pixel sum of said at least two adjusted basic partial images and / or to a pixel-by-pixel difference between said at least two adjusted basic partial images.

10. Generation method (50) according to any one of claims 1 to 9, comprising compressing / decompressing the basic partial images after the capture thereof by the observation instrument and before the processing thereof by the processing circuit.

11. Computer program product comprising instructions which, when they are executed by a processing circuit of a satellite further comprising an observation instrument, configure said processing circuit to implement an image generation method (50) according to any one of claims 1 to 10.

12. Satellite (10) intended to be placed in earth orbit, comprising an observation instrument and a processing circuit, said processing circuit being configured to implement an image generation method (50) according to any one of claims 1 to 10.

13. Satellite (10) according to claim 12, wherein the observation instrument comprises at least two linear sensors (22), each of the at least two linear sensors being arranged transversely to a direction of travel of said satellite and being adapted to produce a row of pixels, each basic partial image being a line image corresponding to one or more pixel rows captured successively by one of the at least two linear sensors.

14. Satellite (10) according to claim 13, wherein each of the at least two linear sensors is a time-delay and integration sensor.

15. Satellite (10) according to claim 12, wherein the observation instrument comprises at least one matrix sensor (23) comprising a plurality of capture cells arranged according to a plurality of rows and a plurality of columns, said capture cell rows extending transversely to a direction of travel of said satellite, each capture cell being adapted to produce a pixel so that said at least one matrix sensor is adapted to produce a matrix image comprising a plurality of pixel rows and a plurality of pixel columns, each basic partial image being a line image corresponding to one or more adjacent pixel rows of the same matrix image.