Method for automatically labeling a radar spectrum with an identifier of the surface illuminated by the radar; Associated system and computer program

Abstract

Method for automatically labeling a radar spectrum with an identifier of the surface illuminated by the radar; Associated system and computer program. The present invention relates to a method (100), implemented by computer, for automatically labeling data corresponding to a radar spectrum, the radar spectrum being acquired by means of a radar on board an aircraft and illuminating a ground surface, with a view to training an artificial intelligence algorithm for identifying the ground surface, the method comprising: acquiring (110) a radar spectrum of a domain observed by the radar; obtaining (120, 140), by means of a reference system, a true identifier of a ground surface illuminated by the radar during the acquisition of the radar spectrum; and, associating (150), in a labeled data item, the data corresponding to the radar spectrum and the true identifier obtained. Figure for the abstract: Figure 2

Description

Title of the invention: Method for automatically labeling a radar spectrum with an identifier of the surface illuminated by the radar; Associated system and computer program.

[0001] The technical field of the invention is that of processing radar spectra, in particular for the recognition, by means of an airborne radar, of surfaces illuminated by the radar, whether it be a sea surface, a land surface, a lake surface, a river surface, a sandy surface, a plant surface (forest or crops for example), etc., or a combination of these surfaces.

[0002] Radar processing must be adapted in a dedicated manner to the spectra induced by the different natures of the ground surfaces illuminated by the radar during the acquisition of these spectra.

[0003] For example, in the case of target detection, echoes reflected by the ground surface illuminated by the radar generate background noise in the received spectrum. However, the signature of a target in the received spectrum must be extracted from this background noise.

[0004] In another example, illuminating the ground surface can make it possible to measure the speed of the aircraft carrying the radar by Doppler effect. However, this is made particularly difficult when the illuminated surface is a very calm sea, the radar equivalent surface - SER then being particularly small.

[0005] All the echoes reflected by the ground surface are called clutter.

[0006] The characteristics of the clutter (level, statistics, etc.) are different depending on the nature of the surface illuminated by the radar, whether it is land, sea, a lake, vegetation, etc.

[0007] The characteristics of the clutter then make it possible to identify the nature of the illuminated ground surface.

[0008] For this purpose, artificial intelligence algorithms for identifying the nature of the ground surface illuminated by the radar from the radar spectrum are implemented.

[0009] For training such identification algorithms, it is necessary to constitute a reference database, storing labeled spectra, that is to say a doublet associating a radar spectrum (or characteristics derived from the radar spectrum) of the clutter and the true nature of the ground surface illuminated during the acquisition of this spectrum (or an identifier corresponding to the true nature of the illuminated surface).

[0010] However, in addition to the nature of the illuminated surface, the characteristics of the clutter depend on the flight parameters of the aircraft, such as its height, its speed, or even radar operating parameters, such as its observation angles in elevation and bearing.

[0011] The constitution of the reference database then requires numerous in-flight measurement campaigns, in order to record varied spectra, then their analysis by expert personnel in order to label them with the true nature of the illuminated surface.

[0012] There is therefore a need for a method for automatically constructing, or at least with a minimum of human intervention, a reference database allowing training and validation of algorithms for identifying the nature of ground surfaces illuminated by radar, while limiting the number of in-flight measurement campaigns necessary to collect the spectra necessary for populating this database.

[0013] The aim of the invention is to meet this need.

[0014] For this purpose, the invention relates to a method, implemented by computer, of automatic labeling of data corresponding to a radar spectrum, the radar spectrum being acquired by means of a radar on board an aircraft and illuminating a ground surface, with a view to training an artificial intelligence algorithm for identifying the ground surface, the method consisting of: acquiring a radar spectrum of a domain observed by the radar; obtaining, by means of a reference system, a true identifier of a ground surface illuminated by the radar during the acquisition of the radar spectrum; and, associating, in a labeled data item, the data corresponding to the radar spectrum and the true identifier obtained.

[0015] According to particular embodiments, the method comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0016] - the reference system comprising at least one reference sensor, the step obtaining the true identifier consists of acquiring an image of the area observed by the radar at the time of spectrum acquisition, then analyzing the acquired image using a suitably trained artificial intelligence classification algorithm to determine a class to which the ground surface belongs from the acquired image, the true identifier being equal to the class thus determined.

[0017] - the reference system comprising at least one digital terrain model reference, the step of obtaining the true identifier consists of determining a location of the ground surface illuminated by the radar at the time of spectrum acquisition, then extracting from the reference digital terrain model the true identifier corresponding to the location determined for the ground surface illuminated by the radar.

[0018] - the method further comprises a step consisting of recording the labeled data in a training database, and, after populating the training database, a step consisting of training and / or validating, using batches of labeled data from the training database, the artificial intelligence identification algorithm automatically associating, with data corresponding to a radar spectrum, an identifier of the ground surface illuminated by the radar during the acquisition of said radar spectrum.

[0019] - the data corresponding to a radar spectrum being a raw radar spectrum, the method further comprising a pre-processing step making it possible, from the raw radar spectrum, to obtain a pre-processed radar spectrum, the pre-processed radar spectrum constituting the input to the artificial intelligence identification algorithm.

[0020] - the preprocessing step includes a step of normalizing the raw radar spectrum in order to obtain a standardized radar spectrum.

[0021] - the pre-processing step comprises a step of extracting one or more intermediate magnitude(s) of the radar spectrum, raw or preprocessed, so as to define a characteristic signature of the ground surface illuminated by the radar, the data corresponding to the radar spectrum being made up of said characteristic signature.

[0022] The invention also relates to a system for implementing the preceding method and a computer program comprising software instructions which, when executed by a computer of the preceding system, allow the implementation of the preceding method.

[0023] The invention and its advantages will be better understood on reading the detailed description which follows of a particular embodiment, given solely by way of non-limiting example, this description being made with reference to the appended drawings in which:

[0024] [Fig-1] [Fig. 1] is a schematic representation of the system for implementing the identification method according to the invention; and,

[0025] [Fig.2] [Fig.2] is a block representation of the process identification according to the invention.

[0026] [Fig. 1] represents a first embodiment of a system allowing automatic labeling of a radar spectrum with a label corresponding to the true nature of the ground surface illuminated by the radar during the acquisition of this spectrum.

[0027] The data thus labeled is recorded in a reference database 60 of the system 1.

[0028] The final purpose of the reference database 60 is to enable the training and / or validation of an artificial intelligence algorithm - identification AI 75, which, once correctly configured, is suitable for automatically identifying the nature of the ground surface illuminated by a radar from a spectrum acquired by this same radar.

[0029] Labeled data associates:

[0030] - a piece of data, constituting an input to the identification algorithm (in particular the data respects the format of the data expected as input to the identification algorithm 75 to be trained);

[0031] - a label corresponding to the true identifier to be associated with this data (in in particular the label respects the format of the results obtained at the output of the identification algorithm 75 to be trained).

[0032] The training process consists of progressively modifying the parameters of the algorithm 75 to be trained so that the identifier estimated by this algorithm on the data approaches the true identifier associated with this data in the labeled data.

[0033] The system 1 is advantageously carried on board an aircraft to carry out in-flight measurement campaigns making it possible to dynamically populate the reference database 60.

[0034] The system 1 comprises, in addition to the reference database 60, a radar 10, at least one reference sensor 20, and a computer 2.

[0035] The radar 10 allows, during an operating recurrence, the acquisition of a radar spectrum. The radar spectrum corresponds to the observation by the radar of a domain of space. For ground observation, this domain includes in particular a ground surface which is illuminated by the radar.

[0036] The radar 10 is for example a radar of the FMCW type.

[0037] At each recurrence, an FMCW radar emits, during an emission duration (or illumination time), a wave taking the form of a frequency ramp, then receives, during a reception duration, a received wave resulting from the reflection of the emitted wave on reflective objects present in the observation domain of the radar, in particular the surface of the ground.

[0038] A Fast Fourier Transform - FFT (Fast Fourier Transform) is performed on the background samples received for each recurrence.

[0039] A measurement of a frequency difference between the transmitted wave and the received wave makes it possible to deduce the distance between the reflecting object and the radar.

[0040] For this transformation, the frequency interval is subdivided into N frequency bins, with for example N equal to 1024. The frequency resolution of the FFT depends on the radar recurrence period and the number of bins. Frequency bins can therefore be considered whose width is equal to the frequency resolution of the FFT.

[0041] The radar spectrum obtained at the end of the FFT is an amplitude-frequency spectrum. It is characteristic of the nature of the ground surface illuminated by the radar.

[0042] This spectrum spreads in frequency proportionally to the opening of the radar beam and the height of the carrier aircraft.

[0043] The raw data (before processing) delivered by the radar therefore form, in the case of an FMCW radar, a two-dimensional matrix with the frequency (or distance) boxes on the abscissa and the amplitudes of the signals on the ordinate.

[0044] The reference sensor 20 is for example an optical sensor, such as a camera. It is for example a camera operating in the visible range. Alternatively, it may be a camera operating in the infrared or ultraviolet range.

[0045] The reference sensor is adapted to acquire an image of the domain observed by the radar at the time of acquisition of a spectrum. This image is an image of the ground surface illuminated by the radar at the time of acquisition of the radar spectrum.

[0046] The computer 2 is connected on the one hand to the radar 10 and on the other hand to the reference sensor 20. In the embodiment of [Fig.l], the computer 2 is on board the aircraft carrying the radar and the reference sensor. It can therefore be connected directly to the radar 10 and to the reference sensor 20.

[0047] Alternatively, the computer 2 is on the ground and communication with the radar and the reference sensor is carried out via a suitable air link.

[0048] The computer 2 executes a program, which can be subdivided into a plurality of functional modules.

[0049] The first module or preprocessing module 30, which is optional, is adapted to process the radar spectrum acquired by the radar 10 at the current time. For example, the module 30 comprises a sub-module 32 adapted to normalize the spectrum acquired by the radar 10 at the current time, or raw spectrum, so as to obtain a normalized spectrum. For example, the module 30 comprises a sub-module 34 adapted to extract a characteristic signature of the surface illuminated by the radar from the raw or normalized spectrum.

[0050] The second module or reference module 40 is adapted to execute an artificial intelligence classification algorithm 45, which is capable of determining, from the information acquired by the reference sensor 20 at the time of spectrum acquisition, the class to which the ground surface illuminated by the radar belongs.

[0051] For example, the artificial intelligence classification algorithm is adapted to classify the image acquired by the optical sensor into a plurality of predetermined classes. Each class corresponds to a possible nature of the ground surface.

[0052] As will become more clear during the presentation of the method according to the invention, the format of the outputs of the identification algorithm 75 constrains that of the outputs of the classification algorithm 45: the plurality of classes is chosen in such a way that each identifier of the plurality of identifiers constituting the possible outputs of the identification algorithm 75 corresponds to at least one class of the plurality of classes constituting the possible outputs of the classification algorithm 45.

[0053] It should be noted that image recognition by artificial intelligence algorithms is now very robust. Image databases for training such artificial intelligence algorithms are very numerous, which makes it possible to obtain a substantial training database. Confidence in the classification of the nature of surfaces seen from the sky is therefore very good, even excellent.

[0054] The third module 50 is adapted to associate the outputs of the first and second modules 30 and 40 in a labeled data and, preferably, to store this labeled data in the training database 60.

[0055] Advantageously, the computer 2 comprises a training module 70 making it possible to train the identification algorithm 75 from the database 60.

[0056] [Fig.2] represents a preferred embodiment of a labeling method according to the invention.

[0057] The method 100 makes it possible to automatically label a radar spectrum with a true identifier relating to the nature of the ground surface illuminated by the radar during the acquisition of this spectrum.

[0058] The method 100 begins with a step 110 consisting of acquiring a radar spectrum of a domain observed by the radar 10.

[0059] The observed domain has a ground surface illuminated by the radar. The domain may also include targets, the ground surface constituting a background. It is on this background that the targets are to be extracted by suitable radar processing. These can be optimized depending on the nature of this background.

[0060] Simultaneously, in step 120, reference information is acquired by means of the reference sensor 20. In the case of an optical sensor, the reference information is an image of the domain observed by the radar at the current time.

[0061] Then, in step 140, the image just acquired is analyzed by means of an artificial intelligence classification algorithm 45, having been previously suitably trained.

[0062] This classification algorithm is suitable for determining the class to which the ground surface appearing on the acquired image belongs. The classification algorithm is suitable for determining one class from a plurality of classes. The plurality of classes includes, for example, the classes “land”, “sea”, “lake”, “vegetation”, etc. For example, the sea class can be subdivided to take into account the state of the sea as identified on the image, to obtain, for example, classes “rough sea”, “calm sea”, etc.

[0063] Optionally, an additional class is provided which corresponds to images which could not be classified by the algorithm into one or other of the classes.

[0064] Finally, the method 100 comprises a step 150 consisting of associating the radar spectrum acquired in step 110 with the class determined in step 140 as a true identifier. of the ground surface illuminated by the radar. This association leads to a labeled radar spectrum.

[0065] In a step 160, the labeled spectrum obtained in step 150 is recorded in the training database 60 as new training data.

[0066] Once the training database 60 is sufficiently populated, in a step 170, an artificial intelligence identification algorithm 75 is trained and / or validated with batches of labeled spectra extracted from the database 60.

[0067] The identification artificial intelligence algorithm 75 takes a radar spectrum as input and delivers as output an identifier of the nature of the ground surface illuminated by the radar during the acquisition of this radar spectrum.

[0068] Once suitably trained, the artificial intelligence identification algorithm can be used to, from a radar spectrum newly acquired by the radar 10 (or an equivalent radar), automatically determine the identifier characterizing the ground surface illuminated by this radar. This identifier can be used to choose or optimize radar processing adapted to the radar spectrum to detect targets or any other useful information.

[0069] Alternatively, one or more newly labeled radar spectra, obtained at the output of step 150, are used immediately for training the artificial intelligence identification algorithm 75. This variant is particularly interesting for allowing, following a first training of the identification algorithm 75, an update of the parameters of this algorithm 75 in order to broaden its identification capabilities. This is therefore a “real-time” training, for example during an operational flight, as opposed to a “delayed-time” training, for example following a flight campaign for acquiring recordings making it possible to populate the training database and then subsequently carry out the training of the identification algorithm, for example on the ground on another computer.

[0070] Advantageously, it is not the raw radar spectrum that constitutes the input to the identification algorithm 75 but a standardized radar spectrum. This makes it possible to make the spectra applied as input to the artificial intelligence identification algorithm comparable.

[0071] Thus, in this advantageous variant, the method 100 comprises a step 130 of preprocessing the radar spectrum obtained at the output of step 110.

[0072] For example, in a step 132, the spectrum is normalized. This preprocessing consists of homogenizing the spectra according to a normalization criterion to obtain a normalized radar spectrum. This step is detailed at the end of this description.

[0073] For example again, it is not a radar spectrum, raw or normalized, which constitutes the input of the identification algorithm, but a set of intermediate quantities extracted from the radar spectrum and allowing the definition of a characteristic signature of the ground surface illuminated by the radar at the time of acquisition of the radar spectrum.

[0074] Thus, in this variant, step 130 comprises a step 134 consisting of extracting one or more intermediate quantity(ies) from the radar spectrum acquired in step 110. This may for example be a statistical quantity extracted from the spectrum, raw or normalized, or even minimum and maximum quantities in amplitude and / or frequency of the spectrum.

[0075] In the preferred embodiment, which has been presented in detail above, the radar is an FMCW radar, making it possible to obtain raw data taking the form of a 2D matrix.

[0076] Alternatively, the radar is a pulse radar. During a recurrence, a pulse radar transmits during a transmission period and receives over a reception period, segmented into distance boxes of equivalent lengths.

[0077] An FFT is carried out for each distance box on a block of recurrences (comprising for example 1024 boxes) [notion of recurrence block].

[0078] The resolution of the FFT depends on the recurrence period and the number of points.

[0079] It can therefore be considered frequency boxes of the width of the frequency resolution of the FFT.

[0080] The raw data (before processing) therefore form a 3D matrix here with the distance boxes in depth, the frequency boxes in abscissa and the amplitudes of the signals in ordinate.

[0081] For a pulse radar, the spectrum of interest is located on one or more distance boxes. It is therefore necessary to wait for the end of the acquisition on the N recurrences necessary for the FFT, before launching the processing dedicated to the recognition of the nature of the illuminated surfaces.

[0082] This is in contrast to the case of an FMCW radar, for which the data are available at the end of each recurrence and presented in 2D, frequency and amplitude.

[0083] The algorithm trained with one type of radar cannot be used for the other type.

[0084] The labeling step and / or the learning step can be carried out during the flight or on the ground after in-flight recording.

[0085] This learning avoids information leakage phenomena (or “data leakage” in English) because it only compares the output values ​​of sensors A and B.

[0086] Those skilled in the art will note that the method according to the invention ultimately consists of teaching a radar system to identify the nature of the ground surface illuminated by the radar, based on the identification results given by another reference system, in this case optical, taken as a reference. This reference system is independent of the radar.

[0087] It is therefore supervised learning with criteria from another system already trained. There is therefore a transfer of the experience acquired by the reference system, or teacher system, to the radar system, or student system.

[0088] The present method has many advantages.

[0089] It makes it possible to limit the number of in-flight measurement campaigns necessary for populating a training database so that this database is sufficiently large and complete to lead to satisfactory training of the artificial intelligence identification algorithm.

[0090] The step of normalizing and / or extracting a signature from the raw spectrum makes it possible to reduce the space of radar spectra, which facilitates the identification and training of the algorithm. Consequently, this makes it possible to further reduce the number of flight campaigns required to populate the training database.

[0091] The labeled data is generated automatically, without human intervention. The method according to the invention makes it possible to dispense with the need for experts capable of recognizing the nature of surfaces seen from the sky from radar data.

[0092] The field of application of the invention is that of airborne or satellite radars, for observation, surveillance, etc.

[0093] It can also find an application for FMCW radars used as radio altimeters. Determining the nature of the surface illuminated by the radar makes it possible, for example, to improve its accuracy or increase it with additional functionalities.

[0094] In a second embodiment of the invention, the reference system, instead of implementing a sensor and an artificial intelligence algorithm, is based on the use of a digital reference terrain model.

[0095] More precisely, at the time of acquisition of a spectrum by the radar, the position of the carrier (longitude, latitude and altitude - or height), as well as the elevation angle and possibly the aperture of the radar are recorded. The position of the carrier can for example be obtained by a satellite positioning device on board the aircraft carrying the radar system.

[0096] From this data, the location of the ground surface illuminated by the radar is determined.

[0097] A reference digital terrain model is then used. This associates with each point on the terrain an identifier indicating the nature of the ground surface at that point. It is a kind of topographic database.

[0098] Thus, the location of the surface illuminated by the radar makes it possible, by interrogating the reference digital terrain model, to obtain an identifier to be associated with the ground surface illuminated by the radar.

[0099] This identifier is used as a true identifier to label the acquired spectrum and create a new entry in the training database.

[0100] This second embodiment is particularly simple to implement, but cannot allow identification of the nature of the ground surface, when this nature corresponds to the current state of the ground surface, such as for example the state of a sea surface.

[0101] To do this, it would then be necessary to combine the information obtained with the digital terrain model, with a source of meteorological and / or marine information to determine the state of the sea and thus obtain a true identifier making it possible to distinguish a sea surface according to its different states.

[0102] In a variant, the radar system 20 is on board an aircraft while the computer 2 and the database 60 for the automatic labeling of spectra are located on the ground. The radar system transmits the spectra that it acquires to the computer on the ground, with the necessary information, i.e. the image(s) of the surface on the ground illuminated by the radar for the first embodiment, and the location of the aircraft and the observation parameters of the radar for the second embodiment. It is on the ground that the computer processes each spectrum to label it. This variant is particularly well suited to the second embodiment where the reference system uses a terrain model, which can therefore be stored by the computer on the ground.

[0103] In the following, one way of carrying out the normalization step is presented.

[0104] The amplitudes and frequency spread of a spectrum generally depend on the height H of the carrier, the elevation angle (also called the elevation angle) a of the beam, and the aperture & of the beam.

[0105] For the particular case of an FMCW radar, for which the beam has a fixed orientation and a predefined aperture, the characteristics of a spectrum no longer depend only on the height.

[0106] In absolute terms, it would be necessary to record spectra for different configurations in the database, i.e. several heights, several elevation angles and several apertures. But this solution would then require acquiring a large number of spectra and, consequently, would increase the necessary number of flight measurement campaigns.

[0107] It is therefore preferable that the identification is done on standardized rather than raw spectra.

[0108] For a fixed antenna aperture and beam site, the greater the height, the more the spectrum is spread (due to the increase in distances) and the lower the amplitudes.

[0109] Normalization then consists of reporting the different raw spectra on the same scales in amplitudes and frequencies. The scales are for example those of a calibration spectrum.

[0110] To normalize a raw spectrum along the frequency axis, its spectral spread Es is compensated.

[0111] It is shown that the spectral spread is written, as a function of the height, the elevation angle and the aperture: [OUZ] p -wul__1 I ] ° \ cos^a-Hjj cos^aH;) /

[0113] with K a coefficient equal to the ratio of the emitted frequency band to the recurrence period, H the height, a the elevation angle and 0 the beam opening

[0114] The first step therefore consists of normalizing the raw spectrum along the frequency axis by applying a scale factor ESq / E , where Es is the spectral spread of the raw spectrum and Eso is the spectral spread of the calibration spectrum.

[0115] The second step consists of adjusting the amplitudes of the raw spectrum along the amplitude axis by applying a scale factor JÉ, where A is the maximum amplitude of the raw spectrum and A0 the maximum amplitude of the calibration spectrum.

[0116] In a third step, the normalized spectrum is obtained by taking a sample on R, with = E, where D is the maximum frequency of the raw spectrum and D0 the maximum frequency of the calibration spectrum.

Claims

Claims

1. A computer-implemented method (100) for automatically labeling data corresponding to a radar spectrum, the radar spectrum being acquired by means of a radar (10) on board an aircraft and illuminating a ground surface, with a view to training an artificial intelligence algorithm for identifying (75) the ground surface from data corresponding to a radar spectrum, the method consisting of: - acquiring (110), by means of the radar, a radar spectrum of an observed domain;- obtaining, by means of a reference system, called a teacher system, the teacher system comprising at least one reference sensor (20) and a classification artificial intelligence algorithm (45), a true identifier of a ground surface illuminated by the radar during the acquisition of the radar spectrum, by carrying out an acquisition, by means of the reference sensor, of an image of the domain observed by the radar at the time of the acquisition of the spectrum, then by analyzing (140) the acquired image by executing the classification artificial intelligence algorithm (45) suitably trained to determine a class to which the ground surface belongs from the acquired image, the true identifier being equal to the class thus determined; and, - associating (150), in a labeled data item, the data item corresponding to the radar spectrum and the true identifier obtained.;

2. Method according to claim 1, further comprising a step (160) consisting of recording the labeled data in a training database (60), and, at the end of the population of the training database (60), a step (170) consisting of training and / or validating, using batches of labeled data from the training database (60), the artificial intelligence identification algorithm (75) automatically associating, with a data item corresponding to a radar spectrum acquired by the radar or a radar equivalent to said radar, an identifier of the ground surface illuminated by the radar or the equivalent radar during the acquisition of said radar spectrum.

3. A method according to any one of claims 1 to 2, wherein the data corresponding to a radar spectrum is a radar spectrum raw, the method further comprises a pre-processing step (130) making it possible, from the raw radar spectrum, to obtain a pre-processed radar spectrum, the pre-processed radar spectrum constituting the input of the artificial intelligence identification algorithm.

4. The method of claim 3, wherein the preprocessing step comprises a step (132) of normalizing the raw radar spectrum so as to obtain a normalized radar spectrum.

5. Method according to any one of claims 3 to 4, in which the pre-processing step comprises a step (134) of extracting one or more intermediate quantities from the radar spectrum, raw or pre-processed, so as to define a characteristic signature of the ground surface illuminated by the radar or the equivalent radar during the acquisition of said radar spectrum, the data corresponding to the radar spectrum being made up of said characteristic signature.

6. System (1) for implementing a method (100) according to any one of claims 1 to 5, the system comprising a radar (10) and a reference system (20).

7. A computer program comprising software instructions which, when executed by a computer (2) of a system according to claim 6, implement a method (100) according to any one of claims 1 to 5.

Images