Method for determining a free surface elevation of an area of a body of water

Abstract

The invention relates to a method for determining a free surface elevation (ESL) of an area of a body of water: 1) forming a training database of pairs of images (W, R), each pair consisting of an FSE image (W) and a radar return image (R), the two images of a single pair corresponding to a single area and an identical or similar point in time; 2) adding, to each pair of images (W, R), spatial characteristics of the area, comprising the position with respect to the radar and to the main direction(s) of the waves; 3) training deep neural networks with this database, using the spatial characteristics to specialise the neural networks and obtain an FSE reconstruction model; 4) acquiring real return images by means of a radar; and 5) applying the model to the real radar return images to obtain the FSE.

Description

[0001] Method for determining the elevation of the free surface of an area of ​​a body of water

[0002] Technical field

[0003] The present invention relates to the field of wave characterization, in particular for the monitoring, operation and control of a system subject to waves, whether floating or not.

[0004] In order to characterize waves within a body of water (for example: a sea, an ocean, a lake), we use in particular time series of elevation of the free water surface. This elevation of the free surface gives the water height (therefore the wave height), relative to the situation where the surface is not disturbed (flat sea, for example), at any point of the water surface. The measurement of the elevation of the free surface is a widespread problem, in order to respond to several challenges. One of the first challenges is the fine characterization (at the scale of individual waves) of the state of the sea, which is most often characterized by descriptions of the statistical distribution of the heights of individual waves, such as the spectrum, the significant height and the average period.These descriptors are commonly used for navigation monitoring, offshore operations monitoring, energy production platform monitoring, and site monitoring for the exploitation of marine energies. Having a free surface elevation measurement not only allows for more precise statistical characterizations, but also for short-term predictions (from a few seconds to a few minutes) of the wave height and the resulting movements of a floating system. These predictions make it possible, for example, to detect the arrival of a wave train that is potentially dangerous for a given operation (for example, the transfer of personnel on board a wind turbine or a boat) or, on the contrary, a period of calm, which would allow the operation in question to be carried out.Another challenge of wave and swell predictions made from the free surface elevation is the control of floating systems (in particular by means of predictive control): for example, to ensure the stabilization of a vessel, to ensure the compensation of a movement, for example heave compensation (which can be interesting for a floating hydrocarbon production platform), to ensure the real-time control of floating wind turbines or wave energy systems (in particular with the aim of maximizing the energy produced and / or to reduce the fatigue of the components of such systems).

[0005] Prior Art Several technologies have been developed to measure and determine free surface elevation.

[0006] Some of these solutions allow the measurement of a characteristic resulting from waves or swell at only one measurement point, for example by means of an instrumented buoy, which does not allow an indication of the elevation of the free surface at any point in a body of water.

[0007] Maritime radars, such as X-band radars, which are found on all large ships and many offshore installations, are a particularly interesting technology for measuring waves at a distance, although their primary use was navigation and collision avoidance. Images produced by maritime radars detect not only targets or obstacles such as ships and coastlines, but also reflections from the sea surface, known as "sea clutter," caused by the interaction between radar waves and small ripples in the surface caused by the wind. Backscatter from this rough surface reveals the underlying shape of the waves.With appropriate processing, it is in principle possible to carry out wave measurements over a wide field of view (up to about 5 km range, depending on the installation height of the radar antenna and the sea state), with good spatial resolution (of the order of 5 m in distance and 1° in azimuth) and sufficient temporal resolution (one image every 1 to 3 seconds) to track individual waves.

[0008] More precisely, the principle of wave measurement by radar is as follows: the electromagnetic waves emitted by the radar interact, by virtue of Bragg's law, with the ripples (variations in height) of the water surface, whose wavelength is of the order of centimeters (like those of radar waves). This results in a backscattered signal, the sea clutter or sea echo, which returns to the radar receiver, and whose intensity is modulated by the waves (which can have a length of between ten and several hundred meters) via a set of mechanisms that are not yet all well understood. Radar images therefore present patterns that resemble waves, and from which these can theoretically be reconstructed, with appropriate processing.

[0009] Radar images can provide sea state estimates, i.e., statistical information on wave height and energy content according to frequency and direction. But they should also allow us to go much further, with a true “wave-by-wave” reconstruction of the free surface elevation (ESL), like a three-dimensional movie of the sea surface. Such wave-by-wave reconstruction, with the vast range and resolution of radar, lends itself to a wide range of applications. It is particularly ideal for monitoring purposes in the field of marine renewable energies, and paves the way for real-time prediction of waves, or the movement of a vessel, over horizons of several minutes, thus improving the feasibility and safety of a large number of operations at sea.

[0010] However, despite their promising prospects, radars only provide a very indirect measurement of waves, via images, or intensity signals, of sea clutter. The intensity of sea clutter is modulated by the orbital velocity of the fluid at the surface and the angle of incidence of the radar beam on the sea surface, which can be linearly related to the free surface elevation that we are trying to measure. Other wave-related factors complicate this modulation, in particular shadowing effects (when certain areas of the sea surface are geometrically hidden by waves closer to the radar) or the presence of micro-breaking.In addition to these wave-induced modulations, there are other factors that alter the wave signal, in particular the amplification function of the signal received by the radar, the presence of speckle noise, or even meteorological factors such as rain or sea spray, which can generate backscattering.

[0011] To circumvent this problem, wave reconstruction from radar images generally relies on the "standard method" which is described in particular in the following documents:

[0012] Young, I.R., Rosenthal, W., & Ziemer, F. (1985). A three-dimensional analysis of marine radar images for the determination of ocean wave directionality and surface currents. Journal of Geophysical Research: Oceans, 90(C1), 1049-1059

[0013] Nieto Borge, J., Rodriguez, G. R., Hessner, K., & Gonzâlez, P. I. (2004). Inversion of marine radar images for surface wave analysis. Journal of Atmospheric and Oceanic Technology, 21 (8), 1291-1300.

[0014] This method does without an explicit model for the formation (and inversion) of radar images: in this approach, the radar field of view is divided into rectangular areas, in which a three-dimensional Fourier transform of the signal is applied. The wave-related components of the signal are then identified and filtered using the dispersion relation for gravity waves, which relates frequency to wavelength. The components thus obtained are then considered to be linearly related to the components of the free surface elevation signal. Finally, the reconstructed signal needs to be calibrated in amplitude, to obtain sea state images with the correct energy.As we can see, the standard method avoids the need for detailed knowledge of image formation mechanisms, but relies on a considerable number of empirical parameters, such as filtering parameters, modulation transfer function parameters, and amplitude calibration parameters. Optimization or verification of these parameters may rely on the use of additional sensors, such as instrumented buoys, a ship's inertial unit, laser remote sensing (LiDAR sensor), or a microseismic wave sensor on the shore.

[0015] Another avenue for improving the standard method is to use coherent radars, which also detect the Doppler signal due to surface motion. Indeed, the Doppler velocity can, in principle, be used to reconstruct the free surface elevation without going through the calibration step via a modulation transfer function; however, to obtain stable Doppler signals, the radar must observe the same points over a relatively long period, which makes the proposed measurement procedures quite complex. Moreover, the vast majority of current maritime radars do not have this feature.

[0016] The standard method is nothing more than linear filtering. However, the relationship between surface elevation and sea clutter is highly non-linear. The performance achievable by the standard method is therefore inherently limited. Finally, this filtering is performed a posteriori, which limits its applicability in real time. It would be preferable to perform a "frame-by-frame" inversion of the sea clutter, which would open the way to a real-time application.

[0017] Furthermore, sea clutter is highly inhomogeneous, depending on the considered area of ​​the radar field of view. Near the radar, sea clutter is mainly modulated by the radial slope of the free surface elevation, while far from the radar, it is the occlusion (or masking) of troughs and small waves by larger waves in front that dominates the image formation. Furthermore, sea clutter patterns are very pronounced along the main wave propagation axis, but much less so along the axis perpendicular to it. Thus, a method for reconstructing the free surface elevation (FSE) that takes into account the location in the radar field of view is desirable.

[0018] In order to overcome the limitations of the standard method, the following works focus on the frame-by-frame inversion of sea clutter using deep learning methods to solve highly nonlinear problems: - Ehlers, Svenja; Klein, Marco; Heinlein, Alexander; Wedler, Mathies; Desmars, Nicolas; Hoffmann, Norbert; Stender, Merten (2023) Machine learning for phase-resolved reconstruction of nonlinear ocean wave surface elevations from sparse remote sensing data. In: Ocean Engineering, vol. 288, p. 116059. DOI: 10.1016 / j.oceaneng.2023.116059. - Zhao, Mingxu; Zheng, Yaokun; Lin, Zhiliang (2023) Sea surface reconstruction from marine radar images using deep convolutional neural networks. In: Journal of Ocean Engineering and Science, vol. 8, no. 6, p. 647-661. DOI: 10.1016 / j.joes.2023.09.002.

[0019] However, neither of these two works adjusts the inversion model based on the location in the radar field of view. Moreover, the first of these two references only performs inversion along a radial section of a radar field of view, not on a surface.

[0020] Furthermore, the method described in patent FR3108152 (corresponding to patent applications WO2021 / 180502 and US 2023 / 0167796) implements one or more sensors (for example a radar, a LiDAR, an accelerometer, a displacement sensor, a pressure sensor, etc.) measuring the free surface elevation of the swell or the resulting characteristics at one or more points and deduces predictions of the swell or resulting characteristics of the swell by means of transfer functions. This method allows a good prediction of a resultant of the swell but does not allow the free surface elevation to be determined at any point in an area from a radar signal.

[0021] The present invention therefore aims to develop an improved method for determining the free surface elevation of a body of water using radar. This involves, in particular, overcoming the drawbacks of previous solutions by taking into account the spatial variability of sea clutter in the radar's field of vision.

[0022] Summary of the invention

[0023] The invention firstly relates to a method for determining a free surface elevation (ESL) of an area of ​​a body of water, such that, according to said method,

[0024] 1) - at least one learning database is constituted of a plurality of image pairs (W,R), each pair consisting of a first image (W) of free surface elevation (ESL) and a second radar return image (R), the two images of the same pair corresponding to the same area of ​​the surface of the body of water and to an identical or close instant in time, said images (W) and (R) being real or simulated

[0025] 2) - each pair of images (W,R) is added spatial characteristics of said area of ​​the surface of the body of water corresponding to this pair, spatial characteristics comprising at least the position relative to the radar and to the main direction(s) of the waves, said spatial characteristics being directly known during the acquisition of the images, or being deduced from the analysis of the images themselves, or even provided by an external source.

[0026] 3) - at least one deep neural network, in particular of the convolutional type, or "transformer" or Fourier neural operator, is trained on at least said training database, using said spatial characteristics to specialize said deep neural network(s), and to obtain a free surface elevation (ESL) reconstruction model.

[0027] 4) - real return images are acquired by radar

[0028] 5) - said reconstruction model is applied to said real radar return images acquired to obtain the free surface elevation (ESL).

[0029] For the purposes of the invention, “near” time means a time that is different but shifted by an instant less than the rotation period of a radar, in particular by an instant less than three seconds.

[0030] Advantageously, the first image (W) of each pair of images (W,R) can be obtained by discretizing three-dimensional time series of free surface elevation (ESL) generated by a numerical wave field simulation code, and the second image (R) of each pair of images (W,R) can be obtained by discretizing radar returns generated by a numerical radar simulation code.

[0031] The first images (W) of each image duo (W,R) may come from actual free surface elevation (ESL) measurements over at least part of the surface covered by the radar. They may be produced in particular with a remote sensor of the LiDAR type (an acronym for "Light Detection And Ranging" designating a remote measurement technique based on the analysis of the properties of a light beam returned to its transmitter) or a stereoscopic camera system or a set of measurement buoys.

[0032] Second radar return images (R) may come from actual measurements made by at least one radar, including an X-band or S-band maritime radar.

[0033] The training database may comprise a first base of duos of surface elevation images (W,R) and radar return images (R), all of the images of which are simulated, and a second base of duos of surface elevation images (W,R) and radar return images, all of the images of which are real, the relative weight of said two bases possibly being able to be adjusted.

[0034] Said spatial characteristics may include a distance to an origin, said origin corresponding in particular to the position of the radar, and an angular orientation, in particular an azimuth.

[0035] The or at least one of the deep neural networks can be an encoder-decoder network, for example of U-Net architecture. To obtain the reconstruction model of the free surface elevation (ESL), we can proceed as follows:

[0036] - we divide each pair of images (W,R) into n distinct zones, according to spatial characteristics, n being greater than or equal to 2, and in particular being equal to 4

[0037] - we use n deep neural networks, each deep neural network being specialized on a dedicated area of ​​image duo,

[0038] - we train each neural network on its dedicated image duo area (W,R).

[0039] In this case, we can cut the said image duos (W,R) into n zones according to the azimuth and the distance to the radar.

[0040] To obtain the said reconstruction model, we can proceed as follows:

[0041] - at least one first guide image (Rg1) is formed for each radar return image (R) of the learning base by means of a radar distance map, and the deep neural network is trained by means of said surface elevation image (W) and said at least one associated first guide image (Rg1).

[0042] In this case, at least one second guide image (Rg2) can be formed using an angular orientation map, and the deep neural network is trained also using said associated second guide image (Rg2).

[0043] The method for determining a free surface elevation (ESL) of an area of ​​a body of water according to the invention can implement the following steps:

[0044] A1) a first database of images (W) obtained from three-dimensional time series of free surface elevation, in particular the elevation (z), the position in Cartesian coordinates (x,y) or polar coordinates (r, <|>), and the time (t), generated by a wave field simulation code corresponding to different sea states, A2) a second database of so-called return images (R) is constructed obtained from radar returns generated by a radar simulation code using the time series generated by the wave field simulation code, parameterized according to environmental conditions, in particular the wind speed and direction and the presence of precipitation

[0045] A3) a learning base is constructed comprising a first set of images from the first database (W) constructed in step A1) and a first set of corresponding return images from the second database (R) constructed in step A2) by integrating the environmental conditions used in step A2), so as to obtain image pairs (W,R) each consisting of a first surface elevation image (W) and a second radar return image (R), the two images of the same pair corresponding to the same area of ​​the surface of the body of water and to an identical or close instant in time,

[0046] A4) to each pair of images (W,R) of the learning database are added the spatial characteristics of the area of ​​the surface of the body of water corresponding to this pair, said spatial characteristics comprising a distance to an origin, said origin corresponding in particular to the position of the radar, and an orientation, in particular in azimuth, relative to the main direction(s) of the waves

[0047] A5) a free surface elevation reconstruction model is constructed from the training database obtained in step A3) and the spatial characteristics specified in step A4), by training at least one deep neural network of the convolutional or “transformer” type, in particular of the U-net type or a transformer architecture adapted to vision, or a Fourier neural operator, said network(s) being specialized with the spatial characteristics obtained in step A4) A6) real images are acquired using at least one radar, in particular maritime, to which their spatial characteristics defined in step A4 are added

[0048] A7) the free surface elevation is determined by applying the free surface elevation reconstruction model obtained in step A5) at least to the real images acquired with the radar in step A6) with their spatial characteristics.

[0049] Preferably, in step A1) of constructing the first image database, the images are generated with scanning parameters comprising a distance (range) discretization Ar, an azimuth angle discretization A< > in a polar reference frame centered around the radar, and a time discretization At = Ol j « is | a speed of radar rotation in rpm.

[0050] Preferably, in step A2) of constructing the second return image database, the radar simulation code is parameterized according to environmental conditions chosen from at least: the presence or absence of wind, the wind speed, the wind direction, the presence or absence of precipitation such as rain, the intensity of precipitation, the presence or absence of fog, the temperature and the state of the sea.

[0051] Advantageously, in step A2) of constructing the second return image database, each image can be generated with spatial discretizations Ar and A< > and temporal discretizations At according to the same digitization parameters as those used in step A1) of constructing the first image database.

[0052] Advantageously, in step A5) of constructing the free surface elevation reconstruction model, said model can be validated with a validation database composed of a second set of images from the first database constructed in step A1) and a second set of corresponding return images from the second database constructed in step A2), said sets being different from those used for learning.

[0053] Advantageously, in step A6) of acquiring real images using at least one maritime radar, said images can be recorded, and, in step A7) of determining the free surface elevation (ESL), a means of communication, in particular a computer means, can be used to access said recorded radar images.

[0054] Steps A1) to A5) can be carried out at least without connection to a communication network, such as an intranet or the Internet.

[0055] Step A7) can be carried out without connection to a communication network, on real radar images acquired in step A6) and pre-recorded. Alternatively, steps A6) and A7) can be carried out by connection to a communication network, with acquisition and recording of the radar images in step A6) and use of said radar images in step A7) as they are recorded.

[0056] The method according to the invention can also implement the following additional steps which are carried out between step A5 and step A6):

[0057] BO) free surface elevation (ESL) measurements are carried out on at least part of the area covered by the radar, in particular maritime, simulated in step A1), in particular by means of a stereoscopic camera system, a LIDAR or a set of measurement buoys

[0058] B1) we build a database of images obtained by digitizing the actual wave measurements obtained in step BO),

[0059] B2) a database of images obtained from the real return signals obtained with a maritime radar is constructed, synchronized in time with the image database constructed in step B1),

[0060] B3) a real learning database is constructed from a set of images of measurements of the free surface state ESL from the database obtained in step B1) and from a set of images of real radar returns from the database obtained in step B2), in the form of image pairs,

[0061] B4) we add to each pair of images from step B3) associating real measurements of free surface elevation ESL, and real radar returns, the spatial characteristics of the corresponding area of ​​the water surface, in particular in terms of distance and direction relative to the main wave direction(s)

[0062] B5) The free surface elevation reconstruction model constructed in step A5) is made more robust from the real training database constructed in step B3) and the spatial characteristics determined in step B4), using machine learning or deep learning methods, on areas that match real measurements and real radar returns.

[0063] (By "more robust" we mean that by applying this reconstruction algorithm to real radar data, the reconstruction errors of the free surface elevation ESL are lower than those that would have been obtained with the algorithm trained only on simulated data).

[0064] B6) real images are acquired using at least one radar, in particular maritime B7) Step A7) is replaced by step B7) by applying the improved free surface elevation reconstruction model obtained in step B5) at least to the real images acquired with the radar in step B6).

[0065] To perform step B5), one can use the real data from step B3) directly with the simulated data from step A3) and perform training from scratch in the same way as in step A5), or use a so-called "transfer learning" approach, in which the network pre-trained in step A5) is adjusted on the real data base constructed in step B4).

[0066] Steps BO) to B5) can be carried out at least offline, without connection to a communications network, after having carried out steps A1) to A5).

[0067] The invention also relates to a method for monitoring, operating or controlling a system subject to waves within a body of water, in which the following steps are implemented:

[0068] - the elevation of the free surface of said body of water is determined by means of the free surface elevation determination method as described above; and

[0069] - the said system subject to waves is supervised, operated or controlled according to the elevation of the free surface of the said body of water thus determined.

[0070] The invention also relates to a computer, server or calculator configured to implement the method for determining the elevation of a free surface as described above.

[0071] The invention also relates to a computer program product downloadable from a communication network and / or recorded on a medium readable by a computer, a server or a calculator and / or executable by a processor, comprising program code instructions for implementing the method for determining the free surface elevation as described above, when said program is executed on a computer, a server or a calculator.

[0072] The invention also relates to a storage medium readable by a computer, server or calculator and storing instructions, which, when executed by a computer, server or calculator, imply that the computer, server or calculator implements the method for determining the free surface elevation as described above.

[0073] The invention therefore makes it possible to reconstruct wave fields from radar images using deep neural networks, for example convolutional networks, which are specialized by direction and distance: the main direction(s) of the wave are determined on the basis of the raw radar signals (images), then deep neural networks (or other model derived from machine or deep learning) are applied, specialized according to the directions resulting from the previous analysis and the distance between the measurement made by the radar and the radar itself.

[0074] In fact, since deep neural networks provide a prediction that is intrinsically translation invariant, information indicating how far from the origin the data is located, as well as its orientation, can improve the prediction quality: the integration of this type of spatial information, which is not explicitly contained in the data, into the training of neural networks had never been proposed until now.

[0075] In summary, the invention thus determines, in real time, the free surface elevation at any point in a body of water area, in a precise and simple manner, from images of the body of water area provided by a radar. For this, the invention implements a learning campaign to be carried out, in particular offline. This learning campaign is carried out from data from a radar simulator, in particular synthetic (simulated) free surface elevations and synthetic (simulated) radar returns calculated by said simulator. It makes it possible to construct a free surface elevation reconstruction model from radar images, based on deep neural networks (or another machine learning method, or deep learning), networks which are preferably specialized or guided by direction and distance of the radar return (echo or sea clutter).

[0076] The invention proposes to integrate information indicating the distance from the origin of the data, as well as its orientation, into the training of neural networks. Several implementations / variants of the invention are possible, in particular

[0077] - using at least n (n > 2) deep neural networks on as many different areas of the image, or

[0078] - using a single neural network, modified from the standard formulation, guiding it from spatial maps informing it of the distance to the radar position and / or the direction(s) of wave propagation, or

[0079] - by combining the two approaches (several spatially guided neural networks).

[0080] Other characteristics and advantages of the method and system according to the invention will appear on reading the following description of non-limiting examples of embodiment, with reference to the figures appended and described below.

[0081] List of figures

[0082] Figure 1 is an example of a radar image obtained by an X-band radar system on board a ship.

[0083] Figure 2 is an image obtained from a simulated wave field.

[0084] Figure 3 is an image obtained from simulated radar returns.

[0085] Figure 4a is a cross-section of a radar image for an azimuth of 10°.

[0086] Figure 4a is a cross-section of a radar image for an azimuth of 90°.

[0087] Figure 5 illustrates the relationship between V waves and R radar images

[0088] Figure 6 represents a synoptic of free surface elevation reconstruction by a deep neural network RN.

[0089] Figure 7 is an application of the invention with four different neural networks RN1 to RN4, specialized according to the azimuth range and the range (distance) range.

[0090] Figure 8a is a representation of reconstruction obtained with a single deep neural network (here convolutional U-Net type).

[0091] Figure 8b is a representation of reconstruction obtained with four deep neural networks (here convolutional U-Net type) specialized by area of ​​the image. Figure 9 represents in the form of a block diagram the method of the invention according to a first embodiment.

[0092] Figure 10 represents in block diagram form the method of the invention according to a second embodiment.

[0093] Figure 11 represents Radar and Guide Images, with at the top, the initial radar image (in polar representation) on the left, with the corresponding guide on the right (distance function, intensity from black to white indicating low to high distances respectively), and at the bottom the radar image in Cartesian representation on the left, and the corresponding guide on the right.

[0094] The references keep the same meaning from one figure to another.

[0095] Description of the embodiments

[0096] The invention relates to a method for determining the free surface elevation of an area of ​​a body of water. It is recalled that the free surface elevation is the height of water at a point of a body of water relative to the water surface which would have no variation in height (which would not be disturbed in any way). Thus, the free surface elevation reflects the height of the waves or the swell. A body of water can be a sea, an ocean, a lake, a river, etc. The area considered by the method according to the invention is an area of ​​interest, in particular for the monitoring, operation or control of a system subjected to waves. The extent of this area of ​​interest may depend on the system subjected to waves. Alternatively, the extent of this area of ​​interest may correspond to a measurement area of ​​the radar implemented in the method.

[0097] Figure 5 illustrates the relationship between waves V and radar images R which correspond to sea clutter Fm, the reconstruction model H according to the invention aiming to obtain image sequences or a 3D movie of the sea surface from a sequence of radar images.

[0098] The method for determining the free surface elevation of an area of ​​a body of water according to the invention can in particular be implemented by computer and can be used for monitoring, operating, or controlling a system subject to waves within said body of water.

[0099] The method according to the invention presents, in particular, two non-limiting embodiments. The first embodiment is shown schematically in Figure 9, and the second embodiment in Figure 10.

[0100] The references in the two figures have the following meanings:

[0101] A: Wave field simulation

[0102] B: Simulation of radar returns

[0103] C: Construction of the free surface elevation image database D: Construction of the radar return image database E Construction of the training database

[0104] E': Enlargement of the training database

[0105] F: Acquisition of radar images from a real radar

[0106] G: Recognition of spatial characteristics of images

[0107] H: Construction of a free surface elevation reconstruction model

[0108] H': Improvement of the free surface elevation reconstruction model

[0109] I: Determination of the free surface elevation

[0110] J: Acquisition of direct free surface elevation measurements over at least part of the area covered by the radar

[0111] K: Construction of the directly measured free surface elevation image database

[0112] 1: discretization parameters from the radar digitization characteristics

[0113] 2: 3D time series of free surface elevation

[0114] 4: Simulated radar return images

[0115] 5: Simulated environmental conditions

[0116] 6: Image duos (simulated)

[0117] 6': duets of (real) images

[0118] 7: Image pairs, conditions

[0119] 8: images of real radar returns

[0120] 9: spatial characteristics

[0121] 10: Reconstruction model

[0122] 10': Improved reconstruction model

[0123] 11: Real environmental conditions

[0124] In X-band maritime radar systems, the signals corresponding to radar backscatter, after post-processing, are sent to a screen, as the radar beam rotates. Most modern radars allow these signals to be digitized and recorded, making them accessible. Generally, the backscatter signals recorded during an antenna rotation are grouped into a single image, a kind of "snapshot" of the sea clutter. Figure 1 represents such an example of a radar image obtained by an X-band radar system on board a ship. In terms of image processing, the problem of reconstructing the wave surface elevation consists of finding from a radar image R, a "snapshot" of the sea clutter, an image of the wave field W, a "snapshot" of the free surface elevation in the area covered by the radar.

[0125] A first embodiment of the invention will be described with the aid of figure 9: from discretization parameters derived from the digitization characteristics of a radar 1, by simulating wave fields A, series of images 2 of 3D time series of simulated free surface elevation are obtained, and, from these series of images, and taking into account simulated environmental conditions 5, radar returns B are simulated: images of associated simulated radar returns 4 are obtained. A database of free surface elevation images C and a database of radar return images D are then constructed.By associating the two databases C and D and by introducing simulated environmental conditions 5, a training database E is constructed, comprising image pairs 6 (preferably at least 3 pairs) associating simulated images and their simulated radar returns and the environmental conditions 7 corresponding to the image pairs. A recognition G of the spatial characteristics 9 of the images is carried out, which is used to construct H a reconstruction model 10 of free surface elevation, which will allow, with the acquisition F of radar images from a real radar leading to the obtaining of real radar return images 8, the determination I of the free surface elevation taking into account the real environmental conditions 11.

[0126] A second embodiment of the invention will be described using Figure 10. This second embodiment makes it possible to improve the results obtained by enlarging the learning database with real images in the following way:

[0127] An acquisition J of direct free surface elevation measurements is made over at least part of the area covered by the radar, the images 12 of direct free surface elevation measurements are digitized for the construction K of a database of directly measured free surface elevation images. Furthermore, the acquisition F of real radar images 8 is made from a real radar F, and a database D' of real radar return images synchronized with the direct measurements is constructed. The pairs of real images obtained 6' enlarge, E', the initial learning database E obtained according to the previous embodiment (see figure 9).

[0128] From the image pairs 6 and 6' and the corresponding environmental conditions 7, the spatial characteristics of the images 9 are recognized G, which make it possible to improve H' the free surface elevation reconstruction model 10', in order, as in the previous embodiment, to acquire F real radar images 8 from a real radar, and to obtain the determination of the free surface elevation from the improved model 10' and the real return images 8, under real environmental conditions 11.

[0129] The free surface elevation determination system applying the method according to the invention may comprise means of communication between different devices. Advantageously, the free surface elevation determined by the system may be shared normally on a computer communication bus, in particular a computer communication bus connected to a system exploiting this measurement in real time or not (for example a system for monitoring, operating or controlling a system subject to waves).

[0130] Furthermore, the invention relates to a method for monitoring (supervision), operating or controlling (command), preferably an operating or controlling method, of a system subjected to waves (such as a floating or non-floating platform, a ship, a wave energy system for example), in which the following steps are implemented: a) The elevation of the free surface of an area of ​​a body of water is determined by means of the free surface elevation determination method according to any one of the variants / embodiments or combinations of variants / embodiments described above; and b) The system subjected to waves is controlled as a function of the determined free surface elevation.

[0131] Thus, the method according to the invention allows monitoring, operation or control of a system subjected to waves, to increase its performance, and / or limit the fatigue of the system subjected to waves and / or to improve the feasibility and safety of a large number of operations at sea.

[0132] According to one embodiment of the invention, the monitoring, operating or controlling method may comprise a step of predicting (in advance of phase) the elevation of the free surface of the water area by means of the elevation of the determined free surface, and the control is carried out as a function of the prediction of the elevation of the free surface so as to anticipate the control at the future elevation of the predicted free surface, before its arrival on the system to be monitored, operated or controlled.

[0133] The system subjected to waves may in particular be chosen from a ship, an aircraft carrier, an energy production platform (for example an oil platform), a renewable energy production system (for example an offshore wind turbine, a wave energy system), or a device for transferring personnel or equipment at sea (gangway, crane), or any similar system.

[0134] The monitoring, exploitation or control stage may notably consist of:

[0135] - the navigation of a vessel in an area where the free surface elevation is minimal, so the vessel can avoid severe sea conditions such as large waves

[0136] - the stabilization of a floating platform or a vessel according to the free surface elevation,

[0137] - control (e.g. predictive control) of a renewable energy production system to maximize the energy produced or to reduce the fatigue of the production system as a function of the free surface elevation.

[0138] This may involve controlling an electrical machine equipping a wave energy system to optimize the power produced according to the height of the wave,

[0139] It can also involve controlling the orientation angle of the blades and / or the nacelle of a wind turbine depending on the waves and swell.

[0140] The monitoring, exploitation or control stage may also consist of carrying out an operation at sea when the free surface elevation is minimal, for example:

[0141] - a rescue of a shipwrecked person from a ship,

[0142] - recovery of an object from the surface of the water from a ship,

[0143] - installation of a marine or underwater object from a ship or platform (for example, an oil pipeline),

[0144] - a takeoff or landing of an aircraft on an aircraft carrier, etc.

[0145] - a transfer of personnel from a ship or platform (for example, maintenance of an offshore wind turbine)

[0146] The characteristics and advantages of the method according to the invention will appear more clearly on reading the application examples below.

[0147] Consider a radar image R and the free surface elevation image W of a corresponding wave field. These two images, composed of pixels, have the same width and height.

[0148] Figure 2 shows the image W generated with a wave field simulation code, for a given sea state, characterized by a mean period and a significant wave height and a directional spectrum, and represents a "snapshot" of the sea surface on a circle of 2000 m radius centered around the radar, extracted from the generated elevation time series which is at least as long as one rotation of the radar (2 seconds here). In the image, which is 1024 x 1024 pixels, the horizontal axis represents the azimuth , which ranges from 0 to 360°, and the vertical axis represents the distance to the radar r, which ranges from 48 to 2000 meters. Each pixel in the image represents the free surface elevation, normalized to have the same order of magnitude as the intensity in the radar image. On the abscissa, the figure represents the angle 0 in degrees, on the left the ordinate the range in meters and on the right the standardized free surface elevation.

[0149] The R image, shown in Figure 3, is obtained from radar returns generated by a radar simulation code using the time series generated by the wave field simulation code, parameterized according to different environmental conditions (presence or absence of wind, rain, fog, etc.). Each image is generated with the same spatial discretizations Ar and A< > as the wave field image (same units represented on the abscissa and ordinate as in Figure 2, with the intensity of the radar return on the right).

[0150] If we analyze the R image, we see that the characteristics of the radar return are quite different along the direction of wave propagation (which corresponds to azimuths of 90° - 100° and 270° - 280°, in this example) compared, in particular, to the perpendicular direction. They are also different depending on the distance from the radar, between the lower and upper part of the image.

[0151] These differences can also be seen in the sections at different azimuths of the radar image shown in Figures 4a (section at an azimuth of 10°) and 4b (section at an azimuth of 90°): we see differences between what we can call the close range, up to 1000 m, and the far range beyond 1000 m, but also between the two sections, especially in the far range.

[0152] A learning base is created from image pairs (R, W) corresponding to the same situation. It is necessary to provide at least three image pairs to constitute this learning base. But it is preferable that the base contains more pairs to obtain a model capable of reconstructing the free surface elevation of the swell in several scenarios.

[0153] During the learning phase, a deep neural network of the "transformer" type or of the convolutional f type can be used, for example a U-Net encoder-decoder network which is the non-limiting example chosen for the examples, preferably with a set of weights to be optimized 9 (As is known, each neural network is associated with a set of parameters (weights) to be optimized).

[0154] During this phase, a radar image R is given as input to the convolutional neural network f and the latter calculates a result image W. This output is compared to the expected wave field image l / l / by a loss function L.

[0155] This loss function is for example a measure of structural similarity index (see the publication Wang et al. 2004: Zhou Wang, AC Bovik, HR Sheikh, and EP Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.)

[0156] The result of L is used to update the set of weights 9 of the network G by a stochastic gradient backpropagation method and thus obtain optimal values ​​of weights 9 .

[0157] For example, a possible optimization method is described by Adam in the publication Diederik & Ba, 2015 (Diederik, PK, & Ba, J. (2015). Adam: A method for stochastic optimization. CoRR, abs / 141 .6980), with 50 epochs: the set of image pairs in the training base is seen 50 times by the network, with a learning rate of 3.10 4 .

[0158] Formally: with n mean, o variance, = O.Olet c2= 0.03.

[0159] This process of free surface elevation reconstruction by a convolutional neural network is schematically represented in Figure 6, which is a synopsis of the process of free surface elevation reconstruction by a deep neural network RN like an ll-Net network: we see the image W processed by the radar model RM, to obtain the radar return R, then the use of a neural network like U-Net to obtain the result image W.

[0160] We see that using multiple standard convolutional neural networks, training them on different areas of the image pair, rather than just one, yields much better results. In this example, we consider four convolutional neural networks specialized according to azimuth range and range (distance):

[0161] - Network No. 1 RN1 (figure 7) learns over the close range, up to 1000 m, and over the in-wave azimuth range, i.e. two 90° intervals centered around the main wave direction (-95°) and its 360° complement (-275°).

[0162] - Network No. 2 RN2 learns over the close range, up to 1000 m, and over the off-wave azimuth range, i.e. two 90° intervals centered around directions perpendicular to the main wave direction (~5° and -185°).

[0163] - Network No. 3 RN3 learns over the far range, beyond 1000 m, and over the in-wave azimuth range.

[0164] - Network No. 4 RN4 learns on the far range, and on the off-wave azimuth range.

[0165] These four neural networks RN1 to RN4 are represented in Figure 7: they are thus specialized in the image which is divided into four zones, both according to the azimuth range and according to the range (distance).

[0166] This choice of four neural networks is of course an example, the invention being able to apply to a lower or higher number of neural networks, depending on the number of zones that one wants to distinguish in the image.

[0167] Figures 8a and 8b represent reconstruction results (with the same units on the abscissa and ordinate as for Figures 2 and 3):

[0168] - with a single neural network (standard U-Net): figure 8a

[0169] - and with 4 neural networks (U-Net) specialized per area of ​​the image: figure 8b

[0170] The metric used is the local correlation between V and V in both cases: values ​​close to 1 indicate a strong similarity between the images, zone by zone. We clearly obtain a better reconstruction with specialized U-Nets, more particularly in the “off-wave” ranges in azimuth.

[0171] Example 2

[0172] In this example, the standard convolutional neural network (U-Net type) is modified by guiding it from a distance map informing about the distance to the radar position. This information, which is not initially present in the data, can be constructed from a distance map, that is to say an image of the same size as the radar image, indicating at each point the distance to the radar. Examples of these guides, in polar representation and Cartesian representation are given in figure 11, which represents radar images and guides, with at the top, the initial radar image (in polar representation) on the left, with the corresponding guide on the right (distance function, intensity from black to white indicating low to high distances respectively), and at the bottom the radar image in Cartesian representation on the left, and the corresponding guide on the right.

[0173] The information corresponding to the orientation can be constructed in the same way. Training and prediction from the network is carried out using this guide(s).

[0174] In practice:

[0175] - A guide G is constructed from the problem to be solved. Here, it is a question of constructing an image of the same size as the radar image indicating the distance to the radar at each point. Another guide can also be an image giving an angular indication

[0176] - The input data is modified into a pair of images, radar image and distance map, and given to the convolutional neural network. It would also be possible to use a trio or more guide images.

[0177] Formally: f G ,eW = fe <M

[0178] Applied to wave height estimation, on a training base of only 5 image pairs, the following improvements were obtained, both in initial data representation (polar representation) and in Cartesian representation. Table 1 below is a comparison of estimation by standard or guided U-net network, in polar or Cartesian representation. The closer the SSIM score is to 1, the greater the similarity

[0179] To quantify the results, the network estimates are compared to the expected results by a structural similarity index measure [Wang et al. 2004],

[0180] [Table 1] From these data, we can see that with a U-Net type network, we already achieve good similarity, and that with a guided U-Net network, the results are even better (SSIM scores even closer to 1). We recall that SSIM is an acronym for the Anglo-Saxon expression "Structural Similarity Index Measure", which can be translated as structural similarity index measure.

[0181] In conclusion, the invention is therefore very effective, and offers different levels of performance depending on the chosen embodiment.

[0182] It is emphasized that if the examples use, for illustration purposes, U-Net neural networks, the invention applies analogously with other types of deep neural networks, in particular those called "transformers", (in a "transformer" architecture adapted to vision), in particular such as those described in the publication: Tianyang Lin, Yuxin Wang, Xiangyang Liu, Xipeng Qiu, A survey of transformers, Al Open, Volume 3, 2022, Pages 111-132, ISSN 2666-6510. (https: / / doi.Org / 10.1016 / j.aiopen.2022.10.001).

[0183] It also applies analogously with at least one Fourier neural operator, notably as described in the publication Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2020). “Fourier neural operator for parametric partial differentia”l equations. arXiv preprint arXiv:2010.08895.

[0184] It is also emphasized that the method according to the invention comprises different steps, which can be implemented with computer means, in particular a computer, a server or a calculator. The steps in question can be carried out online (computer means connected by computer communication means, internet / intranet for example) or offline: thus, in particular, the acquisition of real images can be done online or offline (by being pre-recorded and made accessible).

Claims

Claims 1. Method for determining an elevation of the free surface ESL of an area of a body of water, characterized in that, according to said method, 1) - at least one learning database (E) is constituted of a plurality of pairs (6) of images (W,R), each pair consisting of a first image (W) of elevation of the free surface ESL and a second radar return image (R), the two images of the same pair corresponding to the same area of the surface of the body of water and to an identical or close instant in time, said images (W) and (R) being real or simulated 2) - spatial characteristics of said area of the surface of the body of water corresponding to this duo are added to each pair of images (W,R), said spatial characteristics (9) comprising at least the position relative to the radar and to the main direction(s) of waves, said spatial characteristics being directly known during the acquisition of the images, or being deduced from the analysis of the images themselves, or even provided by an external source, 3) - at least one deep neural network, in particular of the convolutional type, or of the “transformer” type or a Fourier neural operator, is trained on at least said training database (E,E'>, using said spatial characteristics (9) to specialize said deep neural network(s), and to obtain a model (H) for reconstructing the elevation of the free surface ESL in the following manner: - we divide each pair of images (W,R) into n distinct zones, according to spatial characteristics, n being greater than or equal to 2, and in particular being equal to 4 - we use n deep neural networks (RN1, RN2, RN3, RN4), each deep neural network being specialized on a dedicated area of duo (6) of images (W, R), - we train each neural network on its dedicated image duo area (W,R) 4) - real return images (8) are acquired by radar 5) - said reconstruction model (H, H') is applied to said real radar return images (8) acquired to obtain the free surface elevation ESL (I).

2. Method according to the preceding claim, characterized in that the first image (W) of each duo (6) of images (W,R) is obtained by discretization of three-dimensional time series (2) of free surface elevation ESL generated by a digital simulation code (A) of wave fields, and the second image (R) of each duo of images (W,R) by discretization of radar returns generated by a digital simulation code (B) of radar.

3. Method according to one of the preceding claims, characterized in that the first images (W) of each duo (6) of images (W,R) come from real measurements of free surface elevation ESL, in particular carried out with at least one remote sensor of the LiDAR type or a stereoscopic camera system or a set of measurement buoys, on at least part of the surface covered by the radar, and in that the second radar return images (R) come from real measurements carried out by at least one radar, in particular an X-band or S-band maritime radar.

4. Method according to one of the preceding claims, characterized in that the learning database (E, E') comprises a first base of duos of surface elevation images (W, R) and radar return images (R), all of the images of which are simulated, and a second base of duos of surface elevation images (W, R) and radar return images, all of the images of which are real, the relative weight of said two bases possibly being able to be adjusted.

5. Method according to one of the preceding claims, characterized in that said spatial characteristics (9) comprise a distance to an origin, said origin corresponding in particular to the position of the radar, and an angular orientation, in particular an azimuth.

6. Method according to one of the preceding claims, characterized in that the or at least one of the deep neural networks (RN1, RN2, RN3, RN4) is an encoder-decoder network, for example of U-Net architecture.

7. Method according to one of the preceding claims, characterized in that said pairs (6) of images (W,R) are divided into n zones as a function of the azimuth and the distance to the radar.

8. Method according to one of the preceding claims, characterized in that, to obtain said reconstruction model (H, H'), at least one first guide image Rg1 is formed for each radar return image (R) of the learning base (E, E'> by means of a radar distance map, and the deep neural network is trained by means of said surface elevation image (W) and said at least first associated guide image Rg1.

9. Method according to the preceding claim, characterized in that, to obtain said reconstruction model (H, H'), at least one second guide image Rg2 is formed by means of an angular orientation map, and the deep neural network is trained also by means of said associated second guide image Rg2.

10. Method for determining an elevation of the free surface ESL of an area of a body of water according to one of the preceding claims, characterized in that the following steps are implemented: A1) a first database of images (W) obtained from three-dimensional time series (2) of free surface elevation, in particular the elevation (z), the position in Cartesian coordinates (x,y) or polar coordinates (r, <|>), and the time (t), generated by a wave field simulation code (A) corresponding to different sea states, A2) a second database of so-called return images (R) is constructed obtained from radar returns generated by a radar simulation code (B) using said time series generated by the wave field simulation code, parameterized according to environmental conditions (5), in particular the speed and direction of the wind and the presence of precipitation A3) a learning base (E,E') is constructed comprising a first set of images from the first database (W) constructed in step A1) and a first set of corresponding return images from the second database (R) constructed in step A2) by integrating the environmental conditions used in step A2), so as to obtain image pairs (W,R) each consisting of a first surface elevation image (W) and a second radar return image (R), the two images of the same pair corresponding to the same area of the surface of the body of water and to an identical or close instant in time, A4) to each pair of images (W,R) of the learning database (E,E') are added the spatial characteristics (9) of the area of the surface of the body of water corresponding to this pair, said spatial characteristics comprising a distance to an origin, said origin corresponding in particular to the position of the radar, and an orientation, in particular in azimuth, relative to the main direction(s) of the waves A5) a free surface elevation reconstruction model (H, H') is constructed from the training database (E, E') obtained in step A3) and the spatial characteristics specified in step A4), by training at least one deep neural network of convolutional or "transformer" type, in particular of U-Net type, or Fourier neural operator, said network(s) being specialized with the spatial characteristics obtained in step A4) A6) real images (F) are acquired using at least one radar, in particular maritime, to which their spatial characteristics defined in step A4 are added) A7) the free surface elevation ESL is determined by applying the free surface elevation reconstruction model (H, H') obtained in step A5) at least to the real images (8) acquired with the radar in step A6) with their spatial characteristics.

11. Method according to the preceding claim, characterized in that, in step A2) of constructing the second return image database, the radar simulation code is parameterized according to environmental conditions (5) chosen from at least: the presence or absence of wind, the wind speed, the wind direction, the presence or absence of precipitation such as rain, the intensity of precipitation, the presence or absence of fog, the temperature and the state of the sea.

12. Method according to one of claims 10 or 11, characterized in that, in step A5) of construction of the free surface elevation reconstruction model (H, H'), said model is validated with a validation database composed of a second set of images from the first database constructed in step A1) and a second set of corresponding return images from the second database constructed in step A2), said sets being different from those used for learning.

13. Method according to one of claims 10 to 12, characterized in that, in step A6) of acquisition (F) of the real images (8) using at least one maritime radar, said images are recorded, and in that, in step A7) of determination of the free surface elevation ESL, a communication means, in particular a computer means, is used to access said recorded radar images.

14. Method according to one of claims 10 to 13, characterized in that the following additional steps are also implemented, which are carried out between step A5 and step A6): B0) measurements of the elevation of the free surface ESL are carried out on at least part of the area covered by the radar, in particular maritime, simulated in step A1), in particular by means of a stereoscopic camera system, a LIDAR or a set of measurement buoys B1) we build a database of images obtained by digitizing the actual wave measurements obtained in step B0), B2) a database of images obtained from the real return signals obtained with a maritime radar is constructed, synchronized in time with the image database constructed in step B1), B3) a real learning database is constructed from a set of ESL measurement images from the database obtained in step B1) and a set of real radar return images from the database obtained in step B2), in the form of image pairs B4) we add to each pair of images from step B3 associating real measurements of free surface elevation ESL, and real radar returns the spatial characteristics of the corresponding area of the water surface, in particular in terms of distance and direction relative to the main wave direction(s) B5) The free surface elevation reconstruction model constructed in step A5) is made more robust from the real learning database obtained in step B3) and the spatial characteristics determined in step B4), using machine learning or deep learning methods, on areas that match real measurements and real radar returns. B6) real images (8) are acquired (F) using at least one radar, in particular maritime B7) Step A7) is replaced by step B7) by applying the improved free surface elevation reconstruction model (H') obtained in step B5) at least to the real images (8) acquired with the radar in step B6).

15. Method for monitoring, operating, or controlling a system subject to waves within a body of water, in which the following steps are implemented: - the elevation of the free surface of said body of water is determined by means of the method for determining the free surface elevation according to one of the preceding claims; and - the said system subject to waves is supervised, operated or controlled according to the elevation of the free surface of the said body of water thus determined.

16. Computer, server or calculator configured to implement the method according to one of claims 1 to 15.

17. Computer program product downloadable from a communication network and / or recorded on a medium readable by a computer, a server or a calculator and / or executable by a processor, comprising program code instructions for implementing the method according to one of claims 1 to 15, when said program is executed on a computer, a server or a calculator.

18. Storage medium readable by a computer, server or calculator and storing instructions, which, when executed by a computer, server or calculator, imply that the computer, server or calculator implements the method according to one of claims 1 to 15.

Images