Wave radar image inversion method based on convolutional neural network, and application thereof

Abstract

A wave radar image inversion method based on a convolutional neural network, and the application thereof, which relate to the field of sea condition monitoring. A convolutional neural network autoencoder is used, and a training set generated by means of a simulation tool is input into the convolutional neural network autoencoder for supervised training, wherein an X-band radar image of an inversion region is an input of the training set, and a corresponding sea surface wave height map is an output of the training set, thereby obtaining a mapping relationship model therebetween. The convolutional neural network autoencoder comprises an encoder, fully connected layers and a decoder, wherein the encoder comprises five convolutional layers and five maximum pooling layers, the convolutional layers and the maximum pooling layers being alternately connected, the fully connected layers are two symmetric layers, and the decoder comprises five deconvolutional layers. Using a convolutional neural network deep learning algorithm achieves a better effect in restoring nonlinear components. Using an autoencoder technique can directly restore an X-band radar wave image to a real-time sea surface wave height map with a relatively small error, and can more comprehensively display sea condition information.

Description

Wave radar image inversion method based on convolutional neural network and application thereof TECHNICAL FIELD

[0001] The present application relates to the field of sea state monitoring, in particular to a wave radar image inversion method based on convolutional neural network and application thereof. BACKGROUND

[0002] In order to develop marine resources more effectively and safely, the demand for real-time and clear understanding of sea state changes is more prominent. Wave measurement is one of the ways, and its results can be applied to many aspects such as coastal protection, port management, navigation safety, coastal resource management, and marine rescue tasks. At present, the most important means to complete wave measurement is wave buoy and X-band radar. Wave buoy is not suitable for deep water environment and cannot measure in real time with the ship, while X-band radar has the advantages of small volume, low cost, small blind area, strong real-time performance, sustainable monitoring of large area sea changes, and convenient movement, etc., and is a sea state detection technology with broad application prospect. The image measured by X-band radar can provide real-time sea state prediction for ship navigation after processing. However, how to derive real-time sea state parameters from the image measured by X-band radar is currently a difficult point.

[0003] At present, the method for deriving sea state parameters from X-band radar image mainly combines traditional mathematics and physical phenomena. First, N continuous X-band radar image sequences are selected, an inversion region is selected on the X-band radar image, and the time sequence I(x, y, z) of the X-band radar image is obtained by digitizing the inversion region. Then, the three-dimensional discrete Fourier transform is applied to the time sequence I(x, y, z) of the X-band radar image to obtain a three-dimensional image spectrum. Then, the three-dimensional image spectrum is filtered using the dispersion relationship of waves, and integrated in the positive frequency range to obtain a two-dimensional image spectrum. On the basis of the two-dimensional image spectrum, multiplying the empirical modulation transfer function can obtain a two-dimensional sea wave spectrum. Then, other related sea state parameters can be obtained by using the two-dimensional sea wave spectrum.

[0004] However, there are factors such as shadow modulation when X-band radar scans the sea waves. The cause of shadow modulation is shown in FIG. 2. Since the electromagnetic wave emitted by the X-band radar is almost grazing to the sea surface, the angle with the sea surface is very small, and the sea waves have ups and downs, some waves with relatively high height will block the propagation of electromagnetic waves, forming a region where electromagnetic waves cannot be incident, causing no echo or weak echo in the shadowed area. Shadow modulation is a nonlinear imaging modulation, which is very important in marine radar imaging. The traditional X-band radar image derivation method for sea state parameters uses an empirical modulation transfer function to modulate the nonlinear component. This kind of empirical formula has poor restoration effect on the nonlinear component of waves under certain special sea conditions, and has limitations.

[0005] Therefore, the person skilled in the art is committed to developing an inversion method based on X-band radar images. SUMMARY

[0006] In view of the above defects of the prior art, the technical problem to be solved by the present application is how to develop an inversion method based on X-band radar images, which has better restoration effect on the nonlinear components of shadow modulation and other waves.

[0007] To achieve the above-mentioned purpose, the present application is a wave radar image inversion method based on convolutional neural network, which comprises:

[0008] A training set is generated by a simulation tool, the training set being an X-band radar image of an inversion area and a corresponding sea surface wave height map,

[0009] The input data of the training set and the output data of the training set are determined, the X-band radar image of the inversion area being the input of the training set, and the corresponding sea surface wave height map being the output of the training set,

[0010] The training set is input into a convolutional neural network autoencoder for supervised training to obtain a mapping relationship model of the X-band radar image of the inversion area and the corresponding sea surface wave height map;

[0011] The convolutional neural network autoencoder comprises an encoder, a fully connected layer and a decoder:

[0012] The encoder comprises five convolutional layers and five max-pooling layers, each convolutional layer being connected to each max-pooling layer alternately,

[0013] The fully connected layer comprises two symmetrical layers,

[0014] The decoder comprises five deconvolutional layers.

[0015] Further, the simulation tool is a Matlab radar tool kit.

[0016] Further, the distance between the nearest point of the X-band radar image of the inversion area to the X-band radar center and the X-band radar center is greater than 500 meters.

[0017] Further, the distance between the nearest point of the X-band radar image of the inversion area to the X-band radar center and the X-band radar center is greater than 600 meters.

[0018] Further, the resolution of the X-band radar image of the inversion area is 8m.

[0019] Further, the X-band radar image of the inversion area is rectangular.

[0020] Further, the side length of the rectangle is greater than 1200m.

[0021] Further, the distance between the closest point of the rectangle to the X-band radar center and the X-band radar center is greater than 700m.

[0022] Further, the length of the rectangle is 3000m.

[0023] Further, the width of the rectangle is 1500m.

[0024] Further, the method further comprises the step of: resampling the X-band radar image and the corresponding sea surface wave height map of the inversion area into a 512x256 pixel image.

[0025] Further, the method further comprises the step of: converting the X-band radar image and the corresponding sea surface wave height map of the inversion area into a grayscale image.

[0026] Further, the method further comprises the step of: normalizing the grayscale value of the grayscale image from 0-255 to 0-1.

[0027] Further, the method further comprises the step of: adding Gaussian white noise randomly on the X-band radar image of the inversion area to obtain a noisy X-band radar image, taking the noisy X-band radar image as the input of the training set, and obtaining the mapping relationship model of the noisy X-band radar image and the corresponding sea surface wave height map.

[0028] Further, the variance of the Gaussian white noise is 0-0.2.

[0029] Further, the method further comprises the step of: randomly occluding the X-band radar image of the inversion area to obtain an occluded X-band radar image, taking the occluded X-band radar image as the input of the training set, and obtaining the mapping relationship model of the occluded X-band radar image and the corresponding sea surface wave height map.

[0030] The application also provides an application of a wave radar image inversion method based on a convolutional neural network on a shipborne radar system.

[0031] Further, the shipborne radar system comprises an X-band radar and a radar image processing computer, and the wave radar image inversion method is executed by the radar image processing computer.

[0032] Further, the shipborne radar system further comprises a radar image display.

[0033] Further, the X-band radar scans the real-time sea surface to obtain a corresponding real-time X-band radar image, the real-time X-band radar image is displayed on the radar image display and transmitted to the radar image processing computer, the radar image processing computer uses the wave radar image inversion method to perform inversion processing on the real-time X-band radar image to obtain a real-time sea surface wave height map corresponding to the real-time X-band radar image.

[0034] The technical effects of the present application are as follows:

[0035] 1) The convolutional neural network, a deep learning algorithm, has good restoration effect on nonlinear components, and does not need to consider the mathematical and physical principles between the input and output when finding the relationship between the input and output. Based on a large amount of data, the nonlinear relationship can be obtained through training, which is simple and convenient to operate.

[0036] 2) The autoencoder technology can quickly reconstruct a three-dimensional sea surface wave height map from the X-band radar image input into the model, instead of only two-dimensional wave height, period and other sea state parameters, and has a small error. Compared with the wave height, period and other dimensional parameters, the three-dimensional sea surface wave height map can more comprehensively display the sea state information.

[0037] 3) In addition, the convolutional neural network has good applicability in extracting the overall features of the image, and can restore the overall features of the image in the case of local data loss or large noise interference, so it can solve the problem of large error in extracting sea state information from X-band radar images due to noise interference or structural obstruction in actual measurement.

[0038] 4) The simulation tool kit is used for simulation to obtain a large number of X-band radar images and corresponding sea surface wave height maps as inputs and outputs of the convolutional neural network autoencoder training, solving the problem of insufficient number of measured data during training.

[0039] The concept, specific structure and technical effects of the present application will be further described below with reference to the accompanying drawings, so as to fully understand the purpose, features and effects of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0040] Fig. 1 is an inversion method flowchart of an embodiment of the present application;

[0041] Fig. 2 is a schematic diagram of shadow modulation causes;

[0042] Fig. 3 is a schematic diagram of a shipborne radar system hardware of an embodiment of the present application;

[0043] Fig. 4 is a framework diagram of key parts of a convolutional neural network;

[0044] Fig. 5 is a framework diagram of a convolutional neural network autoencoder of an embodiment of the present application;

[0045] Fig. 6 is a selected rectangular inversion region of one embodiment of the present application;

[0046] Fig. 7 is a sea surface wave height map corresponding to the X-band radar image of the inversion region of one embodiment of the present application;

[0047] Fig. 8 is a schematic diagram of randomly adding Gaussian white noise with variance of 0-0.2 on the selected inversion region of the X-band radar image of the training set;

[0048] Fig. 9 is a schematic diagram of randomly covering on the selected inversion region of the X-band radar image of the training set. DETAILED DESCRIPTION

[0049] The present application can be embodied in many different forms and is not limited to the embodiments set forth herein. The present application will be more clearly understood and appreciated from the following detailed description taken in conjunction with the accompanying drawings.

[0050] The size and thickness of each component shown in the drawings are arbitrarily shown, and the present application is not limited to the size and thickness of each component. In order to make the drawing clearer, the thickness of the components is appropriately exaggerated in some places in the drawing.

[0051] As shown in Fig. 1, the wave radar image inversion method based on the convolutional neural network of the present application first generates the X-band radar image and its corresponding sea surface wave height map as the training set through the simulation tool; then determines the X-band radar image of the inversion region as the input of the training set, and the corresponding sea surface wave height map as the output of the training set; then adopts the convolutional neural network autoencoder, inputs the training set into the convolutional neural network autoencoder for supervised training, and finally obtains the mapping relationship model of the X-band radar image of the inversion region and the corresponding sea surface wave height map, that is, obtains the wave radar image inversion method;

[0052] Before training, the appropriate inversion region and its corresponding sea surface wave height map on the known X-band radar image are selected as the training set, the X-band radar image of the inversion region is the input of the training set, and the corresponding sea surface wave height map is the output of the training set;

[0053] The framework of the convolutional neural network autoencoder selected by the present application includes an encoder, a fully connected layer and a decoder, wherein the encoder includes five convolutional layers and five maximum pooling layers, each convolutional layer is connected with each maximum pooling layer alternately, the fully connected layer is symmetrical two layers, and the decoder includes five deconvolutional layers.

[0054] In some embodiments, the known X-band radar images and corresponding sea surface wave height maps can be generated in large quantities using the simulation tool Matlab Radar Toolbox, thereby solving the problem of not having enough number of X-band radar images and corresponding sea surface wave height maps for training in practice.

[0055] In some embodiments, the selection of suitable inversion region is as follows:

[0056] Considering the image distortion caused by the X-band radar sidelobe effect on the near field, in order to reduce the error caused by image distortion, X-band radar images within 500 meters from the center of the X-band radar are not used as the inversion region (such as the X-band radar images within the circle with a radius of 500 m shown in FIG. 6). In order to reduce the influence of distorted radar images to a greater extent, X-band radar images outside 600 m from the center of the X-band radar can be selected as the inversion region. Because when the scanning region is too close to the center of the X-band radar, the sidelobe echo intensity of the X-band radar can be comparable to the target echo intensity, thereby affecting the X-band radar image obtained by scanning.

[0057] X-band radar images with a resolution of 8 m are selected, so that there are four points to characterize a wave, reducing the error. Because for common waves with a period of 5 s to 14 s, the wave length in deep water conditions is about 37.5 m to 306 m; in the nearshore area, the wave length is about 35 m to 102 m. For X-band radar images with a wave length of about 35 m or 37.5 m, a resolution of 8 m can ensure four sampling points.

[0058] The inversion region can be a rectangle, and the side length of the rectangle is at least greater than 1200 m, so as to satisfy that there are at least 3.7 wave lengths in the rectangle.

[0059] Preferably, a rectangle with a length of 3000 m and a width of 1500 m, which is greater than 700 m from the center of the X-band radar, is selected as the X-band radar image of the inversion region, as shown in FIG. 6. And the sea surface wave height map corresponding to the X-band radar image of the inversion region is selected, as shown in FIG. 7.

[0060] Preferably, the selected X-band radar image and the corresponding sea surface wave height map are both resampled to 512x256 pixel images, then converted to grayscale images, and the grayscale is normalized from 0-255 to 0-1, to obtain X-band radar images and corresponding sea surface wave height maps with pixel size of 512x256 and grayscale value between 0-1. Through normalization processing, the data is converted to a unified scale, which can improve the performance and accuracy of the machine learning algorithm.

[0061] Then the selected inversion region is input into the convolutional neural network autoencoder as a training set for training. Specifically, the X-band radar image of the inversion region is input, and the corresponding sea surface wave height image is output for supervised training to find the mapping relationship between the two, thereby obtaining the wave radar image inversion method.

[0062] In some embodiments, the framework of the convolutional neural network autoencoder is shown in FIG. 5, including an encoder, a fully connected layer, and a decoder. The encoder includes five convolutional layers and five max-pooling layers, each convolutional layer being connected to each max-pooling layer alternately, i.e., each convolutional layer is followed by a max-pooling layer, corresponding to (1, 2, 3, 4, 5) in FIG. 5. The fully connected layer is symmetrical and has two layers. The decoder includes five deconvolutional layers, corresponding to (6, 7, 8, 9, 10) in FIG. 5.

[0063] The convolutional neural network is a multi-layer artificial neural network designed to process two-dimensional input data. As shown in FIG. 4, the key part includes a convolutional layer, a pooling layer, and a final fully connected layer. The convolutional process is performed first, followed by the pooling process, wherein the output of the convolutional process is used as the input of the pooling layer, and the result of the pooling layer is used as the input of the next convolutional layer. Finally, the fully connected layer is connected. The convolutional neural network has fewer network connections and weight parameters, which can effectively reduce the learning complexity of the network model and is easier to train.

[0064] The autoencoder is a type of artificial neural network that includes an encoder and a decoder. Given an input space X∈M and a hidden layer feature space h∈F, the autoencoder solves the mapping f, g between the two to minimize the reconstruction error of the input feature: f:M→F g:F→M

[0065] where X is the input data, M is the input data space, h is the hidden layer feature, F is the hidden layer feature space, f is the encoder function mapping, g is the decoder function mapping, and g[f(X)] is the reconstructed data. After solving, the hidden layer feature h output by the encoder can be regarded as a representation of the input data X. Its function is to learn the representation of the input information by taking the input information as the learning target, extract the most important part of the input image to obtain a low-dimensional representation of high-dimensional data, and its accuracy and performance in many applications exceed those of traditional engineering techniques. It can be used for anomaly detection, text generation, image generation, image denoising, etc.

[0066] The present application combines the advantages of the convolutional neural network in image information recognition and extraction with the advantages of the autoencoder in image generation and reconstruction. Through appropriate design of the framework of the convolutional neural network autoencoder, the obtained wave radar image inversion method can realize the reconstruction and reproduction of the actual sea surface wave height image from the X-band radar image with a small error.

[0067] In some embodiments, as shown in FIG. 3, the wave radar image inversion method is combined into the shipborne radar system hardware, which mainly includes the X-band radar loaded on the ship, the radar image display, the radar image processing computer, and the wave radar image inversion method is executed on the radar image processing computer. The real-time X-band radar image obtained by the X-band radar scanning the real-time sea surface is displayed on the radar image display and transmitted to the radar image processing computer, and the wave radar image inversion method on the radar image processing computer is used for inversion processing, and finally the real-time sea surface wave height map corresponding to the X-band radar image is output. Thus, the X-band radar image obtained by the X-band radar scanning can be converted into the actual three-dimensional sea surface wave height map in real time and accurately, and the real-time feedback of the sea state during the ship navigation is provided, and reference and help for the crew to understand the ship navigation state and route decision are provided. Further, according to the real-time sea surface wave height map, the wave height curve of a specific position on the sea surface can be directly obtained. According to the time sequence, the radar image is sequentially converted into the sea surface wave height map, and according to the sea surface wave height map, the wave height time sequence of a specific position on the sea surface can also be directly obtained.

[0068] In some embodiments, for the X-band radar image inversion method containing noise interference, Gaussian white noise with a variance of 0-0.2 is randomly added to the selected inversion region of the X-band radar image in the training set, as shown in FIG. 8, to obtain the X-band radar image with noise interference as the input of the training set, and the output is still the corresponding sea surface wave height map. The mapping relationship model of the X-band radar image with noise interference and the corresponding sea surface wave height map is obtained through supervised training, that is, the method capable of inverting the X-band radar image with noise interference is obtained, so as to remove the influence of noise interference on the X-band radar image.

[0069] In some embodiments, for the X-band radar image inversion method with occlusion, the selected inversion region of the X-band radar image in the training set is randomly covered, as shown in FIG. 9, as the input of the training set, and the output is still the corresponding sea surface wave height map. The mapping relationship model of the X-band radar image with occlusion and the corresponding sea surface wave height map is obtained through supervised training, that is, the method capable of inverting the X-band radar image with occlusion is obtained.

[0070] The wave radar image inversion method based on the convolutional neural network of the present application can not only invert the X-band radar image to obtain the actual three-dimensional sea surface wave height image, but also can be applied to the X-band radar image with a large amount of noise interference and the X-band radar image with signal loss in some areas caused by the obstruction of the structure on the sea surface to the X-band radar signal propagation. Considering the effectiveness of the convolutional neural network in extracting the overall features of the image, the convolutional neural network auto-encoder model is used to take the X-band radar image with noise and obstruction as the input and the actual sea surface wave height image as the output for supervised learning to obtain the trained model, which can realize the sea surface wave height image inversion of the X-band radar image with noise interference and the X-band radar image with obstruction.

[0071] The foregoing detailed description of the preferred embodiments of the application has been presented. It is understood that modifications and changes can be made by those skilled in the art without departing from the spirit of the application. The scope of the application is defined by the appended claims.

Claims

1. A convolutional neural network-based wave radar image inversion method, wherein, The method comprises: generating a training set by a simulation tool, the training set being X-band radar images of an inversion region and corresponding sea surface wave height maps thereof, determining input data of the training set and output data of the training set, the X-band radar images of the inversion region being the input of the training set, and the corresponding sea surface wave height maps being the output of the training set, inputting the training set into a convolutional neural network autoencoder for supervised training to obtain a mapping relationship model of the X-band radar images of the inversion region and the corresponding sea surface wave height maps thereof; the convolutional neural network autoencoder comprises an encoder, a fully connected layer and a decoder: wherein the encoder comprises five convolutional layers and five max-pooling layers, each convolutional layer being connected to each max-pooling layer alternately, the fully connected layer comprises two symmetrical layers, and the decoder comprises five deconvolutional layers.

2. The convolutional neural network-based wave radar image inversion method of claim 1, wherein, The simulation tool is a Matlab radar tool package.

3. The convolutional neural network-based ocean wave radar image inversion method of claim 1, wherein, The distance between the closest point of the X-band radar image of the inversion region to the X-band radar center and the X-band radar center is greater than 500 meters.

4. The convolutional neural network-based ocean wave radar image inversion method of claim 1, wherein, The distance between the closest point of the X-band radar image of the inversion region to the X-band radar center and the X-band radar center is greater than 600 meters.

5. The convolutional neural network-based ocean wave radar image inversion method of claim 1, wherein, The resolution of the X-band radar image of the inversion region is 8 m.

6. The convolutional neural network-based ocean wave radar image inversion method of claim 1, wherein, The X-band radar image of the inversion region is a rectangle.

7. The convolutional neural network-based ocean wave radar image inversion method of claim 6, wherein, The side length of the rectangle is greater than 1200 m.

8. The convolutional neural network-based ocean wave radar image inversion method of claim 6, wherein, The distance between the closest point of the rectangle to the X-band radar center and the X-band radar center is greater than 700 m.

9. The convolutional neural network-based ocean wave radar image inversion method of claim 8, wherein, The length of the rectangle is 3000 m.

10. The convolutional neural network-based ocean wave radar image inversion method of claim 8, wherein, The width of the rectangle is 1500 m.

11. The wave radar image inversion method based on a convolutional neural network according to claim 1, further comprising the step of: resampling the X-band radar image of the inversion region and the corresponding sea surface wave height map thereof into an image of 512x256 pixels.

12. The wave radar image inversion method based on a convolutional neural network according to claim 1, further comprising the step of: converting the X-band radar image of the inversion region and the corresponding sea surface wave height map thereof into a grayscale image.

13. The wave radar image inversion method based on a convolutional neural network according to claim 12, further comprising the step of: normalizing the grayscale values of the grayscale image from 0-255 to 0-1.

14. The wave radar image inversion method based on a convolutional neural network according to claim 1, further comprising the step of: adding Gaussian white noise to the X-band radar image of the inversion region randomly to obtain a noisy X-band radar image, using the noisy X-band radar image as the input of the training set, and obtaining a mapping relationship model of the noisy X-band radar image and the corresponding sea surface wave height map thereof.

15. The convolutional neural network-based ocean wave radar image inversion method of claim 14, wherein, The variance of the Gaussian white noise is 0-0.

2.

16. The convolutional neural network based wave radar image inversion method according to claim 1, further comprising the step of: randomly obscuring the X-band radar image of the inversion area to obtain an obscured X-band radar image, taking the obscured X-band radar image as the input of the training set to obtain the mapping relationship model between the obscured X-band radar image and the corresponding sea surface significant wave height map.

17. An application of the convolutional neural network based wave radar image inversion method according to claim 1 to a shipboard radar system.

18. Use of the convolutional neural network-based wave radar image inversion method of claim 17 on a shipboard radar system, wherein, The shipboard radar system comprises an X-band radar and a radar image processing computer, and the wave radar image inversion method is executed by the radar image processing computer.

19. Use of the convolutional neural network-based wave radar image inversion method of claim 17 on a shipboard radar system, wherein, The shipboard radar system further comprises a radar image display.

20. Use of the convolutional neural network-based wave radar image inversion method of claim 19 on a shipboard radar system, wherein, The X-band radar scans the real-time sea surface to obtain a corresponding real-time X-band radar image, which is displayed on the radar image display and transmitted to the radar image processing computer, and the radar image processing computer performs inversion processing on the real-time X-band radar image by using the wave radar image inversion method to obtain a real-time sea surface significant wave height map corresponding to the real-time X-band radar image.

Images