A method for estimating ship motion parameters based on wake matching

Abstract

The present invention relates to the technical field of ship motion in remote sensing images, and discloses a method for estimating ship motion parameters based on wake matching. A ship wake model including an infrared sea surface model, a ship wake height field, and a ship wake temperature field is constructed; using the above model, ship wake infrared images are generated by setting multi-speed, multi-heading, and multi-sea condition parameters, and a ship wake database is constructed; a deep learning convolutional neural network and a multi-task loss function are used to extract the wake to be estimated; a multi-stage image matching model is constructed based on a feature point extraction module, a feature vector description module, and a feature point matching module, and it is used to match the database and a batch of wakes to be estimated. The infrared sea surface model of this method hybridly calculates two sea surfaces, ensuring authenticity while reducing the repeatability of the sea surface; by simulating the wake temperature field and wake height field, the authenticity of the wake is improved, and the constructed wake data set can be reused to achieve efficient estimation of ship motion parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of ship motion in remote sensing images, and specifically relates to a method for estimating ship motion parameters based on wake matching. Background Art

[0002] When a ship sails on the sea surface, it will inevitably generate a wake. The wake is one of the important features of a moving target.

[0003] Currently, there are many problems in the field of ship wake motion parameter estimation at home and abroad. The traditional method of extracting the wake using infrared images and combining empirical formulas to estimate the speed and heading highly depends on the accuracy of wake extraction. In high sea states, the wake is easily interfered by waves, and when the wake encounters sea surface objects or land occlusion and absorption, it is difficult to obtain fine-structured wakes, resulting in significant errors in the method of estimating the ship's speed and heading using the ship wake extracted from infrared images and empirical formulas. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for estimating ship motion parameters based on wake matching, which solves the problem in the existing ship wake motion parameter estimation technology that due to factors such as sea conditions, it is difficult to extract the wake, and further causes serious deviations in estimating the ship's speed and heading using infrared images and empirical formulas.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A method for estimating ship motion parameters based on wake matching includes the following steps:

[0007] Construct a sea surface ship wake model including an infrared sea surface model, a ship wake height field, and a ship wake temperature field. Among them, the infrared sea surface model is calculated and rendered by combining a sine wave and a sea spectrum sea surface. The ship wake height field is constructed based on a thin ship model, integral domain transformation, and an attenuation model. The ship wake temperature field is constructed according to the sea surface temperature characteristics and the ship's thermal wake mechanism, and is adjusted according to the motion parameters;

[0008] Based on the constructed sea surface ship wake model, generate ship wake infrared images by setting multi-speed, multi-heading, and multi-sea condition parameters, and construct a ship wake database;

[0009] Use a deep learning convolutional neural network and a multi-task loss function to extract the wake to be estimated;

[0010] Construct a multi-stage image matching model based on a feature point extraction module, a feature vector description module, and a feature point matching module, and use it to match the database and a batch of wakes to be estimated, and obtain and select the optimal motion parameter estimation result;

[0011] Among them, the steps for constructing the multi-stage image matching model are as follows:

[0012] Based on the feature point extraction module, feature points of the template image and the target image are extracted, and at the same time, image patches are cropped around the feature points according to a certain specification and input into the feature vector description module to obtain the feature point description vectors;

[0013] The feature point matching module is used to find the template image description vector with the smallest Euclidean distance from the feature point description vector, and calculate whether the Euclidean distance between the two is less than the threshold to obtain the feature point matching result.

[0014] Preferably, the infrared sea surface model simulates the sea surface of a sine wave and the sea surface of a sea spectrum through a hybrid calculation of the sea surface. For the sea surface simulation, the sea spectrum model and the sine wave with superimposed noise are respectively used at the near and far sea surface grid points to display details and reduce repetitiveness.

[0015] Preferably, the ship wake database includes simulation data of multiple speeds, multiple headings, and multiple sea conditions.

[0016] Preferably, the deep learning convolutional neural network is Faster RCNN, which can reduce computational redundancy compared with the traditional RCNN network and significantly improve the speed.

[0017] Preferably, the feature extraction module extracts the feature points of ship wakes in different motion states. Specifically, Gaussian blurs are applied to the image using Gaussian kernels with different standard deviations, then the difference between the two Gaussian blurred images is calculated, and the local extreme points of the image are selected as the feature points.

[0018] Preferably, the feature vector description module uses a convolutional neural network with non-shared parameters.

[0019] Preferably, the feature point matching module adopts the structure of a k-d tree.

[0020] Advantages of the present invention: The present invention calculates two sea surfaces of a sine wave and a sea spectrum through a hybrid calculation of the sea surface simulation, ensuring authenticity while reducing the repetitiveness of the sea surface, which is more in line with the real situation. An image matching model is constructed by using the method of image feature matching, and image features with characterization ability are extracted for matching, which can reduce redundant steps during matching and reduce the influence of noise on the model at the same time;

[0021] The present invention estimates the ship wake through the method of wake template matching, breaking through the traditional inversion estimation method. After the database is made, the ship motion parameters can be estimated in batches, significantly improving the accuracy and efficiency of ship motion parameter estimation, especially the application effect under complex sea conditions.

[0022] Of course, it is not necessary for any product implementing the present invention to achieve all the above advantages simultaneously. Description of the Drawings

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0024] Figure 1 It is a flowchart of a method for estimating ship motion parameters based on wake matching according to the present invention;

[0025] Figure 2 It is a flowchart for constructing a ship wake database according to the present invention;

[0026] Figure 3 It is a flowchart of the matching method proposed by the present invention. Specific embodiments

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0028] A method for estimating ship motion parameters based on wake matching specifically includes the following steps:

[0029] Step 1: Construct a ship wake model:

[0030] As Figure 1 and Figure 2 shown, the present invention constructs a sea surface ship wake model. The sea surface ship wake model consists of a sea surface model and a ship wake model. Among them, the sea surface model simulates and calculates the sea surface based on sine waves and the sea surface based on sea spectra through hybrid calculation. The ship wake model simulates and calculates the wake height field and the wake temperature field based on the geometric model and temperature model of the wake respectively, and converts the scene radiation calculation results into image textures for rendering to obtain an infrared sea surface ship wake model;

[0031] The sea surface can be regarded as the superposition of a series of waves with different amplitudes, wavelengths, and phases. The sea spectrum model describes the distribution of wave energy in the frequency domain and direction domain through mathematical functions. The sea surface model of the present invention simulates and generates the sea surface height field results using the sea spectrum model at the near-sea surface grid points, showing the dynamic details on the sea surface, and using sine waves with superimposed noise at the far-distance grid points to reduce the overall repetition of the sea surface.

[0032] The ship wake height field is based on a thin ship model. By introducing an integral domain transformation, the ship's turning during navigation is realized. Using the empirical model of the lateral wave attenuation of the Kelvin wake, the attenuated wake height field is calculated, and an improved height field model of the ship Kelvin wake is established;

[0033] The change in the ship wake temperature is because the ship's power system stirs the deep low-temperature seawater to the sea surface, forming a temperature difference with the sea surface, which appears as a dark band on the infrared detector. Based on the vertical distribution characteristics of the sea surface temperature, combined with the generation mechanism and characteristics of the ship's thermal wake, the width and temperature distribution formula of the ship's thermal wake are calculated, and the temperature field model of the ship wake is constructed accordingly. By adjusting the motion parameters, the ship wake temperature fields under different motion parameters are established;

[0034] The textures that need to be pre-calculated in the infrared simulation of the sea surface wake include: sky radiation texture, atmospheric radiation texture, and solar radiation texture. The scene radiation calculation results are converted into image textures using MODTRAN software, and the above-mentioned sea surface and wake are rendered to obtain an infrared sea surface ship wake model, and a sea surface ship wake model that can simulate multiple speeds, multiple headings, and multiple sea conditions is constructed.

[0035] Step 2: Construct a database, including a ship wake template database and a ship wake actual shot database

[0036] Based on the ship wake model, a ship wake template database is established. Different motion parameters are set to generate infrared images of the ship wake, and a ship wake database is constructed.

[0037] Construct a ship wake template database:

[0038] In this embodiment, 31 different speeds from 0 m / s to 30 m / s of the wake, 10 different headings from 0° to 45° of the turning angle, and 7 different sea conditions from a stationary sea surface to a six-meter wave height of the sea waves are selected. With the help of the Unity3D rendering engine, 2170 images of 1200×1200 are simulated as the template images for subsequent matching estimation.

[0039] Construct a ship wake actual shot database:

[0040] Data collection is carried out by drones, and a total of about 20 minutes of video records are obtained. With the help of the processing tool ffmpeg, the video is split into frames, and a total of 11495 actual shot image data are obtained as the target images for subsequent matching estimation.

[0041] Step 3: Extract the wake to be estimated:

[0042] Use the deep learning convolutional neural network Faster RCNN as the ship wake extraction algorithm, and adopt a multi-task loss function to achieve fast and accurate extraction of ship wakes.

[0043] In this embodiment, the ship training data comes from the public training dataset SeaShips Dataset, which is designed specifically for ship detection. It contains 6 types of ships and approximately 9,000 labeled images. In the experiment, the training iteration times are set to 8x10^4 times and 4×10^4 times respectively, and the batch size is set to 400. After training is completed, wake extraction is performed on the wake real-shot database collected by the present invention to detect the areas with wakes in the images, facilitating subsequent matching.

[0044] Step 4: Construct a feature point extraction module:

[0045] The present invention designs a feature point extraction module, which is constructed by the DOG operator and is used to extract the feature points of the template image and the target image. The module first applies Gaussian blur to the image using Gaussian kernels with different standard deviations, and then subtracts the two Gaussian-blurred images to obtain the DOG response map. Extreme points are detected in the DOG response map as the image feature points. This module has scale invariance and strong anti-interference ability, thus realizing the accurate positioning of feature points.

[0046] Step 5: Construct a feature vector description module:

[0047] The present invention improves based on the L2-Net network model and proposes the EL2-Net (Effective L2-Net) model. A number of key improvements are made for ship wake images: adopting a double-branch non-shared parameter structure to independently extract the feature pairs of images, enhancing the adaptability to differences in viewpoints, lighting, etc.; retaining spatial information and extracting common features through a fully convolutional network to improve feature repeatability; introducing instance normalization to replace batch normalization to accelerate convergence while maintaining the independence between feature instances; using the Leaky ReLU activation function to alleviate the gradient disappearance problem and enhance the non-linear fitting ability. These improvements significantly improve the robustness of the ship wake image feature description, making its key point matching accuracy and stability in complex scenarios superior to the original model.

[0048] The feature point vector description module crops an image block of 32x32 around the feature points extracted in step 4 and inputs it into the EL2-Net feature point vector extraction model proposed by the present invention to extract descriptors for the positions of the key points in the image, obtaining a 1x128-dimensional feature vector description. In view of the significant non-linear differences between different images, the present invention uses a convolutional neural network with non-shared parameters to extract feature point vectors for each image respectively.

[0049] Step 6: Construct a feature point matching module, and the specific steps are as follows:

[0050] The feature point matching module uses the BBF algorithm to find the nearest neighbor descriptor and its Euclidean distance, extracts nodes from the queue, calculates the Euclidean distance from the target point. If the distance is greater than the current nearest distance, the point is ignored and the next node is calculated. If the distance is less than the current nearest distance, the nearest distance is updated, and the nearby child nodes of the node are added to the queue until all nodes are traversed to obtain the node closest to the target point, and it is judged whether the Euclidean distance between the node and the target is less than the threshold. If it is greater than the threshold, the matching fails. If it is less than the threshold, the feature point matching result is obtained.

[0051] Step 7: Estimate the ship motion parameters, and the specific steps are as follows:

[0052] As Figure 3 shown, input the ship wake template database and the actual ship wake data into the multi-stage remote sensing image matching model. After feature point extraction, feature point vector description, and feature point matching, the wake template matching result is output, and the ship motion parameters can be estimated through the best matching result.

[0053] In summary, the present invention provides a method for estimating ship motion parameters based on wake matching. The infrared sea surface model of this method hybridly calculates two sea surfaces, ensuring authenticity while reducing the repeatability of the sea surface; by simulating the wake temperature field and wake height field, the authenticity of the wake is improved; the multi-stage remote sensing image matching algorithm of the present invention can extract the feature vectors of the target wake and achieve accurate matching; the constructed wake dataset can be reused to efficiently estimate the ship motion parameters.

[0054] The above-mentioned invention can also be extended to other ship motion parameter estimation tasks of the same type. Just set appropriate parameters, establish a ship wake template database according to step 2, and perform target matching according to step 7, then the ship motion parameters can be estimated based on the template matching method.

[0055] The above content is only an example and explanation of the concept of the present invention. Those skilled in the art of this technology can make various modifications or supplements to the described specific embodiments or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by this claim book, they should all belong to the protection scope of the present invention.

Claims

1. A ship motion parameter estimation method based on wake matching, characterized in that: The following steps are involved: Construct a sea surface ship wake model including an infrared sea surface model, a ship wake height field and a ship wake temperature field, wherein the infrared sea surface model is calculated and rendered in combination with a sine wave and a sea spectrum sea surface, the ship wake height field is constructed based on a slender ship model and an integral domain transformation and attenuation model, and the ship wake temperature field is constructed according to the sea surface temperature characteristics and the ship thermal wake mechanism, and is adjusted according to motion parameters; Based on the constructed sea surface ship wake model, multiple speed, multiple heading, and multiple sea condition parameters are set to generate ship wake infrared images and build a ship wake database; A deep learning convolutional neural network and a multi-task loss function are used to extract the tail to be estimated; A multi-stage image matching model is constructed based on the feature point extraction module, feature vector description module and feature point matching module, and is used to match the database and batches of trails to be estimated to obtain and select the optimal motion parameter estimation result; The steps of constructing the multi-stage image matching model are as follows: Extracting feature points of the image from the template image and the target image based on the feature point extraction module, cropping image blocks around the feature points according to preset specifications with the feature points as the center, and inputting the image blocks into the feature vector description module to obtain the feature point description vector; The feature point matching module searches for the template image description vector with the smallest Euclidean distance to the feature point description vector, and calculates whether the Euclidean distance between the two is less than a threshold. If it is greater than the threshold, the matching fails; if it is less than the threshold, the feature point matching result is obtained.

2. A method for estimating ship motion parameters based on wake matching according to claim 1, characterized in that: The infrared sea surface model uses a sea surface simulation to mix and calculate a sine wave sea surface and a sea spectrum sea surface. The sea surface simulation uses a sea spectrum model and a sine wave with superimposed noise at near and far sea surface grid points to display details and reduce repetition.

3. The method for estimating ship motion parameters based on wake matching according to claim 1, characterized in that: The ship wake database includes simulation data of multiple speeds, multiple headings and multiple sea conditions.

4. The method for estimating ship motion parameters based on wake matching according to claim 1, characterized in that: The deep learning convolutional neural network is Faster RCNN.

5. The method for estimating ship motion parameters based on wake matching according to claim 1, characterized in that: The feature extraction module is used to extract the feature points of ship wakes in different motion states. Specifically, Gaussian blur is applied to the image using Gaussian kernels with different standard deviations, and then the difference between the two Gaussian blurred images is calculated to screen the local extreme points of the image as feature points.

6. A method for estimating ship motion parameters based on wake matching according to claim 5, characterized in that: The feature vector description module adopts a convolutional neural network with unshared parameters.

7. The method for estimating ship motion parameters based on wake matching according to claim 5, characterized in that: The feature point matching module adopts a kd tree structure.

Images