A non-homologous synthetic aperture radar sea ice drift detection method with intensity difference

Abstract

The application provides a non-homologous synthetic aperture radar sea ice drift detection method with intensity difference. The method comprises the following steps: obtaining a non-homologous SAR image, and pre-processing the non-homologous SAR image to obtain more uniformly distributed control points; extracting regional features by using a deep learning network, and matching the extracted features; removing error points based on a method for removing error matching; and calculating a sea ice drift vector field. The application can realize full-automatic detection of sea ice drift, and by using the advantages of high spatial resolution and high temporal resolution of non-homologous SAR, a high-resolution and quasi-real-time sea ice drift field is obtained, thereby meeting the demand of realizing high-resolution automatic detection of sea ice drift from massive SAR data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of non-homogeneous SAR image sea ice drift, in particular, especially relates to a non-homogeneous synthetic aperture radar sea ice drift detection method with intensity difference. BACKGROUND

[0002] At present, sea ice drift has a strong impact on sea transportation and platform activities, leading to a surge in demand, which requires near real-time high-resolution sea ice drift tracking. In order to carry out near real-time all-weather sea ice drift tracking, a short time interval and high-resolution remote sensing image sequence needs to be established. SAR has the advantages of all-weather and all-day operation. However, due to the long transit period of the homogeneous spaceborne SAR, the time accuracy of sea ice drift tracking is limited, so using non-homogeneous SAR to construct a short time interval image sequence for high-precision sea ice drift tracking will become a trend. With the development of satellite technology, the launch of Advanced SAR (ASAR), RADARSAT-2, European Remote Sensing 1 (ERS-1) SAR, and GAOFEN-3 satellites, SAR data is increasing, providing us with non-homogeneous data guarantee.

[0003] The key to realizing sea ice drift tracking is image registration. Compared with ground features (such as buildings, city squares, etc.), the sea ice structure features are fuzzy, which increases the difficulty of registration. Due to different sensors and product modes, there are many differences in sea ice features of non-homogeneous SAR data, including image intensity. These differences lead to differences in the features of the same sea ice, further reducing the correlation of sea ice features, and thus causing a large number of false matches when matching.

[0004] The intensity difference of non-homogeneous SAR data will cause the gradient direction of the similar feature points extracted by SIFT, AKAZE and other commonly used sea ice drift algorithms to be inconsistent, resulting in inconsistent main directions of the feature description points, and ultimately leading to low correlation of the feature description points. Therefore, when using traditional feature descriptors for matching, a large number of false matches will occur, so the traditional feature descriptors have limited discrimination ability for such non-homogeneous SAR sea ice data, and the matching effect is poor, resulting in a large error in the result. Through the above analysis, the problems and defects of the prior art are:

[0005] 1. The current SAR data has the characteristics of large swath and high resolution, and the data volume increases significantly, so the time resolution of the sea ice drift monitoring algorithm needs to be improved.

[0006] 2. At the method level, the influence of image intensity difference on features and the accuracy of multi-time same point matching need to be overcome in the face of massive SAR images with intensity difference. SUMMARY

[0007] According to the technical problems proposed above, a non-homologous synthetic aperture radar sea ice drift detection method with intensity difference is provided. The present application first pre-processes the image, obtains training samples, extracts and matches features through a deep learning network, calculates the initial sea ice drift velocity vector field, removes the wrong velocity vector according to the kinematic law of the sea, and obtains the final sea ice drift result.

[0008] The technical means adopted by the present application are as follows:

[0009] A non-homologous synthetic aperture radar sea ice drift detection method with intensity difference, comprising:

[0010] Obtain non-homologous SAR images and pre-process the non-homologous SAR images to obtain more uniformly distributed control points;

[0011] Extract regional features using a deep learning network and match the extracted features;

[0012] Based on the method of removing false matches, the false points are removed;

[0013] Calculate the sea ice drift vector field.

[0014] Further, the obtaining of the non-homologous SAR images and the pre-processing of the non-homologous SAR images to obtain more uniformly distributed control points comprises:

[0015] Calculate the variance matrix of the non-homologous SAR image, each element of the variance matrix is calculated from the window w x w, and the calculation formula is as follows:

[0016]

[0017] Where w is the window size, a ij is the pixel value in the window, is the average pixel value in the window, and the generated variance matrix is used as the candidate of the control point;

[0018] The elements in the variance matrix are screened twice to obtain the final control point.

[0019] Further, the screening of the elements in the variance matrix twice to obtain the final control point specifically comprises:

[0020] The first screening is to traverse the local maximum points defined by the surrounding (2R+1) region, where R is a pre-set parameter, and after this screening, there will also be a region without maximum value;

[0021] The second screening is to add variance points to the region without maximum value after the first screening to obtain a relatively uniformly distributed control point.

[0022] Further, the region feature is extracted by using the deep learning network, and the extracted feature is matched, comprising:

[0023] A training sample is obtained, and a group of image blocks are cut from a pair of non-homologous SAR images with the coordinates of the matched feature points as the center points of the control points.

[0024] The obtained training sample is input into a deep learning network to obtain a training model, and the deep learning network comprises a feature network module, a feature fusion module and a decision module.

[0025] Further, in the deep learning network:

[0026] The feature network module comprises 7 convolutional layers, each layer is respectively provided with a certain number of filters, two branches are independently trained, and the stride value of each convolutional layer is set to 1;

[0027] The feature fusion module fuses the features through a connection layer two by two, performs convolution operation through two layers of 3x3 convolution, and further fuses the features of the two branches; a 3x3 convolution and a pooling layer are used, and finally a 7x7 convolution is used to obtain a 1x512-dimensional feature vector. The obtained feature vector is taken as an input into two fully convolutional layers in the metric layer, and finally input into the decision module;

[0028] The decision module is used to determine whether the two image patches output by the feature fusion module are matched by using a label and a loss function, wherein:

[0029] The data set composed of the image block pairs with label information is defined as:

[0030]

[0031] Wherein, N is the number of paired image patches;

[0032] The loss function is defined as the binary cross-entropy loss of the network, and the formula is as follows:

[0033]

[0034] Wherein, n is the number of input pairs, is the label of the ith input pair, is a rough 0 / 1 label, y i is the matching probability corresponding to the comparison input pair.

[0035] Further, the method based on removing the error matching, eliminating the error points, comprising: the obtained matching points ​Comparing with the matching points of surrounding points, the matching points are removed if exceeding the threshold, specifically including:

[0036] The relative consistency of the distance of adjacent matching points:

[0037] The relative displacement distance is obtained based on the matching pair, the Euclidean distance between the matching pairs is calculated according to the initial matching, and the distance similarity of the matching pairs is defined, as follows:

[0038]

[0039] Wherein, D1 and D2 are the Euclidean distances of the matching pairs of surrounding points in a matching, n is the number of surrounding matching points in the corresponding field, and sigma is a fixed parameter;

[0040] The angle consistency of adjacent matching points:

[0041] The angle of the center point and the n surrounding points is calculated, and the calculation formula is as follows:

[0042]

[0043] The average rotation angle of the surrounding points is calculated, and the calculation formula is as follows:

[0044]

[0045] The deviation of the angle of the center matching point from the average angle is calculated, and the calculation formula is as follows:

[0046]

[0047] Wherein, θ d The value range of is [0, π];

[0048] For each pair of matching points, if d k >K and θ d <r, it means that the point is an inner point, and the others are determined as false matching.

[0049] Further, the sea ice drift vector field is calculated, including:

[0050] The movement distance is calculated, that is, the movement distance between the points in the two matching SAR images;

[0051] The speed of sea ice drift is calculated, that is, the moving distance divided by the time interval.

[0052] Compared with the prior art, the present application has the following advantages:

[0053] 1. The non-homologous synthetic aperture radar sea ice drift detection method with intensity difference provided by the application can realize full-automatic detection of sea ice drift, and by taking advantage of high spatial resolution and high temporal resolution of non-homologous SAR, a high-resolution, quasi-real-time sea ice drift field is obtained, thereby meeting the demand of realizing high-resolution automatic detection of sea ice drift from massive SAR data.

[0054] 2. The non-homologous synthetic aperture radar sea ice drift detection method with intensity difference provided by the application breaks the long time interval of homologous SAR images, and realizes near-real-time sea ice monitoring by cooperating non-homologous SAR images.

[0055] Based on the above reasons, the application can be widely promoted in the field of non-homologous SAR image sea ice drift. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor under the premise of the drawings.

[0057] Figure 1 The method flowchart of the application.

[0058] Figure 2 The deep learning network structure diagram of the application.

[0059] Figure 3 The sea ice drift vector diagram obtained by using two algorithms provided by the embodiment of the application.

[0060] Figure 3 Fig. 2: (a) represents the sea ice drift vector diagram of the algorithm of the application; (b) represents the sea ice drift vector diagram of the traditional SIFT algorithm. DETAILED DESCRIPTION

[0061] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict. The application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0062] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. The following description of at least one example embodiment is merely illustrative in nature and is in no way limiting on the application or its uses. Based upon a review of the embodiments in the present application, all other embodiments that would be obvious to one of ordinary skill in the art are intended to be within the scope of the present application.

[0063] It should be noted that the terminology used herein is for the purpose of describing specific embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0064] The relative arrangement of parts and steps, numerical expressions, and numerical values set forth in the examples are not intended to limit the scope of the application, unless otherwise specifically stated. It should be apparent that the size of the parts shown in the drawings have not been drawn to scale for the sake of convenience and clarity. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, the techniques, methods, and apparatus are considered to be part of the specification and are hereby expressly incorporated by reference. In all examples shown and discussed herein, any specific value should be interpreted as merely an example and not as a limitation. Thus, other examples of the example embodiments can have different values. It should be noted that like reference numerals and letters refer to like items in the following drawings, and thus, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0065] In the description of the present application, it should be understood that the orientation words such as "front, back, up, down, left, right", "transverse, vertical, perpendicular, horizontal", and "top, bottom" and the like indicated orientation or position relationship are generally based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and in the absence of contrary description, these orientation words do not indicate and imply that the indicated device or element must have a particular orientation or be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the scope of protection of the present application: the orientation words "inner, outer" refer to the inner and outer relative to the contour of each component.

[0066] For purposes of the description hereinafter, spatially relative terms are used for convenience to describe the orientation of one device or feature to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. For example, if a device is inverted in the figure, a device or element that is described as being on the top of other devices or elements would then be oriented on the bottom thereof. Thus, the exemplary term "on top of" can encompass both an orientation of above and below. The devices can be oriented in other ways (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly. It will be understood that the terms and words used herein are not intended to limit the scope of the application but are intended to describe specific embodiments so that the patent claims will be interpreted under a broad scope as reasonably permitted by law. Any and all embodiments are intended to mean that at least one embodiment fell within the scope of the appended claims.

[0067] In addition, it should be noted that the use of "first", "second", and the like words of similar meaning used in the description and claims to refer to particular structures or device components are used only for convenience and are not intended to limit the protection scope of the present application, unless otherwise stated, and therefore cannot be understood as limiting the protection scope of the present application.

[0068] As shown in Figure 1 The present application provides a non-homogeneous synthetic aperture radar sea ice drift detection method with intensity difference, comprising:

[0069] S1, obtaining a non-homogeneous SAR image, and pre-processing the non-homogeneous SAR image to obtain more uniformly distributed control points;

[0070] S2, extracting regional features using a deep learning network, and matching the extracted features;

[0071] S3, removing error points based on a method for removing error matches;

[0072] S4, calculating to obtain a sea ice drift vector field.

[0073] In specific implementation, as a preferred embodiment of the present application, in step S1, the non-homogeneous SAR image is obtained, and the non-homogeneous SAR image is pre-processed to obtain more uniformly distributed control points, comprising:

[0074] S11, calculating a variance matrix of the non-homogeneous SAR image, each element of the variance matrix being calculated from a window w x w, and the calculation formula being as follows:

[0075]

[0076] wherein w is the window size, a ij is the pixel value in the window, is the average pixel value in the window, and the generated variance matrix is used as a candidate for the control point;

[0077] S12, two screenings are performed on the elements in the variance matrix to obtain the final control point. Specifically, the screening includes:

[0078] S121, the first screening is to traverse the local maximum points defined by the (2R+1) region, where R is a pre-set parameter. After the first screening, there will be a region without a maximum value;

[0079] S122, the second screening is to add a variance point to the region without a maximum value obtained after the first screening to obtain a relatively uniformly distributed control point.

[0080] In a specific implementation, as a preferred embodiment of the present application, in the step S2, the region features are extracted by using a deep learning network, and the extracted features are matched, including:

[0081] S21, training samples are obtained, and the coordinates of the matching feature points are taken as the center points in a pair of non-homologous SAR images to obtain image blocks to form the training samples; in this embodiment, small blocks of n*n voxels are extracted at the original resolution. In order to improve the receptive field of the network, small blocks of 2n*2n pixels are additionally extracted and down-sampled to n*n pixels. In this way, local features and global features are integrated together to obtain better structural information for distinguishing sea ice and to add context information. Although down-sampling will lose some texture feature information, it can retain more structural features and context information. Image down-sampling can also ensure that the impact on memory consumption and computational overhead is limited. That is, four image blocks are obtained as inputs. Data augmentation methods such as rotation are used to enhance the data.

[0082] S22, the obtained training samples are input into a deep learning network to obtain a training model, and the deep learning network includes a feature network module, a feature fusion module, and a decision module. In the deep learning network:

[0083] The feature network module includes 7 convolutional layers, and a certain number of filters are set for each layer. In this embodiment, the number of filters set for each layer is 32, 32, 64, 64, 128, 128, and 128. Meanwhile, two branches are independently trained, and the weights are not shared to ensure that the two branches do not affect each other. The stride value of each convolutional layer is set to 1 to prevent loss of spatial accuracy;

[0084] The feature fusion module fuses the features through a connection layer two by two, performs convolution operation through two layers of 3x3 convolution, and further fuses the features of the two branches; uses 3x3 convolution and a pooling layer, and finally uses a 7x7 convolution to obtain a 1x512-dimensional feature vector; the obtained feature vector is taken as input and input into two fully convolutional layers in the metric layer, and finally input into the decision module;

[0085] The decision module is used for judging whether the two image patches output by the feature fusion module are matched or not through a label and a loss function, wherein:

[0086] The data set composed of the image block pairs with label information is defined as:

[0087]

[0088] Wherein, N is the number of paired image patches;

[0089] The loss function is defined as the binary cross entropy loss of the network, and the formula is as follows:

[0090]

[0091] Wherein, n is the number of input pairs, is the label of the i-th input pair, is a rough 0 / 1 label, y i is the matching probability corresponding to the comparison input pair.

[0092] In specific implementation, as a preferred embodiment of the present application, the sea ice drift movement is affected by ocean currents and atmospheric circulation. At the same time, the sea surface objects also receive pressure from each other. Therefore, the sea surface object matching points have high similarity with the surrounding sea surface objects. After the rough matching of the SAR image pair, a method for removing false matching is proposed according to the characteristics of sea ice movement. Therefore, in the step S3, the false points are removed based on the method for removing false matching, including: comparing the obtained matching points with the matching points of the surrounding points, and removing the matching points if exceeding the threshold value, specifically including:

[0093] S31, the distance relative consistency of adjacent matching points:

[0094] Based on the relative displacement distance calculated by the matching pair, the Euclidean distance between the matching pairs is calculated according to the initial matching, and the distance similarity of the matching pairs is defined, as follows:

[0095]

[0096] Wherein, ​It is the Euclidean distance between D1 and D2 that matches the surrounding point matching pairs. n is the number of surrounding matching points in the corresponding field, and σ is a fixed parameter; according to experience, σ is set to σ = 0.5.

[0097] S32. Angle consistency of adjacent matching points:

[0098] Calculate the angles of the central point and its surrounding n points. The calculation formula is as follows:

[0099]

[0100] Calculate the average rotation angle of the surrounding points. The calculation formula is as follows:

[0101]

[0102] Calculate the deviation between the angle of the central matching point and the average angle. The calculation formula is as follows:

[0103]

[0104] Among them, θ d ranges from [0, π];

[0105] For each pair of matching points, if d k > K and θ d < r, it means that this point is an inlier, and others are judged as incorrect matches.

[0106] Embodiment

[0107] Experimental image data: The experimental data are images of the Liaodong Bay area. One image comes from Radarsat-2 data; the other image comes from COSMO-Skymed data.

[0108] It can be seen from Figure 3 that SIFT can extract 15 drift vectors, and the method proposed in this invention can extract 208 drift vectors. Compared with the prior art, in this invention, the network in this invention and the traditional SIFT matching algorithm and match-Net network in the prior art are used to evaluate various networks with the same test set samples. The evaluation results are shown in Table 1:

[0109] Table 1 Accuracy of the test sets of three network simulation experiments

[0110] SIFT algorithm herein recall 0.090301003 0.933110368 precision 0.710526316 0.92384106

[0111] It can be seen from Table 1 that compared with the SIFT algorithm, the matching accuracy and recall rate of the test set sample image patch pairs in the network of this invention are the highest. This shows that the network structure proposed in the method of this invention has better performance in feature extraction and matching.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting sea ice drift using non-homogeneous synthetic aperture radar with intensity differences, characterized in that, include: Acquire non-originating SAR images and preprocess them to obtain more evenly distributed control points, including: Calculate the variance matrix of non-originating SAR images, with each element of the variance matrix derived from a window. × The calculation is as follows: in, It refers to the window size. These are the pixel values ​​within the window. It is the average pixel value within the window, and the generated variance matrix serves as a candidate control point. The final control points are obtained by performing two rounds of filtering on the elements of the variance matrix, where: The first filtering step is to iterate through the defined surrounding area (2 R +1) local maxima in the region, where R For pre-set parameters, after this filtering, there will still be areas where no maximum value exists; The second screening involves adding variance points to the regions that do not have maximum values ​​after the first screening, thereby obtaining a relatively evenly distributed control point. Deep learning networks are used to extract region features, and the extracted features are then matched. Based on the method of removing erroneous matches, erroneous points are eliminated, including: the obtained matching points The points are compared with matching points in the surrounding area. If the number of matching points exceeds a threshold, the matching points are removed. Specifically, this includes: The distances between adjacent matching points are relatively consistent: The relative displacement distance is calculated based on the matching pairs. The Euclidean distance between the matching pairs is calculated based on the initial matching, and the distance similarity of the matching pairs is defined as follows: in, It is a matching pair of surrounding points in a match. and Euclidean distance, It is the number of surrounding matching points in the corresponding domain. It is a fixed parameter; Angular consistency between adjacent matching points: Calculate the center point and its surroundings. The angle of each point is calculated using the following formula: The average rotation angle of the surrounding points is calculated using the following formula: The deviation between the angle of the center matching point and the average angle is calculated using the following formula: in, The range of values ​​is ; For each pair of matching points, if and If the match is true, it means that the point is an interior point; all others are considered incorrect matches. The sea ice drift vector field was calculated.

2. The method for detecting sea ice drift with intensities using non-homogeneous synthetic aperture radar according to claim 1, characterized in that, The step of extracting region features using a deep learning network and matching the extracted features includes: To obtain training samples, image patches are extracted from a pair of non-originating SAR images, using the coordinates of the matching feature points obtained from the above control points as the center point, to form training samples. The acquired training samples are input into a deep learning network to obtain a training model. The deep learning network includes a feature network module, a feature fusion module, and a decision module.

3. The method for detecting sea ice drift with intensities using non-synthetic aperture radar according to claim 2, characterized in that, In the deep learning network: The feature network module includes 7 convolutional layers, each of which is equipped with a certain number of filters. The two branches are trained independently, and the stride value of each convolutional layer is set to 1. The feature fusion module fuses features pairwise through a connection layer, performs convolution operations through two 3×3 convolution layers, and further fuses the features of the two branches; using 3×3 convolution and pooling layers, and finally using a 7×7 convolution to obtain a 1×512-dimensional feature vector, the obtained feature vector is used as input to two fully convolutional layers in the metric layer, and finally input to the decision module. The decision module is used to determine whether two image patches output by the feature fusion module match based on labels and a loss function, wherein: The dataset consisting of image patch pairs with labeled information is defined as follows: in, N The number of paired image patches; The loss function is defined as the binary cross-entropy loss of the network, as shown in the following formula: in, To input the number of pairs, For the first The label for each input pair is a rough 0 / 1 label. To compare input pairs The corresponding matching probability.

4. The method for detecting sea ice drift with intensities using non-synthetic aperture radar according to claim 1, characterized in that, The calculation to obtain the sea ice drift vector field includes: Calculate the motion distance, i.e., the motion distance between points in two matched SAR images; Calculate the speed of sea ice drift, which is the distance traveled divided by the time interval.

Images