A method for estimating and imaging motion parameters of a moving target on the sea surface based on deep learning

Abstract

The application relates to the technical field of image data processing, in particular to a sea surface moving target motion parameter estimation and imaging method based on deep learning, which comprises the following steps: acquiring a data set; constructing a feature extraction model and a mapping model; acquiring an image domain of a SAR image, and determining a target motion trajectory area in the image domain; acquiring motion trajectory parameter properties of the image domain by using the number of pixels of the target motion trajectory area and the main direction angle value; acquiring highlight properties and stretch distortion properties of the image domain, and then acquiring convolution feature defocusing properties of the image domain; taking the SAR image as the input of the feature extraction model, taking the motion trajectory parameter properties as the fusion weight of a main network, taking the convolution feature defocusing properties as the fusion weight of a secondary network, and outputting the fusion features of the SAR image; and taking the fusion features of the SAR image as the input of the mapping model, and outputting the motion parameters of the moving target. The application can improve the accuracy of moving target motion parameter estimation.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, specifically to a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface. Background Technology

[0002] In fields such as traffic safety and environmental protection, it is often necessary to monitor and identify dynamic targets on the sea surface.

[0003] Existing problems: Currently, synthetic aperture radar (SAR) imaging is mainly used to monitor and identify dynamic targets on the sea surface. However, since dynamic targets on the sea surface are in a state of constant motion, their range velocity will cause Doppler frequency shift, resulting in target offset and misalignment. Furthermore, the azimuth velocity will cause changes in the azimuth modulation frequency, resulting in target defocus. Therefore, when using SAR imaging to monitor and identify dynamic targets, there will be image offset and defocus problems, which will prevent the provision of high-precision imaging results and high-accuracy estimation results of moving target motion parameters. Summary of the Invention

[0004] This invention provides a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface, in order to solve existing problems.

[0005] The present invention provides a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface, which employs the following technical solution:

[0006] One embodiment of the present invention provides a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface, the method comprising the following steps:

[0007] Obtain the dataset; the dataset includes several SAR images;

[0008] A feature extraction model and a mapping model are constructed; the feature extraction model is used for feature extraction of SAR images, and the mapping model is used to obtain the motion parameters of moving targets. The feature extraction model includes a main network and a secondary network.

[0009] All image domains of each SAR image are obtained, and the target motion trajectory region in each image domain is determined using the grayscale value of each image domain. The motion trajectory parameters of each image domain of each SAR image are obtained using the number of pixels and the principal direction angle value of the target motion trajectory region in each image domain.

[0010] Based on the contrast value and grayscale mean of each image domain, the brightness of each image domain of each SAR image is obtained; using the principal direction angle value of the target motion trajectory region in each image domain, the stretching and distortion of each image domain of each SAR image is obtained; based on the brightness and stretching and distortion of each image domain, the defocusing of the convolution feature of each image domain of each SAR image is obtained.

[0011] Each SAR image is used as the input to the feature extraction model. The motion trajectory parameters of each image domain of each SAR image are used as the fusion weights of the main network, and the defocusing of the convolutional features of each image domain of each SAR image are used as the fusion weights of the secondary network. The fused features of each SAR image are output.

[0012] The fused features of each SAR image are used as input to the mapping model, and the motion parameters of the moving target are output.

[0013] Furthermore, the specific steps for acquiring all image domains of each SAR image and determining the target motion trajectory region in each image domain using the grayscale value of each image domain are as follows:

[0014] Each SAR image is uniformly divided to obtain all image domains corresponding to each SAR image;

[0015] Perform edge detection on each image domain to obtain all connected components in each image domain;

[0016] Obtain the mean gray value of each connected component, and determine the connected component with the largest mean gray value as the target motion trajectory part in each image domain;

[0017] Morphological algorithms are used to extract the skeleton of the target motion trajectory in each image domain, thereby obtaining the target motion trajectory region in each image domain.

[0018] Furthermore, the specific steps for obtaining the motion trajectory parameters of each image domain of each SAR image by utilizing the number of pixels and the principal direction angle value of the target motion trajectory region in each image domain are as follows:

[0019] The principal direction angle value of the target motion trajectory region in each image domain is obtained by principal component analysis algorithm;

[0020] Calculate the number of pixels in the target motion trajectory region in each image domain of each SAR image;

[0021] Obtain the sum of the number of pixels in the target motion trajectory region of the same image domain in consecutive frames of SAR images;

[0022] Calculate the difference in the number of pixels in the target motion trajectory region of the same image domain in adjacent SAR image frames;

[0023] The number of consecutive SAR images is obtained, as well as the difference in the principal direction angle values ​​of the target motion trajectory region in adjacent SAR images within the same image domain.

[0024] Based on the number of consecutive frame SAR images, the sum of the number of pixels in the target motion trajectory region of the same image domain in consecutive frame SAR images, the difference in the number of pixels in the target motion trajectory region of the same image domain in adjacent frame SAR images, and the difference in the principal direction angle value of the target motion trajectory region of the same image domain in adjacent frame SAR images, the motion trajectory parameter of each image domain of each SAR image is obtained.

[0025] Furthermore, the specific steps for obtaining the motion trajectory parameters of each image domain in each SAR image based on the number of consecutive frame SAR images, the sum of the number of pixels in the target motion trajectory region of the same image domain in consecutive frame SAR images, the difference in the number of pixels in the target motion trajectory region of the same image domain in adjacent frame SAR images, and the difference in the principal direction angle value of the target motion trajectory region of the same image domain in adjacent frame SAR images are as follows:

[0026] The absolute value of the difference in the number of pixels in the target motion trajectory region in adjacent frame SAR images within the same image domain is determined as the pixel number difference between adjacent frames.

[0027] The absolute value of the difference between the principal direction angle values ​​of the target motion trajectory regions in adjacent frame SAR images within the same image domain is determined as the principal direction angle difference between adjacent frames.

[0028] Divide the difference in the number of pixels between adjacent frames by the difference in the main direction angle between the adjacent frames to obtain the quotient of the number of pixels and the main direction angle for each SAR image.

[0029] Based on the number of consecutive frame SAR images, the quotient of the number of pixels in the consecutive frame SAR images and the principal direction angle is summed. The summation result is multiplied by the sum of the number of pixels in the target motion trajectory region of the same image domain in the consecutive frame SAR images. The product is used to determine the motion trajectory parameter of each image domain of each SAR image.

[0030] Furthermore, the specific steps for obtaining the brightness of each image domain of each SAR image based on the contrast value and grayscale mean of each image domain are as follows:

[0031] Obtain the contrast value and grayscale mean of each image domain in each SAR image, as well as the grayscale mean of the neighboring image domains of each image domain;

[0032] The brightness of each image domain of each SAR image is obtained based on the contrast value and grayscale mean of each image domain of each SAR image, as well as the grayscale mean of the adjacent image domains of each image domain.

[0033] Furthermore, the specific steps for obtaining the brightness of each pixel region of each SAR image based on the contrast value and average gray value of each pixel region of each SAR image, as well as the average gray value of the neighboring pixel regions of each pixel region, are as follows:

[0034] Obtain the average gray value of each neighboring image domain for each image domain;

[0035] The mean of the gray values ​​of all neighboring image domains of each image domain is determined as the neighborhood gray value of each image domain.

[0036] Obtain the difference between the mean gray value of each image domain and the gray value of its neighbors, and determine the absolute value of the difference between each image domain and its neighbors.

[0037] The contrast value of each image domain and the grayscale difference between each image domain and its neighbors are multiplied together, and the product is used to determine the brightness of each image domain in each SAR image.

[0038] Furthermore, the specific steps for obtaining the stretching and distortion of each image domain of each SAR image using the principal direction angle value of the target motion trajectory region in each image domain are as follows:

[0039] The absolute value of the difference in the principal direction angle values ​​of the target motion trajectory region in adjacent SAR images within the same image domain is determined as the principal direction difference for each SAR image.

[0040] The principal orientation differences of consecutive SAR images are summed, and the normalized value of the summation result is used to determine the stretching distortion of each image domain of each SAR image.

[0041] Furthermore, the specific steps for obtaining the defocusing properties of the convolutional features of each SAR image domain based on the brightness and stretching distortion of each domain are as follows:

[0042] Set the defocus weight, and obtain the first and second coefficients based on the defocus weight;

[0043] The convolutional feature defocusing of each SAR image domain is obtained by adding the product of the first coefficient and the brightness of each image domain, and the product of the second coefficient and the stretching and distortion of each image domain.

[0044] Furthermore, the specific steps of using each SAR image as input to the feature extraction model, using the motion trajectory parameters of each pixel domain of each SAR image as the fusion weights of the main network, and using the defocusing convolutional features of each pixel domain of each SAR image as the fusion weights of the secondary network, to output the fused features of each SAR image, are as follows:

[0045] Each SAR image is simultaneously input into the main network and the secondary network to obtain the main network features and secondary network features of each SAR image.

[0046] The motion trajectory parameters of each image domain of each SAR image are used as the fusion weights of the main network, and the defocusing of the convolutional features of each image domain of each SAR image are used as the fusion weights of the secondary network.

[0047] The fusion features of each SAR image are obtained based on the main network features and secondary network features of each SAR image, as well as the fusion weights of the main network and secondary network.

[0048] Furthermore, the specific steps of using the fused features of each SAR image as input to the mapping model and outputting the motion parameters of the moving target are as follows:

[0049] The fused features of each SAR image are input into the convolutional layer of the mapping model, and the features extracted by the convolutional layer are output.

[0050] The features extracted by the convolutional layer are input into the first fully connected layer for processing and fusion to obtain the fused features from the first fully connected layer.

[0051] The features fused from the first fully connected layer are input into the second fully connected layer, which outputs the predicted motion parameters of the moving target.

[0052] The beneficial effects of the technical solution of the present invention are as follows: The embodiments of the present invention propose a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface. By analyzing SAR images, the method obtains the fusion weights of the primary and secondary networks of the feature extraction model, so that when the feature extraction model extracts features from the image, it can take into account the misalignment and defocusing effects caused by moving targets, thereby improving the accuracy of the feature extraction model. Then, based on the high-accuracy image fusion features, the motion parameters of the moving targets are estimated, which can improve the accuracy of the motion parameter estimation of the moving targets. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating the steps of a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface according to the present invention.

[0055] Figure 2 This is a schematic diagram of a SAR image;

[0056] Figure 3 This is a schematic diagram of the feature extraction model structure. Detailed Implementation

[0057] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface, proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0059] The following description, in conjunction with the accompanying drawings, details a specific scheme for a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface, provided by this invention.

[0060] Please see Figure 1 The diagram illustrates a flowchart of a deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface, according to an embodiment of the present invention. The method includes the following steps:

[0061] Step S001: Obtain the dataset; wherein the dataset includes several SAR images.

[0062] It should be noted that: This embodiment analyzes SAR images to obtain the fusion weights of the primary and secondary networks of the feature extraction model, so that when the feature extraction model extracts features from the image, it can take into account the misalignment and defocusing effects caused by moving targets, thereby improving the accuracy of the feature extraction model. Then, based on the high-accuracy image fusion features, the motion parameters of the moving targets are estimated, which can improve the accuracy of the motion parameter estimation of the moving targets.

[0063] To train and test the feature extraction model, SAR (Synthetic Aperture Radar) images are needed as a dataset. These SAR images represent coarse imaging results of moving targets obtained through SAR technology, such as... Figure 2 As shown in the figure. These datasets include images of moving targets at different speeds to improve the robustness of the model.

[0064] The training set should contain images of moving targets at multiple different speeds, and these images should include corresponding speed labels. The test set should contain images of targets at unseen speed ranges to evaluate the model's generalization ability.

[0065] Specifically, SAR images of moving targets at different speeds are acquired, and then the motion velocity (range velocity and azimuth velocity) of each image in the training set is labeled using a physical model or external sensors to ensure the accuracy of the training data.

[0066] Step S002: Construct a feature extraction model and a mapping model; wherein, the feature extraction model is used for feature extraction of SAR images, and the mapping model is used to obtain motion parameters of moving targets. The feature extraction model includes a main network and a secondary network.

[0067] It should be noted that traditional SAR imaging technology is typically used for monitoring and analyzing stationary targets, while in practical applications, targets are often non-stationary, such as moving targets on the sea surface. Due to the non-cooperative motion of moving targets on the sea surface, their range velocity causes Doppler frequency shift, while their azimuth velocity causes changes in azimuth frequency modulation. Range velocity causes target offset, while azimuth velocity causes defocus. Therefore, during target motion parameter estimation and imaging, monostatic radar inevitably produces corresponding offset and defocus effects, which have a particularly severe impact on target localization results. These effects prevent traditional SAR imaging technology from providing high-precision imaging results for the monitoring and identification of moving targets on the sea surface. Therefore, solving the image offset and defocus problems caused by moving targets has become a significant technical challenge in the monitoring of moving targets on the sea surface.

[0068] In this embodiment, a feature extraction model and a mapping model are first constructed, and then the feature extraction model and the mapping model are trained using a training set to obtain the trained feature extraction model and the mapping model. Finally, the generalization ability of the trained feature extraction model and the mapping model is evaluated using a test set.

[0069] It should be noted that the feature extraction model in this embodiment is a pixel-level feature extraction network with primary and secondary parallel inputs. This network includes a primary network and a secondary network:

[0070] The main network is used to extract image domain features from SAR images of moving targets. The main network's task is to extract global and local detail features from SAR images to predict the motion parameters of moving targets.

[0071] The sub-network is used to provide image-domain features at similar velocities. Its role is to provide supplementary defocusing feature information, especially when the azimuth velocity of a moving target is small or does not change significantly. The sub-network can improve the estimation accuracy of azimuth velocity by comparing image differences at similar velocities.

[0072] The main network and the secondary network have the same structure, usually a convolutional neural network (CNN) structure, which ensures that their feature extraction capabilities are equivalent.

[0073] At the end of the network, pixel-level feature differences between the main network and the secondary network are extracted. These differences not only include global changes in the moving target but also capture feature differences in locally defocused areas. Through these differences, the network can more accurately infer the two-dimensional motion velocity of the moving target.

[0074] like Figure 3 As shown, the feature extraction model includes a convolutional module, a spatial attention module, a feature fusion module, and a differential feature extraction module. The convolutional module is used for feature extraction and includes convolutional layers, activation functions, and pooling layers. The spatial attention module allows the model to automatically focus on certain regions, such as a spatial transformer network. The feature fusion module performs feature fusion. The differential feature extraction module calculates subtle and low-level differences in the image, such as a pixel-level differential feature extractor.

[0075] The mapping model represents the mapping relationship between the image domain information of a moving target and its motion parameters. The two-dimensional motion velocity of the moving target (i.e., range velocity and azimuth velocity) is learned through a neural network.

[0076] Range velocity: This parameter is the main factor affecting the positional shift of a moving target in an image. The range motion of a moving target will cause a change in the target's position in the range direction, thus affecting the degree of image blur.

[0077] Azimuth velocity: The azimuth velocity of a moving target is often associated with defocusing features in an image. Azimuth motion can cause image distortion of moving targets, especially at high speeds, resulting in significant defocusing.

[0078] The mapping relationship between the image domain information of a moving target and its motion parameters is as follows:

[0079] Range velocity and position shift: Range velocity causes a shift in the target's position in the image. This is represented in the image as displacement of the moving target in the range direction, reflecting the target's actual motion.

[0080] Azimuth velocity and defocus: Azimuth velocity often causes defocusing. When the velocity of a moving target does not change significantly, the defocus features in the image are not sensitive to changes in velocity. Due to this characteristic, the estimation accuracy of azimuth velocity is relatively low, especially at low speeds.

[0081] To estimate the motion parameters of a moving target, a mapping model needs to be established. The mapping model includes a convolutional layer, a first fully connected layer, and a second fully connected layer. The convolutional layer is used to extract local features from the image, the first fully connected layer is used to further process and fuse the features extracted by the convolutional layer, and the second fully connected layer is used to output the predicted motion parameters of the moving target based on the fused features.

[0082] The neural network is trained using the acquired dataset to optimize the model's parameters. During training, the network learns to extract effective motion parameter prediction information from SAR images and is able to handle different imaging features of moving targets at different speeds.

[0083] The loss function uses mean squared error to minimize the difference between the predicted and actual speeds. Standard deep learning optimization algorithms are employed to optimize the network weights.

[0084] It should be noted that after obtaining the fused features through the feature extraction model, the fused features are input into the mapping model, and the motion parameters of the moving target are output.

[0085] Specifically, the image fusion features of the moving target are used as network input. These features typically contain visual and motion state information of the moving target. Convolutional layers are used to extract local features, including the shape, texture, and motion information of the moving target. The features extracted by the convolutional layers are further processed and fused through several first fully connected layers. The last layer of the model is a second fully connected layer, which outputs the predicted motion parameters, including range velocity and azimuth velocity. The number of neurons in the output layer is the same as the number of motion parameters. The mapping model uses a loss function (such as the mean squared error loss function) to measure the difference between the predicted and true motion parameters, and backpropagates to update the network weights.

[0086] It should be noted that the training of the feature extraction model involves the fusion weights of the main network and the fusion weights of the secondary network. In order to eliminate the image displacement and defocusing effects caused by the moving target, this embodiment analyzes the SAR image and obtains the fusion weights of the main and secondary networks based on the information in the image.

[0087] When a single-base radar is used to monitor moving targets, the movement of the target can cause defocusing, which makes it impossible for the radar's SAR image to accurately estimate the range velocity and azimuth velocity of the moving target.

[0088] To address the defocusing problem and improve the accuracy and convergence efficiency of motion parameter estimation, this embodiment establishes a primary and secondary network for feature extraction from SAR images, followed by training and testing.

[0089] Furthermore, the azimuth velocity of a moving target is a significant cause of defocusing. However, when the velocity of the moving target does not change significantly, the corresponding defocusing features do not change significantly. That is, the defocusing features in the moving target image domain are not sensitive to changes in the parameter domain, which leads to a decrease in the estimation accuracy of the azimuth velocity. Therefore, a feature extraction model with primary and secondary parallel inputs is used for extraction.

[0090] Furthermore, in order to extract the image domain features of SAR images in all aspects and avoid the inaccurate capture of local defocus characteristics caused by indiscriminate feature extraction of SAR images, thus failing to achieve the purpose of accurate detail extraction of SAR images through the sub-network, a spatial attention module was added to the sub-network.

[0091] The spatial attention module is located after the convolution module. It is used to analyze and optimize the effect of the convolution results on the extraction of detailed features of moving targets on the sea surface, and transform it into attention fusion bias, which is then fused with the bias of the main network.

[0092] Similarly, in the dual-feature fusion process, the fusion weight of the main network is also increased to improve the accuracy of the main network in estimating the main parameters (such as the motion trajectory of moving targets on the sea surface) during the motion parameter estimation process.

[0093] Therefore, we first extract the features from the main network to obtain the corresponding motion parameter features and then perform feature fusion.

[0094] For moving targets on the sea surface, such as ships, their movement will result in a certain path as their position in the image changes over time. By capturing the target's trajectory, the network can infer the target's movement pattern or direction.

[0095] Therefore, for the recognition accuracy of the main network, the motion trajectory of the moving target in the main network has high importance and priority. During the network training process, it is necessary to highlight such features and increase their fusion weight.

[0096] Step S003: Obtain all image domains of each SAR image, and use the grayscale value of each image domain to determine the target motion trajectory region in each image domain; use the number of pixels and the principal direction angle value of the target motion trajectory region in each image domain to obtain the motion trajectory parameters of each image domain of each SAR image.

[0097] In this embodiment, each SAR image is uniformly divided to obtain all image domains corresponding to each SAR image; edge detection is performed on each image domain to obtain all connected components in each image domain; the gray-scale mean of each connected component is obtained, and the connected component with the largest gray-scale mean is determined as the target motion trajectory part in each image domain; the skeleton of the target motion trajectory part in each image domain is extracted using a morphological algorithm to obtain the target motion trajectory region in each image domain.

[0098] Specifically, several images in the training set are acquired, and their motion trajectory main parametric properties are analyzed. The higher the value of this property, the higher the corresponding feature fusion bias, because the image domain can better reflect the motion attributes of the moving target, and the motion parameter values ​​of the moving target can be determined more accurately through the model.

[0099] In this context, the imaging image domain refers to a local pixel region within a SAR image. Specifically, a SAR image is uniformly divided into 100 image domains, each with a size of [missing information]. .

[0100] Analyzing the parametric properties of motion trajectories in the image domain: The motion trajectory of a moving target (such as a ship) on the sea surface is usually represented in the image as a gradually fading whitish line, reflecting the changing position of the moving target over time during imaging. Due to the motion of the moving target, the trajectory is usually affected by factors such as image resolution, motion speed, and defocus, and appears as one or several broken whitish line segments.

[0101] Acquire consecutive frames of SAR images acquired over time, and analyze the first frame. An image domain, within the time dimension The motion trajectory parameters of consecutive frame SAR images. The time sampling rate is 1 frame / second. Within seconds, the total number of images was .

[0102] For each SAR image, the first In each SAR image domain, Canny edge detection is performed to obtain several edge contours and connected components. Further, these connected components and their mean gray values ​​are sorted, with the connected component with the highest mean gray value considered as the target motion trajectory (the motion trajectory exhibits a washed-out visual characteristic in the image). A morphological algorithm is then used to extract the skeleton of the target motion trajectory, obtaining the first edge contour in each SAR image. The target motion trajectory area in the image domain.

[0103] In this embodiment, the principal component analysis algorithm is used to obtain the principal direction angle value of the target motion trajectory region in each image domain; the number of pixels of the target motion trajectory region in each image domain of each SAR image is calculated; the sum of the number of pixels of the target motion trajectory region in consecutive frame SAR images of the same image domain is obtained; the difference of the number of pixels of the target motion trajectory region in adjacent frame SAR images of the same image domain is calculated; the number of consecutive frame SAR images and the difference of the principal direction angle value of the target motion trajectory region in adjacent frame SAR images of the same image domain are obtained; based on the number of consecutive frame SAR images, the sum of the number of pixels of the target motion trajectory region in consecutive frame SAR images of the same image domain, the difference of the number of pixels of the target motion trajectory region in adjacent frame SAR images of the same image domain, and the difference of the principal direction angle value of the target motion trajectory region in adjacent frame SAR images of the same image domain, the motion trajectory parameter of each image domain of each SAR image is obtained.

[0104] In this embodiment, the absolute value of the difference in the number of pixels in the target motion trajectory region of the same image domain in adjacent frame SAR images is determined as the pixel number difference between adjacent frames; the absolute value of the difference in the principal direction angle value of the target motion trajectory region of the same image domain in adjacent frame SAR images is determined as the principal direction angle difference between adjacent frames; the pixel number difference between adjacent frames is divided by its corresponding principal direction angle difference between adjacent frames to obtain the quotient of the number of pixels and the principal direction angle of each SAR image; based on the number of consecutive frame SAR images, the quotient of the number of pixels and the principal direction angle of consecutive frame SAR images is summed, and the summation result is multiplied by the sum of the number of pixels in the target motion trajectory region of the same image domain in consecutive frame SAR images, and the product is determined as the motion trajectory parameter of each image domain of each SAR image.

[0105] Specifically, the direction of the target motion trajectory region is obtained through principal component analysis (PCA) algorithm. ,

[0106] Calculate the parametric properties of the motion trajectory:

[0107]

[0108] in, Indicates the first The motion trajectory parameters of each image domain; Indicates the first The sum of the number of pixels in the target motion trajectory region of each image domain in consecutive frames of SAR images; Indicates the first The image domain in the first... The number of pixels in the target motion trajectory region of a frame SAR image; Indicates the first The image domain in the first... The principal direction angle value of the target motion trajectory region in a frame SAR image; Indicates the number of consecutive SAR images; This indicates the difference between the preceding and following frames for this parameter; Indicates the first The difference in the number of pixels in the target motion trajectory region of each image domain in adjacent SAR image frames; Indicates the first The difference in the principal direction angle values ​​of the target motion trajectory region in adjacent SAR image frames.

[0109] By quantitatively analyzing the number of motion trajectory pixels within a pixel domain, we can determine whether that pixel domain plays a significant role in identifying moving targets in the SAR image. When there are no motion trajectory pixels within a pixel domain, the value is 0, meaning that motion trajectory parameters for that pixel domain do not exist. Since there are no moving target results, there is no need for high-weight fusion.

[0110] Furthermore, at different times, when the pixel difference of the motion trajectory within the image domain is large, it indicates that a moving target has passed through the image domain. In this case, the necessity for estimating the target's motion parameters in that image domain is higher, thus requiring a higher fusion weight to be assigned, i.e., possessing higher motion trajectory parameterization. Therefore, the difference... The larger the value, the higher the parametric properties of the motion trajectory.

[0111] The higher the stability of the corresponding motion direction, that is, the higher the motion stability of the moving target on the sea surface, then the direction of the motion trajectory tends to be consistent in different time modes, i.e., the difference... The smaller the value, the higher the parametric properties of the motion trajectory.

[0112] Furthermore, the motion trajectory parameters of each image domain are calculated and normalized, and the normalized values ​​are used as the bias values ​​for feature fusion of the image domain in the SAR image after convolutional feature extraction in the main network.

[0113] Step S004: Based on the contrast value and grayscale mean of each image domain, obtain the brightness of each image domain of each SAR image; using the principal direction angle value of the target motion trajectory region in each image domain, obtain the stretching distortion of each image domain of each SAR image; based on the brightness and stretching distortion of each image domain, obtain the convolutional feature defocus of each image domain of each SAR image.

[0114] It should be noted that after obtaining the fusion bias of the main network, it is necessary to further analyze the extraction effect of the secondary network on the detailed features of the moving target, and remove the influence of defocus on the estimation of target motion parameters.

[0115] To effectively remove defocusing, similarly, a fusion weighting of convolutional attention is added after the convolutional module. The main purpose of this weighting is to distinguish the target defocusing in the SAR image and to calculate the fusion weighting.

[0116] Furthermore, the details of the moving targets on the sea surface are mainly as follows:

[0117] Stretching. During the movement of a moving target, its shape will appear stretched or compressed in the image due to factors such as speed, changing viewpoint, target size, and image quality. Specifically, the aspect ratio of a ship changes at different viewpoints, causing the ship's outline to appear longer or shorter.

[0118] Reflectivity (specular reflection). Specular reflection on the metal surfaces of ships, the sea surface, and other objects causes localized areas of brightness.

[0119] The aforementioned details are manifested in the elongation and irregularity of the edge lines, as well as local highlights.

[0120] To quantify and calculate such features, based on the above SAR image processing, the high contrast characteristics within the image domain are analyzed, thereby determining the characteristics of local high brightness, that is, the high brightness reflection characteristics of local moving targets.

[0121] Analyze the principal component directions within the image domain. When the angle changes drastically, it indicates that the position of the moving target relative to the observer has changed significantly. Especially during motion, if the shape of the moving target is not kept fixed, it will be stretched or compressed, causing the shape of the moving target to become irregular in the image. When the angle changes gently, it means that the shape of the moving target remains relatively stable, that is, there is no obvious stretching or distortion.

[0122] In this embodiment, the contrast value and grayscale mean of each image domain of each SAR image are obtained, as well as the grayscale mean of the adjacent image domains of each image domain; based on the contrast value and grayscale mean of each image domain of each SAR image, as well as the grayscale mean of the adjacent image domains of each image domain, the brightness of each image domain of each SAR image is obtained.

[0123] In this embodiment, the average gray value of each neighboring image domain is obtained; the average of the average gray values ​​of all neighboring image domains of each image domain is determined as the neighborhood gray value of each image domain; the difference between the average gray value of each image domain and the neighborhood gray value is obtained, and the absolute value of the difference is determined as the gray value difference between each image domain and the neighborhood; the contrast value of each image domain and the gray value difference between each image domain and the neighborhood are multiplied together, and the product is determined as the brightness of each image domain of each SAR image.

[0124] Specifically, within the output results of the computational network, the first... Highlighting of individual image domains:

[0125]

[0126] in, Indicates the first Highlighting of each image domain; Indicates the first Contrast values ​​per pixel domain; It represents the average gray level of several (15) surrounding image regions; Indicates the first The average gray level of each image region.

[0127] It should be noted that: first obtain the first... Several neighboring image domains of an image domain and ,calculate and The mean of the grayscale values ​​is obtained .

[0128] When a highlight exists within a pixel area, the contrast value is often high, and the average grayscale value of that pixel area is relatively high compared to other pixel areas, therefore the difference... The larger it gets, the bigger it becomes.

[0129] In this embodiment, the absolute value of the difference between the principal direction angle values ​​of the target motion trajectory region in adjacent frame SAR images of the same image domain is determined as the principal direction difference of each SAR image; the principal direction differences of consecutive frame SAR images are summed, and the normalized value of the summation result is determined as the stretching distortion of each image domain of each SAR image.

[0130] Specifically, further, the stretching or torsion of the moving target can be determined by the change in the principal component angle:

[0131]

[0132] in, Indicates the first Stretching and tortuosity of an image domain; Indicates the first The image domain in the first... The principal direction angle value of the target motion trajectory region in the frame SAR image.

[0133] By analyzing the changes in the principal direction of moving targets in images under different time modalities, and based on the above logic, the stretching and distortion of the target within the image domain is determined. The higher the stretching and distortion, the more detailed features of the moving target need to be analyzed, meaning that the network needs to devote a higher proportion of attention to this part during training, in order to better remove the defocus of the target.

[0134] In this embodiment, a defocus weight is set, and a first coefficient and a second coefficient are obtained based on the defocus weight. The product of the first coefficient and the brightness of each image domain, and the product of the second coefficient and the stretching and distortion of each image domain are added together to obtain the convolutional feature defocus of each image domain of each SAR image.

[0135] Specifically, further, the defocusing property of convolutional features is determined:

[0136]

[0137] in, Indicates the first Defocusing of convolutional features in each image domain; This represents the defocus weight, set to 0.4. This increases the contribution of the moving target's morphological features to the defocus, causing the network to focus more attention on the moving target region during training, thus improving the efficiency of motion parameter conversion. This defocus weight is the convolutional attention fusion weight during network training.

[0138] Step S005: Take each SAR image as input to the feature extraction model, use the motion trajectory parameters of each image domain of each SAR image as the fusion weights of the main network, use the defocusing features of the convolutional features of each image domain of each SAR image as the fusion weights of the secondary network, and output the fusion features of each SAR image.

[0139] In this embodiment, each SAR image is simultaneously input into the main network and the secondary network to obtain the main network features and secondary network features of each SAR image; the motion trajectory parameters of each image domain of each SAR image are used as the fusion weights of the main network, and the defocusing features of the convolutional features of each image domain of each SAR image are used as the fusion weights of the secondary network; based on the main network features and secondary network features of each SAR image, as well as the fusion weights of the main network and the secondary network, the fusion features of each SAR image are obtained.

[0140] Specifically, based on the fusion bias of the main network, primary and secondary feature fusion is performed:

[0141]

[0142] in, This indicates the characteristics after the fusion of primary and secondary elements; This represents the original characteristics of the main network; This represents the original features of the subnetwork. The features in each image domain undergo the aforementioned optimization during the fusion process. It indicates fusion.

[0143] Step S006: Use the fusion features of each SAR image as input to the mapping model and output the motion parameters of the moving target.

[0144] In this embodiment, the fused features of each SAR image are input into the convolutional layer of the mapping model, and the features extracted by the convolutional layer are output. The features extracted by the convolutional layer are input into the first fully connected layer for processing and fusion to obtain the fused features of the first fully connected layer. The fused features of the first fully connected layer are input into the second fully connected layer to output the predicted motion parameters of the moving target.

[0145] After obtaining the fused features, the estimated values ​​of the motion parameters of the moving target on the sea surface are obtained by combining them with the mapping model.

[0146] Traditional networks suffer from errors in estimating motion parameters when monitoring and identifying moving targets on the sea surface due to the differences in target characteristics. This embodiment addresses this by using a primary-secondary parallel network input approach to establish targeted analysis strategies for different networks, effectively eliminating the errors in motion parameters obtained after image recognition.

[0147] In order to improve the feature extraction effect of the main and secondary networks and enhance the accuracy of continuous monitoring of moving targets, this embodiment assigns different fusion weights to the moving target features in the later feature extraction stage based on the different convolution extraction effects of the main and secondary networks. This makes the network have better performance in the process of iterating and extracting features, and thus the moving target motion parameters calculated by the mapping model are more accurate.

[0148] In the process of applying convolutional attention fusion to the subnetwork, the motion characteristics of actual moving targets on the sea surface are comprehensively considered. Predictive analysis is performed by combining motion results under multiple time modalities. Based on the deviation between the prediction and the actual results, the bias value of the feature extraction results of moving targets in the subnetwork is determined, thereby improving the ability to extract detailed features of moving targets on the sea surface and enhancing the network's ability to recognize images.

[0149] This invention is now complete.

[0150] In summary, in this embodiment of the invention, by analyzing SAR images, the primary and secondary network fusion weights of the feature extraction model are obtained. This allows the feature extraction model to take into account the misalignment and defocusing effects caused by moving targets when extracting features from the image, thereby improving the accuracy of the feature extraction model. Then, based on the high-accuracy image fusion features, the motion parameters of the moving targets are estimated, which can improve the accuracy of the motion parameter estimation of the moving targets.

[0151] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning-based method for estimating motion parameters and imaging moving targets on the sea surface, characterized in that, The method includes the following steps: Obtain the dataset; the dataset includes several SAR images; A feature extraction model and a mapping model are constructed; the feature extraction model is used for feature extraction of SAR images, and the mapping model is used to obtain the motion parameters of moving targets. The feature extraction model includes a main network and a secondary network. All image domains of each SAR image are obtained, and the target motion trajectory region in each image domain is determined using the grayscale value of each image domain. The motion trajectory parameters of each image domain of each SAR image are obtained using the number of pixels and the principal direction angle value of the target motion trajectory region in each image domain. Based on the contrast value and grayscale mean of each image domain, the brightness of each image domain of each SAR image is obtained; using the principal direction angle value of the target motion trajectory region in each image domain, the stretching and distortion of each image domain of each SAR image is obtained; based on the brightness and stretching and distortion of each image domain, the defocusing of the convolution feature of each image domain of each SAR image is obtained. Each SAR image is used as the input to the feature extraction model. The motion trajectory parameters of each image domain of each SAR image are used as the fusion weights of the main network, and the defocusing of the convolutional features of each image domain of each SAR image are used as the fusion weights of the secondary network. The fused features of each SAR image are output. The fused features of each SAR image are used as input to the mapping model, and the motion parameters of the moving target are output.

2. The method for estimating and imaging motion parameters of moving targets on the sea surface based on deep learning according to claim 1, characterized in that, The specific steps for acquiring all image domains of each SAR image and determining the target motion trajectory region in each image domain using the grayscale value of each image domain are as follows: Each SAR image is uniformly divided to obtain all image domains corresponding to each SAR image; Perform edge detection on each image domain to obtain all connected components in each image domain; Obtain the mean gray value of each connected component, and determine the connected component with the largest mean gray value as the target motion trajectory part in each image domain; Morphological algorithms are used to extract the skeleton of the target motion trajectory in each image domain, thereby obtaining the target motion trajectory region in each image domain.

3. The method for estimating and imaging motion parameters of moving targets on the sea surface based on deep learning according to claim 1, characterized in that, The specific steps for obtaining the motion trajectory parameters of each image domain of each SAR image by utilizing the number of pixels and the principal direction angle value of the target motion trajectory region in each image domain are as follows: The principal direction angle value of the target motion trajectory region in each image domain is obtained by principal component analysis algorithm; Calculate the number of pixels in the target motion trajectory region in each image domain of each SAR image; Obtain the sum of the number of pixels in the target motion trajectory region of the same image domain in consecutive frames of SAR images; Calculate the difference in the number of pixels in the target motion trajectory region of the same image domain in adjacent SAR image frames; The number of consecutive SAR images is obtained, as well as the difference in the principal direction angle values ​​of the target motion trajectory region in adjacent SAR images within the same image domain. Based on the number of consecutive frame SAR images, the sum of the number of pixels in the target motion trajectory region of the same image domain in consecutive frame SAR images, the difference in the number of pixels in the target motion trajectory region of the same image domain in adjacent frame SAR images, and the difference in the principal direction angle value of the target motion trajectory region of the same image domain in adjacent frame SAR images, the motion trajectory parameter of each image domain of each SAR image is obtained.

4. The deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface according to claim 3, characterized in that, The method for obtaining the motion trajectory parameters of each image domain in each SAR image based on the number of consecutive frame SAR images, the sum of the number of pixels in the target motion trajectory region of the same image domain in consecutive frame SAR images, the difference in the number of pixels in the target motion trajectory region of the same image domain in adjacent frame SAR images, and the difference in the principal direction angle value of the target motion trajectory region of the same image domain in adjacent frame SAR images includes the following specific steps: The absolute value of the difference in the number of pixels in the target motion trajectory region in adjacent frame SAR images within the same image domain is determined as the pixel number difference between adjacent frames. The absolute value of the difference between the principal direction angle values ​​of the target motion trajectory regions in adjacent frame SAR images within the same image domain is determined as the principal direction angle difference between adjacent frames. Divide the difference in the number of pixels between adjacent frames by the difference in the main direction angle between the adjacent frames to obtain the quotient of the number of pixels and the main direction angle for each SAR image. Based on the number of consecutive frame SAR images, the quotient of the number of pixels in the consecutive frame SAR images and the principal direction angle is summed. The summation result is multiplied by the sum of the number of pixels in the target motion trajectory region of the same image domain in the consecutive frame SAR images. The product is used to determine the motion trajectory parameter of each image domain of each SAR image.

5. The deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface according to claim 1, characterized in that, The specific steps for obtaining the brightness of each image domain of each SAR image based on the contrast value and grayscale mean of each image domain are as follows: Obtain the contrast value and grayscale mean of each image domain in each SAR image, as well as the grayscale mean of the neighboring image domains of each image domain; The brightness of each image domain of each SAR image is obtained based on the contrast value and grayscale mean of each image domain of each SAR image, as well as the grayscale mean of the adjacent image domains of each image domain.

6. The deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface according to claim 5, characterized in that, The specific steps for obtaining the brightness of each pixel region of each SAR image based on the contrast value and average gray value of each pixel region of each SAR image, as well as the average gray value of the adjacent pixel regions of each pixel region, are as follows: Obtain the average gray value of each neighboring image domain for each image domain; The mean of the gray values ​​of all neighboring image domains of each image domain is determined as the neighborhood gray value of each image domain. Obtain the difference between the mean gray value of each image domain and the gray value of its neighbors, and determine the absolute value of the difference between each image domain and its neighbors. The contrast value of each image domain and the grayscale difference between each image domain and its neighbors are multiplied together, and the product is used to determine the brightness of each image domain in each SAR image.

7. The deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface according to claim 1, characterized in that, The specific steps for obtaining the stretching and distortion of each image domain of each SAR image by utilizing the principal direction angle value of the target motion trajectory region in each image domain are as follows: The absolute value of the difference in the principal direction angle values ​​of the target motion trajectory region in adjacent SAR images within the same image domain is determined as the principal direction difference for each SAR image. The main direction differences of consecutive frame SAR images are summed, and the normalized value of the summation result is used to determine the stretching distortion of each image domain of each SAR image.

8. The method for estimating and imaging motion parameters of moving targets on the sea surface based on deep learning according to claim 1, characterized in that, The specific steps for obtaining the defocusing properties of the convolutional features of each pixel in each SAR image based on the brightness and stretching distortion of each pixel are as follows: Set the defocus weight, and obtain the first and second coefficients based on the defocus weight; The convolutional feature defocus of each SAR image is obtained by adding the product of the first coefficient and the brightness of each image domain, and the product of the second coefficient and the stretching and distortion of each image domain.

9. The method for estimating and imaging motion parameters of moving targets on the sea surface based on deep learning according to claim 1, characterized in that, The specific steps of using each SAR image as input to the feature extraction model, using the motion trajectory parameters of each pixel of each SAR image as the fusion weights of the main network, and using the defocusing convolutional features of each pixel of each SAR image as the fusion weights of the secondary network, to output the fused features of each SAR image, are as follows: Each SAR image is simultaneously input into the main network and the secondary network to obtain the main network features and secondary network features of each SAR image. The motion trajectory parameters of each image domain of each SAR image are used as the fusion weights of the main network, and the defocusing of the convolutional features of each image domain of each SAR image are used as the fusion weights of the secondary network. The fusion features of each SAR image are obtained based on the main network features and secondary network features of each SAR image, as well as the fusion weights of the main network and secondary network.

10. The deep learning-based method for estimating and imaging motion parameters of moving targets on the sea surface according to claim 1, characterized in that, The specific steps involved in using the fused features of each SAR image as input to the mapping model and outputting the motion parameters of the moving target are as follows: The fused features of each SAR image are input into the convolutional layer of the mapping model, and the features extracted by the convolutional layer are output. The features extracted by the convolutional layer are input into the first fully connected layer for processing and fusion to obtain the fused features from the first fully connected layer. The features fused from the first fully connected layer are input into the second fully connected layer, which outputs the predicted motion parameters of the moving target.

Images