Ship target tracking method based on radar imaging in shielding environment

Abstract

The invention discloses a ship target tracking method based on radar imaging in a shielding environment, belongs to the technical field of target tracking, is used for ship target tracking, and comprises the following steps: obtaining multiple frames of echo images of a time sequence, obtaining a potential target on each frame by using a ship image feature detection method, and obtaining a time accumulation space diagram; initial tracks are extracted from the time accumulation space diagram, and a tracking ID is distributed to each initial track; a frame of echo image is read, the position of a potential target is detected, and the position of the potential target is associated and matched with the tracking ID; and updating the position of a potential target which is not normally associated and matched by using a ship target tracking method combining unscented Kalman filtering and LSTM. According to the method, more potential targets are extracted, and a local matching result is optimized; more and more accurate target information is provided for the VTS system, so that the VTS system can have more prior information of early warning, and safe passage of a main channel is ensured.

Description

Technical Field

[0001] The invention discloses a ship target tracking method based on radar imaging in an obstructed environment, and belongs to the technical field of target tracking. Background Art

[0002] The basic function of the Vessel Traffic Service (VTS) is ship tracking, which mainly relies on maritime radar and AIS base stations. The transmission frequency of the ship-borne AIS transceiver is not fixed, and there are delays and errors. A single maritime radar is difficult to provide stable and reliable tracking effects in complex marine environments, such as under occlusion conditions. Therefore, how to achieve high-precision ship target tracking under occlusion conditions has become a hot topic and difficulty in current research.

[0003] Maritime radar imaging uses pulse compression technology to send a series of modulated pulse signals, which can generate radar echo images in complex marine environments. These images can not only reflect the spatial position of the target, but also provide the target's motion information, such as speed and heading. However, the radar echo image contains not only target information, but also background reflection information, such as reefs and navigation marks. In the marine environment, ship targets are often blocked by other ships and interfered by floating objects and offshore infrastructure, which makes the target confused with the clutter background, making it extremely difficult to detect and continuously track the target. In addition, there are a certain number of non-cooperative small targets in the area close to the main channel and the fishing area for illegal fishing. These non-cooperative small target ships have the characteristics of high maneuverability, low reflectivity, small reflection area, and AIS transmitter turned off. Such targets appear intermittently in the echo image of the maritime radar, and the echo pattern is small. Non-cooperative small targets pose a certain security threat to the main channel and are objects that need to be monitored and warned in advance. The premise of target supervision and warning is target detection and tracking.

[0004] Target tracking can be divided into single target tracking and multi-target tracking according to the number of targets. In maritime radar applications, ship target tracking belongs to the typical multi-target tracking category. Multi-target tracking technology can be divided into the following categories according to its different methodologies: Model-based tracking methods rely on prior knowledge of the target, predict and match the target's motion trajectory through a pre-built model, and are suitable for situations with clear features and low scene complexity; Active contour-based tracking methods gradually approximate the target contour in the image through an evolution curve to extract the object of interest. It can generate smooth and continuous boundaries. It has high flexibility and can handle targets with complex shapes, but it may converge to the wrong boundary in images with complex backgrounds or more noise; Region-based tracking methods achieve stable tracking by analyzing the overall characteristics of the target area, but the algorithm is complex, the computational overhead is high, and the effect is limited when dealing with occlusion or target deformation; Feature-based tracking methods rely on extracting global or local features of the target for matching. In radar images, this method is often not ideal because the shape and features of the ship are highly similar to the offshore buoys in the background. Therefore, when tracking multiple target ships in maritime radar images, it is necessary to comprehensively consider the stability of target features, environmental complexity and the computational efficiency of the algorithm and select an appropriate tracking method. Summary of the invention

[0005] The object of the present invention is to provide a ship target tracking method based on radar imaging in an obstructed environment, so as to solve the problem of poor ship target tracking effect in an obstructed environment in the prior art.

[0006] A ship target tracking method based on radar imaging in an obstructed environment, comprising: Step S1, using radar to scan the channel sea area, obtaining radar echo data and preprocessing the radar echo data, converting the radar echo data into an echo image; performing denoising on the echo image, separating the background and foreground targets, and obtaining an echo image with multiple foreground targets; converting the image into a grayscale image, using Gaussian filtering to remove Gaussian noise of the image, and using threshold processing to binarize the image to obtain a binary echo image; Step S2, obtaining multiple frames of echo images, repeating step S1, obtaining multiple frames of echo images in a time series, obtaining potential targets on each frame by using a ship image feature detection method, and accumulating and drawing multiple targets in multiple images on the same plane according to the time series to obtain a time-accumulated spatial map; obtaining potential targets on each frame by using a ship image feature detection method includes connected domain analysis, boundary tracking, chain code representation, hierarchical structure, contour approximation and feature acquisition; Step S3, extracting the initial track from the time-accumulated spatial map, assigning a tracking ID to each initial track; reading a frame of echo image, detecting the position of the potential target, and associating and matching the position of the potential target with the tracking ID; for the potential target that is normally associated and matched in a frame of radar echo data, extending the track corresponding to the potential target to obtain the latest position of the track; for the potential target that is not normally associated and matched, using the ship target tracking method of the unscented Kalman filter combined with the long short-term memory neural network model LSTM to update the position.

[0007] The preprocessing of radar echo data includes: ; ; ; In the formula, It's Radar The minimum echo unit is mapped to the image coordinate, It's Radar The minimum echo unit is mapped to the image coordinate, is the radial distance of the current echo unit from the radar, It is The azimuth of each echo, with the azimuth starting from due west as 0 and increasing in counterclockwise direction. is the width of the converted image. is the height of the converted image. is the current maximum scanning radial distance, The echo intensity Map to Functions in color space, is the maximum echo intensity. When the echo intensity is 0 Black, when the echo intensity is at its maximum Is red.

[0008] Using Gaussian filtering to remove Gaussian noise from an image involves constructing a Gaussian function for the image. : ; In the formula, the pixel to the left of the center pixel is is a negative value, the right pixel is a positive value, the pixel above the center pixel For negative values, the lower pixel is a positive value, is the standard deviation in the Gaussian filter, which determines the smoothness of the filter; Using threshold processing to binarize an image includes: ; In the formula, is the value of the pixel after segmentation, is the set binarization threshold, is the value of the pixel before segmentation.

[0009] Connected domain analysis includes using connected domain analysis to identify all connected foreground target regions in multiple frames of echo images in a time series. , if the following conditions are met, the pixels belong to the same connected domain: if , and there are adjacent points , so that ; Boundary tracking includes using connected domains to track boundaries and extracting the boundary point set of each connected domain. As a contour, for each connected domain , the corresponding contour is represented as a point set , Each point is a boundary point of the connected domain. for: ; ; The chain code representation includes using the chain code to represent the directions between adjacent points of the contour. and For two consecutive points of the contour, the relationship between the two consecutive points is expressed as a direction encoding: ; In the formula, for The coordinates of for The coordinates of each contour point, the moving direction of each contour point relative to the previous point is represented by a discrete direction value; The hierarchical structure includes defining the parent-child relationship between contours, constructing the hierarchical relationship of contours through topological analysis, and Contours The hierarchical tuple of , It is the outline The next contour at the same level, It is the outline The parent contour, It is the outline The first subcontour of It is the outline The previous contour of Contour approximation involves using polygon fitting to approximate the original contour with a set of points to ensure that the error is within the threshold. Suppose the approximate contour point set is ,and : ; In the formula, is the distance from each point to the fitted line, and The two endpoints of a straight line segment, is the error threshold; Obtaining features involves calculating the area features of potential targets using the polygon area formula based on the approximate contour point set. : ; In the formula, is the number of points in the contour point set, is the last point of the approximate contour point set. The next point after Defined as the first point of the approximate contour point set ,make , ; Center point of the contour for: ; .

[0010] Associating and matching the potential target's position with the tracking ID includes using a global nearest neighbor algorithm to calculate the optimal distance between the observation set and the target set to minimize the association cost function; Calculating the optimal distance includes reading a frame of image from the echo image, extracting potential target information, and setting the potential target as the observation set , the target set is , calculate each With each The Euclidean distance between , construct the distance matrix : ; Minimizing the associated cost function involves defining a global cost function Measure the matching between observation and target, and minimize the total matching distance: ; The global cost function includes subtracting the minimum value of each row of the distance matrix, retaining the value of 0 in each row, and subtracting the minimum value of each column of the distance matrix, retaining the value of 0 in each column, to obtain a new distance matrix; Use the least straight line to cover all zero elements in the matrix. Each row and column has at most one 0 value involved in the match. If found If the value 0 is matched, the optimal solution is found and the minimum ; If not found For 0 values, adjust the matrix to find the minimum number of lines covering all 0 value elements. If the minimum number of lines is equal to , then continue to look for the optimal solution. If the minimum number of lines is not equal to , then continue to adjust the matrix, find the smallest element value that is not covered, subtract the smallest element value from the uncovered element, and add the smallest element value to the elements covered by two lines until the optimal solution is found, and output .

[0011] For each Add restrictions: ; In the formula, is the minimum speed, is the maximum speed, is the radar sampling time; Matching results that do not meet the restriction conditions are considered to be potential targets that are not properly associated and matched.

[0012] The ship target tracking method using unscented Kalman filter combined with LSTM includes using unscented Kalman filter to estimate the motion state, wherein the motion state includes the lateral position of the ship. , longitudinal position of the ship , the speed of the ship and the ship's heading angle ; set up The state vector of the ship at time : ; The motion of the ship is described by the following nonlinear state transfer equation: ; In the formula, yes The moment process noise, covariance is , is the discrete time step, yes The longitudinal position of the ship at the moment, yes The speed of the ship at that moment, yes The heading angle at the moment, yes Always turn to speed; Ship observation model for: ; ; In the formula, yes The noise is measured at every moment, and the covariance is , obtained by extracting the initial track from the time-accumulated spatial map , yes Observe the vector at any moment.

[0013] Using the unscented Kalman filter to estimate the motion state includes setting the initial ship state vector and the initial covariance matrix ,generate Point and Calculate Point weights, update predictions Point, update the gain parameters of the filter; For the dimension The state vector of indivual point: ; ; ; ; ; In the formula, It is the initial point, For the indivual point, is the scaling parameter, is a small positive number that controls the spread of the distribution, is the suboptimal weight; calculate Point weight: ; In the formula, yes The weight of renew Point, each Points are predicted through the state transfer function: ; In the formula, is the state transition function, It is updated point ; Weighted Average The predicted value of the point gets the predicted state: ; In the formula, is the predicted state; right The weighted covariance of the points is used to estimate the covariance of the predicted state: ; In the formula, is the covariance of the predicted states; Updating the filter gain parameters involves transforming Points are mapped into the measurement space: ; In the formula, yes The measured value of the point, is the observation transfer function; right The measured values ​​of the points are weighted averaged to obtain the predicted measured values : ; pass Weighted covariance of points to calculate the measurement covariance matrix : ; Compute the covariance matrix between states and measurements : ; Calculate Kalman gain using predicted measurement covariance and cross covariance : ; The predicted state and covariance are corrected by the Kalman gain: ; In the formula, is the actual measured value; The filter gain parameters are updated repeatedly until all measurements have been processed.

[0014] Unscented Kalman filter combined The ship target tracking method includes building a closed-loop tracking system with feedback: ; In the formula, is the output of the controller, , , are the proportional coefficient, integral coefficient and differential coefficient respectively, is the deviation, It’s time; Constructing an unscented Kalman filter combined with Closed-loop tracking system with feedback: ; In the formula, is the first one used to predict the state vector Neural network models, The second one is used to predict the difference between the true state vector and the predicted state vector Neural network models, is the second unscented Kalman filter that takes in the predicted observations and computes the predicted state vector, Is to use the first The observations of the neural network model and the ship state estimation vector generated by the second unscented Kalman filter, is the difference between the true state vector and the ship state estimate vector generated by the second unscented Kalman filter, It is before The Kalman gain set calculated by the frame target, It is before The difference between the actual state vector of the frame and the state vector predicted by the neural network.

[0015] Unscented Kalman filter combined The ship target tracking methods include: When the track association is successfully matched, use the observation target The observation vector As the first unscented Kalman filter Input, get Kalman gain and the optimal estimate , the same track past The time window data of length is input into Obtained Kalman gain ,Will As The input and output are The dimension settings are exactly the same ,Will The output result is Input, output ; If the track does not match a potential target that meets the conditional constraints in the current radar image frame, the observation vector of the track is interrupted and cannot be obtained. , According to the relationship between the filtering parameters and the observation vector, Input to And output ; make , select the past The difference in time steps As Input, output , Indicates the predicted error value; if there is no matching result for the target value, use as the predicted error value.

[0016] Compared with the prior art, the present invention has the following beneficial effects: the present invention can improve the radar performance based on the radar echo data without changing the underlying hardware, extract more potential targets, and optimize the local matching results; provide more and more accurate target information to the VTS system, so that it can have more early warning prior information to ensure safe passage through the main channel. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall flow chart of the present invention; Figure 2 This is an example of a radar original image; Figure 3 This is an example image after the radar performs the denoising process; Figure 4 It is the structure diagram of the target state estimation model; Figure 5 It is a schematic diagram of the tracking effect of an occluded target; Figure 6 This is a comparison chart between the tracking model effect and the real GPS. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] A ship target tracking method based on radar imaging in an obstructed environment, comprising: Step S1, using radar to scan the channel sea area, obtaining radar echo data and preprocessing the radar echo data, converting the radar echo data into an echo image; performing denoising on the echo image, separating the background and foreground targets, and obtaining an echo image with multiple foreground targets; converting the image into a grayscale image, using Gaussian filtering to remove Gaussian noise of the image, and using threshold processing to binarize the image to obtain a binary echo image; Step S2, obtaining multiple frames of echo images, repeating step S1, obtaining multiple frames of echo images in a time series, obtaining potential targets on each frame by using a ship image feature detection method, and accumulating and drawing multiple targets in multiple images on the same plane according to the time series to obtain a time-accumulated spatial map; obtaining potential targets on each frame by using a ship image feature detection method includes connected domain analysis, boundary tracking, chain code representation, hierarchical structure, contour approximation and feature acquisition; Step S3, extracting the initial track from the time-accumulated spatial map, assigning a tracking ID to each initial track; reading a frame of echo image, detecting the position of the potential target, and associating and matching the position of the potential target with the tracking ID; for the potential target that is normally associated and matched in a frame of radar echo data, extending the track corresponding to the potential target to obtain the latest position of the track; for the potential target that is not normally associated and matched, using the ship target tracking method of the unscented Kalman filter combined with the long short-term memory neural network model LSTM to update the position.

[0020] The preprocessing of radar echo data includes: ; ; ; In the formula, It's Radar The minimum echo unit is mapped to the image coordinate, It's Radar The minimum echo unit is mapped to the image coordinate, is the radial distance of the current echo unit from the radar, It is The azimuth of each echo, with the azimuth starting from due west as 0 and increasing in counterclockwise direction. is the width of the converted image. is the height of the converted image. is the current maximum scanning radial distance, The echo intensity Map to Functions in color space, is the maximum echo intensity. When the echo intensity is 0 Black, when the echo intensity is at its maximum Is red.

[0021] Using Gaussian filtering to remove Gaussian noise from an image involves constructing a Gaussian function for the image. : ; In the formula, the pixel to the left of the center pixel is is a negative value, the right pixel is a positive value, the pixel above the center pixel For negative values, the lower pixel is a positive value, is the standard deviation in the Gaussian filter, which determines the smoothness of the filter; Using threshold processing to binarize an image includes: ; In the formula, is the value of the pixel after segmentation, is the set binarization threshold, is the value of the pixel before segmentation.

[0022] Connected domain analysis includes using connected domain analysis to identify all connected foreground target regions in multiple frames of echo images in a time series. , if the following conditions are met, the pixels belong to the same connected domain: if , and there are adjacent points , so that ; Boundary tracking includes using connected domains to track boundaries and extracting the boundary point set of each connected domain. As a contour, for each connected domain , the corresponding contour is represented as a point set , Each point is a boundary point of the connected domain. for: ; ; The chain code representation includes using the chain code to represent the directions between adjacent points of the contour. and For two consecutive points of the contour, the relationship between the two consecutive points is expressed as a direction encoding: ; In the formula, for The coordinates of for The coordinates of each contour point, the moving direction of each contour point relative to the previous point is represented by a discrete direction value; The hierarchical structure includes defining the parent-child relationship between contours, constructing the hierarchical relationship of contours through topological analysis, and Contours The hierarchical tuple of , It is the outline The next contour at the same level, It is the outline The parent contour, It is the outline The first subcontour of It is the outline The previous contour of Contour approximation involves using polygon fitting to approximate the original contour with a set of points to ensure that the error is within the threshold. Suppose the approximate contour point set is ,and : ; In the formula, is the distance from each point to the fitted line, and The two endpoints of a straight line segment, is the error threshold; Obtaining features involves calculating the area features of potential targets using the polygon area formula based on the approximate contour point set. : ; In the formula, is the number of points in the contour point set, is the last point of the approximate contour point set. The next point after Defined as the first point of the approximate contour point set ,make , ; Center point of the contour for: ; .

[0023] Associating and matching the potential target's position with the tracking ID includes using a global nearest neighbor algorithm to calculate the optimal distance between the observation set and the target set to minimize the association cost function; Calculating the optimal distance includes reading a frame of image from the echo image, extracting potential target information, and setting the potential target as the observation set , the target set is , calculate each With each The Euclidean distance between , construct the distance matrix : ; Minimizing the associated cost function involves defining a global cost function Measure the matching between observation and target, and minimize the total matching distance: ; The global cost function includes subtracting the minimum value of each row of the distance matrix, retaining the value of 0 in each row, and subtracting the minimum value of each column of the distance matrix, retaining the value of 0 in each column, to obtain a new distance matrix; Use the least straight line to cover all zero elements in the matrix. Each row and column has at most one 0 value involved in the match. If found If the value 0 is matched, the optimal solution is found and the minimum ; If not found For 0 values, adjust the matrix to find the minimum number of lines covering all 0 value elements. If the minimum number of lines is equal to , then continue to look for the optimal solution. If the minimum number of lines is not equal to , then continue to adjust the matrix, find the smallest element value that is not covered, subtract the smallest element value from the uncovered element, and add the smallest element value to the elements covered by two lines until the optimal solution is found, and output .

[0024] For each Add restrictions: ; In the formula, is the minimum speed, is the maximum speed, is the radar sampling time; Matching results that do not meet the restriction conditions are considered to be potential targets that are not properly associated and matched.

[0025] The ship target tracking method using unscented Kalman filter combined with LSTM includes using unscented Kalman filter to estimate the motion state, wherein the motion state includes the lateral position of the ship. , longitudinal position of the ship , the speed of the ship and the ship's heading angle ; set up The state vector of the ship at time : ; The motion of the ship is described by the following nonlinear state transfer equation: ; In the formula, yes The moment process noise, covariance is , is the discrete time step, yes The longitudinal position of the ship at the moment, yes The speed of the ship at that moment, yes The heading angle at the moment, yes Always turn to speed; Ship observation model for: ; ; In the formula, yes The noise is measured at every moment, and the covariance is , obtained by extracting the initial track from the time-accumulated spatial map , yes Observe the vector at any moment.

[0026] Using the unscented Kalman filter to estimate the motion state includes setting the initial ship state vector and the initial covariance matrix ,generate Point and Calculate Point weights, update predictions Point, update the gain parameters of the filter; For the dimension The state vector of indivual point: ; ; ; ; ; In the formula, It is the initial point, For the indivual point, is the scaling parameter, is a small positive number that controls the spread of the distribution, is the suboptimal weight; calculate Point weight: ; In the formula, yes The weight of renew Point, each Points are predicted through the state transfer function: ; In the formula, is the state transition function, It is updated point ; Weighted Average The predicted value of the point gets the predicted state: ; In the formula, is the predicted state; right The weighted covariance of the points is used to estimate the covariance of the predicted state: ; In the formula, is the covariance of the predicted states; Updating the filter gain parameters involves transforming Points are mapped into the measurement space: ; In the formula, yes The measured value of the point, is the observation transfer function; right The measured values ​​of the points are weighted averaged to obtain the predicted measured values : ; pass Weighted covariance of points to calculate the measurement covariance matrix : ; Compute the covariance matrix between states and measurements : ; Calculate Kalman gain using predicted measurement covariance and cross covariance : ; The predicted state and covariance are corrected by the Kalman gain: ; In the formula, is the actual measured value; The filter gain parameters are updated repeatedly until all measurements have been processed.

[0027] Unscented Kalman filter combined The ship target tracking method includes building a closed-loop tracking system with feedback: ; In the formula, is the output of the controller, , , are the proportional coefficient, integral coefficient and differential coefficient respectively, is the deviation, It’s time; Constructing an unscented Kalman filter combined with Closed-loop tracking system with feedback: ; In the formula, is the first one used to predict the state vector Neural network models, The second one is used to predict the difference between the true state vector and the predicted state vector Neural network models, is the second unscented Kalman filter that takes in the predicted observations and computes the predicted state vector, Is to use the first The observations of the neural network model and the ship state estimation vector generated by the second unscented Kalman filter, is the difference between the true state vector and the ship state estimate vector generated by the second unscented Kalman filter, It is before The Kalman gain set calculated by the frame target, It is before The difference between the actual state vector of the frame and the state vector predicted by the neural network.

[0028] Unscented Kalman filter combined The ship target tracking methods include: When the track association is successfully matched, use the observation target The observation vector As the first unscented Kalman filter Input, get Kalman gain and the optimal estimate , the same track past The time window data of length is input into Obtained Kalman gain ,Will As The input and output are The dimension settings are exactly the same ,Will The output result is Input, output ; If the track does not match a potential target that meets the conditional constraints in the current radar image frame, the observation vector of the track is interrupted and cannot be obtained. , According to the relationship between the filtering parameters and the observation vector, Input to And output ; make , select the past The difference in time steps As Input, output , Indicates the predicted error value; if there is no matching result for the target value, use as the predicted error value.

[0029] The current problem that needs to be solved is that when tracking ship targets on radar images, when a large cargo ship or cruise ship meets a small ship at a misaligned position, the radial echo of the radar will be blocked by the large ship, and only the echo of the large ship will appear in the radar image. Depending on the speed difference between the two, the time for the echo of the small boat to disappear also varies. When the speed of both is slow, the radar echo pattern of the small boat will disappear for a long time. For the tracking model, the target is lost in the monitoring area, and when the target reappears, it will be identified as a new target. If a small ship with low reflectivity material appears, the target ship data will be intermittently missing due to the weak radar signal echo, which will cause the model to be unable to stably track the target, and will identify a segment of the same target's trajectory as multiple segments of multiple different targets.

[0030] When a large ship meets a small ship at sea, the small target is in an obscured environment for a long time, and the target radar echo disappears for a long time. Now a new tracking method is provided to stably track the target and complete the obscured information. The problem of track segmentation and frequent changes in target ID is solved. The present invention is suitable for the scenario where a small ship with low reflectivity appears intermittently in the radar echo image, resulting in the inability to track stably. The overall process of the present invention is as follows Figure 1 As shown, the following steps are included: Step S1: Aim the radar at the main detection area or main waterway according to the radial direction of the transmitting antenna, use the radar to scan the sea area, obtain radar echo data, pre-process the echo data, and convert the radar scanned data into image data, such as Figure 2 shown.

[0031] Step S2, perform denoising on the image, separate the background and foreground targets, and obtain an echo image with multiple targets. According to the latitude and longitude of the radar deployment and the scanning radius, draw a circle with the radar deployment location as the center and the maximum scanning range as the radius on the electronic nautical chart. Enclose the land background and the offshore facilities in the circle into a closed figure, find the intersection with the radar echo image, and take the intersecting echo area as the background area and delete it from the radar echo image. The result diagram is as follows: Figure 3 ; Step S3, pre-process the denoised image, convert it into a grayscale image, use Gaussian filtering to remove Gaussian noise from some images, and then use threshold processing technology to binarize the image to obtain a binary radar image. Step S4, obtain multiple frames of data, repeat steps S1 to S3, obtain multiple frames of radar echo images with time information, use ship image feature detection technology to obtain potential targets on each frame, and then accumulate multiple targets in multiple images according to the time series and draw them on the same plane. Step S5, extract the initial track from the time-accumulated spatial map, and assign a tracking ID to each track. Step S6, read a frame of radar data from the radar device, detect potential target positions, and associate these potential target positions with the previous track ID. On the basis of completing the initial track detection, multiple targets have now been obtained. The next step is to use the global nearest neighbor algorithm for data association, which is a multi-target data association algorithm that minimizes the association cost function by finding an optimal match between the observation set and the target set.

[0032] Step S7: For the target normally associated with the frame, it is only necessary to extend the corresponding track and use the target as the latest position information of the track. For the track that has not been updated, the proposed model is used to predict the position information of the target in this frame for update. Figure 5 As shown, the occluded target is predicted, and then the matching and prediction information of the frame is sent to the VTS system as a frame of target information, and this step is executed in a loop; Step S8: Use the unscented Kalman filter to perform an optimal state estimation on the tracked target. The state vector of the ship includes position, velocity and heading angle. Considering that the motion of the ship is affected by nonlinear factors (such as heading changes and environmental disturbances), the state of the ship is estimated by UKF.

[0033] Step S9: Use the LSTM neural network to input a sequence of Kalman gain prediction observations of multiple time steps and a sequence of differences between multiple state estimates and predicted state estimates to predict the error between the two. Pass the input sequence into the trained neural network; Step S10: Input the Kalman gain and state estimation vector of multiple time steps into the closed-loop tracking system with feedback. The following tracking system appears as a function and Represent the abstraction of the model of step S7 and step S8 respectively; Finally, to evaluate the prediction accuracy of the model, a high-precision GPS is installed on the ship observed by the radar, its position information is collected, and the radar is turned on to scan the ship. The output results of multiple tracking models are collected. The output results of the model and the results of the GPS are time-aligned, and finally the root mean square error (RMSE) is used to calculate the accuracy of the model: ; in and are the predicted coordinates, and is the real coordinate. The smaller the RMSE value, the more accurate the prediction. The RMSE predicted by the calculation model is less than 7. Considering that the length of the ship itself is about 10m and the GPS installation position is not at the centroid of the ship, the calculation result of the model is very accurate for small targets. The detection effect is visualized as follows Figure 6 ,According to the comparison between the GPS results and the results of the tracking model shown in the ,image, the trajectories of the two are almost completely consistent in the ,experiment, and the tracking model works well.

[0034] According to the latitude and longitude of the radar deployment and the scanning radius, the present invention draws a circle with the radar deployment position as the center and the maximum scanning range as the radius in the electronic nautical chart. The land background and the offshore facilities in the circle are enclosed into a closed figure, and the intersection is calculated with the radar echo image, and the intersecting echo area is used as the background area and deleted from the radar echo image.

[0035] After the present invention eliminates the land background and the echo of offshore facilities in the radar echo image, there is still some random Gaussian noise, and Gaussian filtering is used to filter this part of the noise. Before the Gaussian smoothing operation, the image needs to be converted into a grayscale image, each pixel point is distributed in the range of 0 to 255, and then the value of each pixel is recalculated through the Gaussian function. The core of Gaussian filtering lies in the Gaussian function, which is used to smooth the radar echo image, and finally the smoothed image is converted into a binary image, that is, there are only pixels of 0 or 1, and a threshold is set. If it is less than the threshold, it is a clutter background, otherwise it is an echo image of a related target.

[0036] After the above steps, the centroid positions of multiple potential targets, that is, the position information of the potential targets, are obtained. The potential target position information of multiple frames is repeatedly calculated, and then all the potential target information is accumulated in the same space to obtain the position information accumulation graph of multiple time series.

[0037] The present invention extracts effective tracks based on the time position information accumulation graph, converts potential target information into real target information, and assigns an independent ID to each track. The Hough transform method is used to distinguish the initial track, which is used to detect the track characteristics in the time series information accumulation space, mainly to search for low observable targets with linear motion and approximate linear motion in the time series information accumulation space. In the two-dimensional plane, the linear equation is expressed in polar coordinates: ; in represents the vertical distance from the straight line to the origin of the coordinate system. This method can be used to represent straight lines of all angles, including vertical lines (slope equals infinity). For each point in the cumulative graph, use the above formula in the parameter space. Corresponding to a curve, points on the same straight line will form an intersection in the parameter space. In order to detect straight lines and approximate straight lines in an image, the Hough transform uses an accumulator array to record the frequency of curve intersections. First, a two-dimensional accumulator array is set to represent and Then for each edge point in the image, calculate the corresponding value, and the corresponding The points are counted. When different points are on the same line, their curves in the accumulator space will intersect at the same point. The points with high scores in the accumulator correspond to the lines detected in the image. Finally, the lines that reach the threshold in the accumulator are marked as tracks and assigned IDs.

[0038] The present invention has obtained multiple targets based on the completion of the initial track detection. After solving the minimum global cost function, the present invention successfully assigns an observation value to each track, but the radar echo has noise or interference. and It may not be a true association. It is possible that the observation value comes from the nearest noise echo near the track. The unscented Kalman filter can be used to make an optimal estimate of the tracked target, but when the target is lost from the observation domain for a long time or is blocked by a large ship for a long time, the state estimate of the Kalman filter will have accumulated errors. This is because the Kalman filter cannot update the observation value. As the observation interruption time continues to increase, the state covariance of the Kalman filter will continue to increase, resulting in an increasing error between the estimated value and the true value. When the radar observes the lost target again, it is impossible or extremely difficult to associate the track, and the classic ID SWITCH problem will occur (the same ship is assigned different tracking IDs before and after the obstruction, and is mistaken for two different ships, but in fact it is the same ship observed in different time intervals).

[0039] In order to effectively avoid the classic ID SWITCH problem in the field, the present invention constructs a prediction model based on the unscented Kalman filter (UKF) and the long short-term memory neural network model (LTSM). Unscented Kalman Filter (UKF) is a recursive estimation method for nonlinear systems, which aims to solve the problem that the standard Kalman filter cannot handle nonlinear systems. During the voyage, the ship is affected by hydrodynamics, and the motion system is a nonlinear system, which is suitable for UKF to estimate its motion state and position information.

[0040] Now, by processing the radar echo data and UKF model, the motion state and position information of the ship can be estimated. However, there is a difference between the current state estimate and the true value. At the same time, UKF can also estimate the state at the next moment, so the difference between the observed value at the next moment and the state estimate at the next moment can be used to effectively express the error of the model estimation and to update the parameters of the Kalman gain and covariance matrix in UFK. Refer to the most widely used control algorithm in industrial control, PID (Proportional Integral Derivative), also known as the proportional, integral, and differential control model. It has been 70 years since its inception, with a simple structure, good stability, and reliable operation. It has become the main technology of industrial control. Its working principle is to continuously measure the deviation between the actual output value of the system and the set value, and calculate the control amount based on this deviation, so that the output value of the system tracks the set value.

[0041] According to the mathematical model, it can be found that the output value of the system is measured again and fed back to the controller to form a closed-loop control system. Referring to this closed-loop control system, a closed-loop tracking system with feedback can be designed using the estimated error of the Kalman filter model and the radar observation value. Although the PID control algorithm is simple, it requires engineers to have rich experience in modeling and adjusting parameters. In particular, the three hyperparameters of proportional, integral and differential coefficients need to be manually set. To improve this problem, neural networks are used to replace the manual setting of these control parameters, and a large amount of collected data is used to train the model to obtain the optimal parameters. Considering that the movement of the ship is a continuous time series, a long short-term memory neural network (Long Short-Term Memory) is used to model the model. The memory unit of the model can effectively consider the data information of multiple time steps, which is very consistent with the time series of the ship's movement.

[0042] In the process of training the network, back propagation and gradient descent are used as a means to find the optimal network weights. The forward propagation state is updated as follows: ; Hide status updates: ; Represents the Hadamard product (element-by-element multiplication), where two matrices of the same dimension are multiplied at corresponding positions. Represents the hyperbolic tangent function. In order to better express the complete closed-loop tracking system with feedback, the LSTM model is abstracted into a function.

[0043] This closed-loop tracking system with feedback can use the error between the real observation data and the state estimation data as system feedback, train the neural network based on the feedback, let the network learn the error law, and use the error prediction to correct the error of the state estimation, which can further optimize the tracking effect.

[0044] The target state estimation model structure is as follows: Figure 4 As shown, in the first step, Step 1, As the first unscented Kalman filter Input, get Kalman gain , and then input the first neural network LSTM1 to obtain Enter the second unscented Kalman filter In the second step, Step 2, Input LSTM1 to obtain , finally select the output according to the situation , or .

[0045] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A ship target tracking method based on radar imaging in an obstructed environment, characterized in that: include: Step S1, using radar to scan the channel sea area, obtaining radar echo data and preprocessing the radar echo data, and converting the radar echo data into an echo image; De-noising the echo image to separate the background and foreground targets, and obtain an echo image with multiple foreground targets; The image is converted into a grayscale image, the Gaussian noise of the image is removed by Gaussian filtering, and the image is binarized by threshold processing to obtain a binary echo image; Step S2, obtaining multiple frames of echo images, repeating step S1, obtaining multiple frames of echo images in a time series, obtaining potential targets on each frame by using a ship image feature detection method, and accumulating and drawing multiple targets in multiple images on the same plane according to the time series to obtain a time-accumulated spatial map; obtaining potential targets on each frame by using a ship image feature detection method includes connected domain analysis, boundary tracking, chain code representation, hierarchical structure, contour approximation and feature acquisition; Step S3, extracting the initial track from the time-accumulated spatial map, assigning a tracking ID to each initial track; reading a frame of echo image, detecting the position of the potential target, and associating and matching the position of the potential target with the tracking ID; for the potential target that is normally associated and matched in a frame of radar echo data, extending the track corresponding to the potential target to obtain the latest position of the track; For potential targets that are not normally associated and matched, the ship target tracking method using unscented Kalman filtering combined with long short-term memory neural network model LSTM is used to update the position.

2. The ship target tracking method based on radar imaging in an obstructed environment according to claim 1 is characterized in that: The preprocessing of radar echo data includes: ； ； ； In the formula, It's Radar The minimum echo unit is mapped to the image coordinate, It's Radar The minimum echo unit is mapped to the image coordinate, is the radial distance of the current echo unit from the radar, It is The azimuth of each echo, with the azimuth starting from due west as 0 and increasing in counterclockwise direction. is the width of the converted image. is the height of the converted image. is the current maximum scanning radial distance, The echo intensity Map to Functions in color space, is the maximum echo intensity. When the echo intensity is 0 Black, when the echo intensity is at its maximum Is red.

3. The ship target tracking method based on radar imaging in an obstructed environment according to claim 2, characterized in that: Using Gaussian filtering to remove Gaussian noise from an image involves constructing a Gaussian function for the image. : ； In the formula, the pixel to the left of the center pixel is is a negative value, the right pixel is a positive value, the pixel above the center pixel For negative values, the lower pixel is a positive value, is the standard deviation in the Gaussian filter, which determines the smoothness of the filter; Using threshold processing to binarize an image includes: ； In the formula, is the value of the pixel after segmentation, is the set binarization threshold, is the value of the pixel before segmentation.

4. The ship target tracking method based on radar imaging in an obstructed environment according to claim 3 is characterized in that: Connected domain analysis includes using connected domain analysis to identify all connected foreground target regions in multiple frames of echo images in a time series. , if the following conditions are met, the pixels belong to the same connected domain: if , and there are adjacent points , so that ; Boundary tracking includes using connected domains to track boundaries and extracting the boundary point set of each connected domain. As a contour, for each connected domain , the corresponding contour is represented as a point set , Each point is a boundary point of the connected domain. for: ； ； The chain code representation includes using the chain code to represent the directions between adjacent points of the contour. and For two consecutive points of the contour, the relationship between the two consecutive points is expressed as a direction encoding: ； In the formula, for The coordinates of for The coordinates of each contour point, the moving direction of each contour point relative to the previous point is represented by a discrete direction value; The hierarchical structure includes defining the parent-child relationship between contours, constructing the hierarchical relationship of contours through topological analysis, and Contours The hierarchical tuple of , It is the outline The next contour at the same level, It is the outline The parent contour, It is the outline The first subcontour of It is the outline The previous contour of Contour approximation involves using polygon fitting to approximate the original contour with a set of points to ensure that the error is within the threshold. Suppose the approximate contour point set is ,and : ； In the formula, is the distance from each point to the fitted line, and The two endpoints of a straight line segment, is the error threshold; Obtaining features involves calculating the area features of potential targets using the polygon area formula based on the approximate contour point set. : ； In the formula, is the number of points in the contour point set, is the last point of the approximate contour point set. The next point after Defined as the first point of the approximate contour point set ,make , ; Center point of the contour for: ； 。 5. The ship target tracking method based on radar imaging in an obstructed environment according to claim 4, characterized in that: Associating and matching the potential target's position with the tracking ID includes using a global nearest neighbor algorithm to calculate the optimal distance between the observation set and the target set to minimize the association cost function; Calculating the optimal distance includes reading a frame of image from the echo image, extracting potential target information, and setting the potential target as the observation set , the target set is , calculate each With each The Euclidean distance between , construct the distance matrix : ； Minimizing the associated cost function involves defining a global cost function Measure the matching between observation and target, and minimize the total matching distance: ； The global cost function includes subtracting the minimum value of each row of the distance matrix, retaining the value of 0 in each row, and subtracting the minimum value of each column of the distance matrix, retaining the value of 0 in each column, to obtain a new distance matrix; Use the least straight line to cover all zero elements in the matrix. Each row and column has at most one 0 value involved in the match. If found If the value 0 is matched, the optimal solution is found and the minimum ; If not found For 0 values, adjust the matrix to find the minimum number of lines covering all 0 value elements. If the minimum number of lines is equal to , then continue to look for the optimal solution. If the minimum number of lines is not equal to , then continue to adjust the matrix, find the smallest element value that is not covered, subtract the smallest element value from the uncovered element, and add the smallest element value to the elements covered by two lines until the optimal solution is found, and output .

6. The ship target tracking method based on radar imaging in an obstructed environment according to claim 5, characterized in that: For each Add restrictions: ； In the formula, is the minimum speed, is the maximum speed, is the radar sampling time; Matching results that do not meet the restriction conditions are considered to be potential targets that are not properly associated and matched.

7. The method for tracking a ship target based on radar imaging in an obstructed environment according to claim 6, characterized in that: The ship target tracking method using unscented Kalman filter combined with LSTM includes using unscented Kalman filter to estimate the motion state, wherein the motion state includes the lateral position of the ship. , longitudinal position of the ship , the speed of the ship and the ship's heading angle ; set up The state vector of the ship at time : ； The motion of the ship is described by the following nonlinear state transfer equation: ； In the formula, yes The moment process noise, covariance is , is the discrete time step, yes The longitudinal position of the ship at the moment, yes The speed of the ship at that moment, yes The heading angle at the moment, yes Always turn to speed; Ship observation model for: ； ； In the formula, yes The noise is measured at every moment, and the covariance is , obtained by extracting the initial track from the time-accumulated spatial map , yes Observe the vector at any moment.

8. The ship target tracking method based on radar imaging in an obstructed environment according to claim 7, characterized in that: Using the unscented Kalman filter to estimate the motion state includes setting the initial ship state vector and the initial covariance matrix ,generate Point and Calculate Point weights, update predictions Point, update the gain parameters of the filter; For the dimension The state vector of indivual point: ； ； ； ； ； In the formula, It is the initial point, For the indivual point, is the scaling parameter, is a small positive number that controls the spread of the distribution, is the suboptimal weight; calculate Point weight: ； In the formula, yes The weight of renew Point, each Points are predicted through the state transfer function: ； In the formula, is the state transition function, It is updated point ; Weighted Average The predicted value of the point gets the predicted state: ； In the formula, is the predicted state; right The weighted covariance of the points is used to estimate the covariance of the predicted state: ； In the formula, is the covariance of the predicted states; Updating the filter gain parameters involves transforming Points are mapped into the measurement space: ； In the formula, yes The measured value of the point, is the observation transfer function; right The measured values ​​of the points are weighted averaged to obtain the predicted measured values : ； pass Weighted covariance of points to calculate the measurement covariance matrix : ； Compute the covariance matrix between states and measurements : ； Calculate Kalman gain using predicted measurement covariance and cross covariance : ； The predicted state and covariance are corrected by the Kalman gain: ； In the formula, is the actual measured value; The filter gain parameters are updated repeatedly until all measurements have been processed.

9. The method for tracking a ship target based on radar imaging in an obstructed environment according to claim 8, characterized in that: Unscented Kalman filter combined The ship target tracking method includes building a closed-loop tracking system with feedback: ； In the formula, is the output of the controller, , , are the proportional coefficient, integral coefficient and differential coefficient respectively, is the deviation, It’s time; Constructing an unscented Kalman filter combined with Closed-loop tracking system with feedback: ； In the formula, is the first one used to predict the state vector Neural network models, The second one is used to predict the difference between the true state vector and the predicted state vector Neural network models, is the second unscented Kalman filter that takes in the predicted observations and computes the predicted state vector, Is to use the first The observations of the neural network model and the ship state estimation vector generated by the second unscented Kalman filter, is the difference between the true state vector and the ship state estimate vector generated by the second unscented Kalman filter, It is before The Kalman gain set calculated by the frame target, It is before The difference between the actual state vector of the frame and the state vector predicted by the neural network.

10. The ship target tracking method based on radar imaging in an obstructed environment according to claim 9, characterized in that: Unscented Kalman filter combined The ship target tracking methods include: When the track association is successfully matched, use the observation target The observation vector As the first unscented Kalman filter Input, get Kalman gain and the optimal estimate , the same track past The time window data of length is input into Obtained Kalman gain ,Will As The input and output are The dimension settings are exactly the same ,Will The output result is Input, output ; If the track does not match a potential target that meets the conditional constraints in the current radar image frame, the observation vector of the track is interrupted and cannot be obtained. , According to the relationship between the filtering parameters and the observation vector, Input to And output ; make , select the past The difference in time steps As Input, output , Indicates the predicted error value; if there is no matching result for the target value, use as the predicted error value.

Images