A target tracking method and device for an escorted ship in autonomous escort operation

Abstract

The present invention provides a target tracking method and device for an escorted vessel during an autonomous escort operation, relating to the field of target tracking, including: S1: The escort vessel obtains the AIS data of the escorted vessel through the escorted vessel's MMSI data. If the acquisition is successful, the method proceeds to step S2; otherwise, the escort state is prohibited; S2: The escort vessel tracks the target through the AIS; if the AIS data is lost, the method returns to S1; if the AIS data successfully matches the TTM data, the method proceeds to step S3; otherwise, step S2 is repeated; S3: The escort vessel tracks the target through the fusion of the AIS and the K-band radar; if the K-band radar loses the target, the method returns to step S2; if the visual recognition is successful, the method proceeds to step S4; otherwise, step S3 is repeated; S4: The escort vessel tracks the escorted vessel through a visual camera and a laser radar; if the visual camera loses the target, the method returns to step S3. The present invention adopts the most accurate corresponding method for target tracking at different stages, ensuring that high-precision target tracking can be achieved throughout the escort process.

Description

Technical Field

[0001] The present invention relates to the field of target tracking, and in particular to a target tracking method and device for an escorted ship in an autonomous escort operation. Background Art

[0002] Autonomous tugboat escort refers to the use of intelligent perception and control systems to enable tugboats to automatically follow and guide other large ships without human intervention. To this end, tugboats are equipped with a variety of environmental perception sensors, such as navigation radar, lidar, and cameras, which are used to monitor and identify surrounding ships in real time, thereby providing necessary input for tugboat control. During the escort process, the tugboat needs to determine its relative position to the escorted ship and adjust its own navigation path according to its motion state to ensure that it is always in the appropriate position to achieve effective escort operations. Since the position and motion state of the escorted ship may change at any time, the role of the target tracking function is to continuously monitor the position and speed of the escorted ship in order to keep abreast of its current status and changing trends.

[0003] Although existing ship target tracking technology is becoming increasingly mature, practical applications still face several challenges. First, AIS can continuously acquire relevant information about escorted vessels, has a long range, and lacks the blind spots associated with radar. However, the infrequent update frequency of AIS can hinder timely assessment of target situation. Second, shipborne navigation radars have a long detection range, and their ARPAs can automatically identify and track surrounding vessels and other objects, outputting their real-time position, speed, and heading. However, these identified targets typically lack identification information, making it impossible to accurately track specific vessels based on their numbers in real time. Furthermore, radar's operating characteristics result in large measurement errors at close ranges, making it difficult to effectively track ship features (such as thrust markers). Finally, visual cameras are suitable for locking onto and tracking ships and their features at close range, but they cannot directly output the coordinates of the tracked target and require integration with lidar data. Therefore, in increasingly busy port environments, a single sensor is no longer sufficient to accurately capture target dynamics, requiring a combination of multiple sensors to achieve information fusion and relay tracking. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a target tracking method and equipment for the escorted ship in autonomous escort operations, so as to solve the problem that the existing escort has low accuracy and is difficult to apply to the entire escort process.

[0005] The present invention provides a target tracking method for an escorted ship in an autonomous escort operation, comprising the steps of:

[0006] S1: The accompanying vessel obtains the AIS data of the accompanied vessel through the MMSI data of the accompanied vessel. If the acquisition is successful, the process proceeds to step S2; otherwise, the accompanying vessel is prohibited from entering the accompanying state.

[0007] S2: In this stage, the escort vessel tracks the target through AIS and obtains the TTM data of the escorted vessel through K-band radar. If the AIS data is lost, the process returns to S1. If the AIS data matches the TTM data, the process proceeds to step S3. Otherwise, step S2 is repeated.

[0008] S3: In this stage, the accompanying ship tracks the target through the fusion of AIS and K-band radar, and controls the visual camera to visually identify the accompanied ship. If the K-band radar loses the target, the process returns to step S2. If the visual recognition is successful, the process proceeds to step S4. Otherwise, step S3 is repeated.

[0009] S4: The accompanying ship tracks the accompanied ship through the visual camera and laser radar. If the visual camera loses the target, it returns to step S3.

[0010] Preferably, in step S2, the escort vessel performs target tracking via AIS as follows:

[0011] S211: The escorting ship obtains the AIS data received at the most recent time t0 from time t, and sets the state of the escorting ship at time t0 t is the current moment;

[0012] S212: Construct a state transition model to convert the state Substitute into the state transfer model and calculate the predicted state of the escorted ship at time t

[0013] S213: Obtain the Kalman gain by calculating the error covariance matrix, construct the observation vector, and predict the state by the Kalman gain and the observation vector. Update and get the updated status

[0014] S214: Updated status The escorted ship is tracked, the error covariance matrix is ​​updated, and then the process returns to step S211.

[0015] Preferred:

[0016] state

[0017] Prediction Status

[0018]

[0019] Among them, * represents the predicted value, is the predicted AIS longitude at time t, is the predicted AIS latitude at time t, is the AIS longitude at time t0, is the AIS latitude at time t0, CRS AIS is the AIS heading, Vel AIS AIS speed, 1.141×10 -5 and 8.99×10 -6 Scaling factors for converting longitude and latitude to meters respectively.

[0020] Preferably, in step S2, the process of matching AIS data with TTM data is specifically as follows:

[0021] S221: AIS data acquired by AIS includes the AIS longitude Lng of the escorted vessel AIS 、AIS latitude Lat AIS 、AIS heading CRS AIS and AIS speed Vel AIS ; The TTM data obtained by the K-band radar includes the K-wave distance d of the escorted ship K 、K wave azimuth θ K , K-wave heading CRS K and K wave speed Vel K ;

[0022] S222: Time-synchronize the AIS data and the TTM data by interpolation, convert the TTM data and the AIS data into the same coordinate system, and obtain the target K-wave radar data and AIS data at the same time in the same coordinate system;

[0023] S223: Calculate the total weighted Euclidean distance between the AIS track and the TTM track using the target K-wave radar data and the AIS data;

[0024] S224: obtaining the similarity between the AIS trajectory and the TTM trajectory by calculating the total weighted Euclidean distance;

[0025] S225: If the similarity is greater than or equal to the preset value, it is determined that the AIS data and the TTM data are successfully matched, and the process proceeds to step S3; otherwise, the process returns to step S222.

[0026] Preferred:

[0027] The weighted Euclidean distance d between the i-th TTM track and the j-th AIS track at time t (j，j，t) The calculation formula is:

[0028]

[0029] Among them, w d and wv are the weights of position and velocity, respectively. and are the coordinates of the track points of TTM track and AIS track at time t, and are the velocity vectors of the track points of TTM track and AIS track at time t respectively;

[0030] The total weighted Euclidean distance D between the i-th TTM track and the j-th AIS track (i，j) The calculation formula is:

[0031]

[0032] The calculation formula for the similarity ρ between the i-th TTM trajectory and the j-th AIS trajectory is:

[0033]

[0034] Where n is the number of trajectory points involved in the matching calculation, and λ is the regularization parameter.

[0035] Preferably, in step S3, the escort ship performs target tracking by fusing AIS and K-band radar as follows:

[0036] S31: After the AIS data and TTM data are successfully matched, the AIS data and TTM data are acquired in real time, and the expected value of the TTM data at the current moment is calculated. If the TTM data at the next moment has the same number as the expected value and the Euclidean distance is less than a preset threshold, the TTM data at the next moment is used for weighted average fusion, and the process proceeds to step S32. If no TTM data at the next moment with the same number is found within the threshold range, the TTM data with the closest Euclidean distance to the expected value is used for weighted average fusion, and the process proceeds to step S32. If there is no new TTM data within the threshold range, it is determined that the target is lost, and the process returns to step S2.

[0037] S32: Perform weighted average fusion on the AIS data and the TTM data to obtain fused target data Data;

[0038] When receiving AIS data, the calculation formula of Data is:

[0039]

[0040] Among them, Data K For TTM data, Data AIS is AIS data, w K is the TTM data weight, w AIS is the AIS data weight;

[0041] When no AIS data is received, the interpolated AIS data weight w is used AIS (t)=w0·e -λΔt Calculate Data;

[0042] Among them, w AIS (t) is the interpolated AIS data weight at time t, w0 is the initial weight, λ is the decay constant, and Δt is the time difference of receiving AIS data;

[0043] S33: Target tracking is performed using the fused target data.

[0044] Preferably, in step S4, target tracking of the escorted vessel by using a visual camera and a laser radar is specifically performed as follows:

[0045] S41: Obtain the actual position of the target detection frame at the current moment through the visual camera and laser radar, and obtain the center pixel of the detection frame;

[0046] S42: Calculate the pixel difference between the center pixel of the detection frame and the center pixel of the screen, and control the gimbal based on the pixel difference to place the target at the center of the screen;

[0047] S43: Predicting the predicted position of the target detection frame at the next moment based on the actual position of the target detection frame at the current moment, and obtaining the actual position of the target detection frame at the next moment; calculating the position difference between the predicted position and the actual position of the target detection frame at the next moment, and the area difference of the target detection frame;

[0048] S44: If both the position difference and the area difference are smaller than the corresponding preset values, the target is determined to be matched successfully, and the target is tracked through the target detection frame, and the process returns to step S41; otherwise, the process returns to step S43.

[0049] A storage medium stores instructions and data for implementing a target tracking method for an escorted ship in an autonomous escort operation.

[0050] A target tracking device for an escorted ship in an autonomous escort operation comprises: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the target tracking method for the escorted ship in the autonomous escort operation.

[0051] The present invention has the following beneficial effects:

[0052] At long distances, target tracking is performed through AIS; at medium distances, target tracking is performed through the fusion of AIS and K-band radar; at close ranges, target tracking of the escorted ship is performed through visual cameras and lidar; target tracking is performed using the most accurate corresponding method at different stages, ensuring high-precision target tracking throughout the entire escort process, and the fusion and coordination of multiple sensor data improves the accuracy and stability of ship escort target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0054] Figure 2 Flowchart for target tracking via AIS for escort vessels;

[0055] Figure 3 Flowchart for target tracking by escort vessel through fusion of AIS and K-band radar;

[0056] Figure 4 This is a flow chart for tracking the escorted vessel using a visual camera and lidar;

[0057] Figure 5 This is a structural diagram of the device according to an embodiment of the present invention;

[0058] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0060] Reference Figure 1 The present invention provides a target tracking method for an escorted ship in an autonomous escort operation, comprising the steps of:

[0061] S1: The accompanying vessel obtains the AIS data of the accompanied vessel through the MMSI data of the accompanied vessel. If the acquisition is successful, the process proceeds to step S2; otherwise, the accompanying vessel is prohibited from entering the accompanying state.

[0062] Specifically, the Automatic Identification System (AIS) is a technology used to automatically exchange position and navigation information between ships. It collects data such as the ship's position, speed, and heading through satellites and ground-based base stations, and broadcasts this information to surrounding ships and relevant shore-based agencies. The AIS system typically consists of an AIS transmitter and an AIS receiver. The escorted ship uses the transmitter to send ship information to surrounding ships and shore-based base stations, while the escort ship's receiver receives and decodes the information sent by other ships.

[0063] Receive AIS data from nearby ships and filter out the AIS data of the escorted ship by the MMSI number of the escorted ship, which contains the latitude and longitude, speed and heading of the escorted ship, and put the AIS data of the ship into the queue.

[0064] The escort vessel receives AIS information from all nearby vessels, so it is necessary to filter out the vessel being escorted. The MMSI, short for Maritime Mobile Service Identity, is a nine-digit code used for automatic identification and tracking of vessels. Received AIS information will include the vessel's MMSI. Since the escort vessel's MMSI is known upon receiving the escort mission, it can be used to filter AIS data and obtain its data.

[0065] S2: In this stage, the escort vessel tracks the target through AIS and obtains the TTM data of the escorted vessel through K-band radar. If the AIS data is lost, the process returns to S1. If the AIS data matches the TTM data, the process proceeds to step S3. Otherwise, step S2 is repeated.

[0066] Furthermore, due to the limitations of shipborne radar functions and performance, at long distances beyond 1.5 km, only AIS can select a specific ship and reliably output its information;

[0067] refer to Figure 2 In step S2, the escort ship tracks the target through AIS as follows:

[0068] S211: The escorting ship obtains the AIS data received at the most recent time t0 from time t, and sets the state of the escorting ship at time t0 t is the current moment;

[0069] Next, we define states and observations. The state variable is s = {Lng, Lat, CRS, Vel}, representing the ship's longitude, latitude, speed, and heading, respectively. The observation variables are directly parsed from the AIS data.

[0070] state

[0071] Prediction Status

[0072]

[0073] Among them, * represents the predicted value, is the predicted AIS longitude at time t, is the predicted AIS latitude at time t, is the AIS longitude at time t0, is the AIS latitude at time t0, CRS AIS is the AIS heading, Vel AIS AIS speed, 1.141×10 -5 and 8.99×10 -6 Scaling factors for converting longitude and latitude to meters respectively.

[0074] S212: Construct a state transition model to convert the state Substitute into the state transfer model and calculate the predicted state of the escorted ship at time t

[0075] Specifically, a state transition model is defined. The tugboat escorts a large container ship or oil and gas tanker. The ship does not perform large maneuvers within the port, and its speed and heading remain nearly constant over short periods of time. The ship's motion can be assumed to be uniform and straight.

[0076] S213: Obtain the Kalman gain by calculating the error covariance matrix, construct the observation vector, and predict the state by the Kalman gain and the observation vector. Update and get the updated status

[0077] Specifically, the first received AIS data from the escorted vessel is set as the initial state. At the same time, an initial error covariance matrix is ​​set to represent the uncertainty of the initial state estimate. The initial error covariance matrix P0 will be a 4×4 matrix, and the specific number of variances is adjusted according to the actual initial state.

[0078]

[0079] Since there are no new observations at the current moment, the update step will rely on the results of the prediction step. However, if new AIS data is received, the new data is used to update the prediction. The new observation data is compared with the prediction data and the estimated position, velocity and heading are adjusted accordingly. The update process is:

[0080] First calculate the Kalman gain:

[0081] K=P t|t-1 H T (HP t|t-1 H T +R) -1

[0082] In the above formula, P t|t-1 is the prediction error covariance matrix, H is the observation model matrix, and R is the observation noise covariance matrix.

[0083] Then construct the observation vector as:

[0084]

[0085] Last updated state estimate:

[0086]

[0087] In the above formula, is the predicted state estimate, z t is the actual observation data, is the updated state estimate.

[0088] S214: Updated status The escorted ship is tracked, the error covariance matrix is ​​updated, and then the process returns to step S211.

[0089] Specifically, the error covariance matrix is ​​adjusted according to the results of the prediction or update step.

[0090] P t =(I-KH)P t|t-1

[0091] In the above formula, I is the unit matrix, P t is the updated covariance matrix;

[0092] Output Kalman filter gives the current position, velocity and heading estimate.

[0093] Furthermore, in step S2, the process of matching AIS data with TTM data is as follows:

[0094] S221: AIS data acquired by AIS includes the AIS longitude Lng of the escorted vessel AIS 、AIS latitude Lat AIS 、AIS heading CRS AIS and AIS speed Vel AIS ; The TTM data obtained by the K-band radar includes the K-wave distance d of the escorted ship K 、K wave azimuth θ K , K-wave heading CRS K and K wave speed Vel K ;

[0095] S222: Time-synchronize the AIS data and the TTM data by interpolation, convert the TTM data and the AIS data into the same coordinate system, and obtain the target K-wave radar data and AIS data at the same time in the same coordinate system;

[0096] Furthermore, since the data to be matched is the TTM data, the timestamp of the TTM data is used to perform linear interpolation on each item of the AIS data, GPS data, and compass data to obtain the sensor interpolation data corresponding to the TTM data timestamp to achieve time synchronization. The interpolation method is:

[0097]

[0098] In the above formula, s(t1) is the data at time t1, s(t2) is the data at time t2, and s(t) is the interpolated data at time t.

[0099] The escorted ship data in the AIS data are latitude and longitude, speed over ground and course over ground. In the north-east coordinate system, the x and y coordinates and velocity components of the AIS target are:

[0100]

[0101] v AIS_x =Vel AIS ·cos CRS AIS

[0102] v AIS_y =Vel AIS sinCRS AIS

[0103] In the above formula, Lng ship and Lat ship are the longitude and latitude of the escort ship, and R is the radius of the earth.

[0104] The target K-wave radar data includes the x-axis coordinate x of the escorted ship. K , y-axis coordinate y K , x-axis velocity vector v K_x and the y-axis velocity vector v K_y , the calculation formula is:

[0105] x K =d K ·cos(θ K +ψ)

[0106] y K =d K ·sin(θ K +ψ)

[0107] v K_x =Vel ship cosCRS ship +Vel K ·cos(CRS K +ψ)

[0108] v K_y=Vel ship sinCRS ship +Vel K sin(CRS K +ψ)

[0109] Among them, Vel ship is the speed of the accompanying ship, CRS ship The heading of the accompanying vessel.

[0110] S223: Calculate the total weighted Euclidean distance between the AIS track and the TTM track using the target K-wave radar data and the AIS data;

[0111] Furthermore, the weighted Euclidean distance d between the i-th TTM trajectory and the j-th AIS trajectory at time t is (i，j，t) The calculation formula is:

[0112]

[0113] Among them, w d and w v are the weights of position and velocity, respectively. and are the coordinates of the track points of TTM track and AIS track at time t, and are the velocity vectors of the track points of TTM track and AIS track at time t respectively;

[0114] The total weighted Euclidean distance D between the i-th TTM track and the j-th AIS track (i，j) The calculation formula is:

[0115]

[0116] S224: obtaining the similarity between the AIS trajectory and the TTM trajectory by calculating the total weighted Euclidean distance;

[0117] Specifically, the calculation formula for the similarity ρ between the i-th TTM trajectory and the j-th AIS trajectory is:

[0118]

[0119] Where n is the number of trajectory points involved in the matching calculation, and λ is the regularization parameter.

[0120] S225: If the similarity is greater than or equal to the preset value, it is determined that the AIS data and the TTM data are successfully matched, and the process proceeds to step S3; otherwise, the process returns to step S222.

[0121] Specifically, if the track similarity index ρ ≥ 0.85, the i-th TTM track is considered to be successfully associated with the j-th AIS track, representing the same target. By matching RS24 radar data with AIS data, static information of a specified radar target can be obtained.

[0122] S3: In this stage, the accompanying ship tracks the target through the fusion of AIS and K-band radar, and controls the visual camera to visually identify the accompanied ship. If the K-band radar loses the target, the process returns to step S2. If the visual recognition is successful, the process proceeds to step S4. Otherwise, step S3 is repeated.

[0123] Furthermore, when the range reaches 1.5km, AIS and K-band radar are used as the input sources of target information within 1.5km. The K-band radar can detect objects with a diameter greater than 0.75m within 1.5km. At the same time, it can continuously track the detected targets and output them in the TTM sentence format of the NMEA-0183 protocol. The format is as follows:

[0124] $-TTM,xx,xx,xx,a,xx,xx,a,xx,xx,a,cc,a,ahhmmss.ss,a*hh <cr> <lf>

[0125] Where $-TTM indicates that the data is the TTM statement in the MMNEA-0183 protocol. The first xx indicates the target number, the first xx indicates the relative distance, xx a indicates the azimuth, xx indicates the speed, and xx,a indicates the heading.

[0126] refer to Figure 3 In step S3, the escort ship tracks the target by integrating AIS and K-band radar as follows:

[0127] S31: After the AIS data and TTM data are successfully matched, the AIS data and TTM data are acquired in real time, and the expected value of the TTM data at the current moment is calculated. If the TTM data at the next moment has the same number as the expected value and the Euclidean distance is less than a preset threshold, the TTM data at the next moment is used for weighted average fusion, and the process proceeds to step S32. If no TTM data at the next moment with the same number is found within the threshold range, the TTM data with the closest Euclidean distance to the expected value is used for weighted average fusion, and the process proceeds to step S32. If there is no new TTM data within the threshold range, it is determined that the target is lost, and the process returns to step S2.

[0128] Specifically, the target captured by the K-band radar may be lost, resulting in a change in the TTM data number of the same target. After successfully matching the TTM data and obtaining the escorted ship number, if only TTM data with the same number is received, the data of the existing TTM queue with the same number is linearly extrapolated to obtain the expected TTM data value at the current TTM data moment:

[0129]

[0130] If the weighted Euclidean distance between the expected value coordinate and the actual received TTM data is less than a certain threshold, it is considered that the ship's number has not changed, and the data is added to the queue and the target's trajectory is updated. The judgment conditions are:

[0131]

[0132] In the above formula, (x k ,y k ,v K ,CRS k ) is the received TTM status, is the predicted TTM state, w1 and w2 are weights, and D is the preset distance threshold

[0133] If the match fails, the K-band radar is considered lost when plotted by the accompanying vessel, and the target ID has changed when it is reacquired. The system then searches for the TTM data with the closest Euclidean distance to the expected value. If the distance is less than a preset threshold, the target ID in that data is considered the changed target, and the target ID in the queue is updated. If the target is not found, tracking is considered lost, and the AIS and K-band radar data are re-fused.

[0134] S32: Perform weighted average fusion on the AIS data and the TTM data to obtain fused target data Data;

[0135] When receiving AIS data, the calculation formula of Data is:

[0136]

[0137] Among them, Data K For TTM data, Data AIS is AIS data, w K is the TTM data weight, w aIS is the AIS data weight;

[0138] When no AIS data is received, the interpolated AIS data weight w is used AIS (t)=w0·e -λΔt Calculate Data;

[0139] Among them, w AIS (t) is the interpolated AIS data weight at time t, w0 is the initial weight, λ is the decay constant, and Δt is the time difference of receiving AIS data;

[0140] S33: Target tracking is performed using the fused target data.

[0141] S4: The accompanying ship tracks the accompanied ship through the visual camera and laser radar. If the visual camera loses the target, it returns to step S3.

[0142] Furthermore, when tracking targets at close range, visual cameras and lidar are used to complete target tracking;

[0143] refer to Figure 4 In step S4, the target tracking of the escorted ship is performed by using the visual camera and the laser radar as follows:

[0144] S41: Obtain the actual position of the target detection frame at the current moment through the visual camera and laser radar, and obtain the center pixel of the detection frame;

[0145] Specifically, when the camera on the gimbal identifies a ship target under the guidance of the AIS and K-band radar, it obtains the detection frame coordinates and boundaries of all targets. The image coordinate system has the x-axis pointing right and the y-axis pointing downward. Assuming the image pixel resolution is H×V, the maximum x value is H and the maximum y value is V. In the image coordinate system, the left and right boundaries of the detection frame are represented by the x-coordinates of the pixel coordinates, L and R respectively, and the upper and lower boundaries of the detection frame are represented by the y-coordinates of the pixel coordinates, T and B respectively. The pixel coordinates of the center of the detection frame are:

[0146]

[0147] S42: Calculate the pixel difference between the center pixel of the detection frame and the center pixel of the screen, and control the gimbal based on the pixel difference to place the target at the center of the screen;

[0148] Specifically, the target closest to the center of the screen is selected for tracking. The pixel difference is calculated as follows:

[0149]

[0150] The gimbal rotation angle is calculated by pixel difference and camera angular resolution, and the gimbal is controlled to place the target at the center of the screen. The category, detection frame coordinates, and boundaries of the target at the center of the screen after rotation are placed in a queue. The gimbal rotation angle calculation method is:

[0151]

[0152] In the above formula, p and t are the horizontal and vertical rotation angles of the gimbal, and HFOV and VFOV are the horizontal and vertical field of view angles of the camera, respectively.

[0153] S43: Predicting the predicted position of the target detection frame at the next moment based on the actual position of the target detection frame at the current moment, and obtaining the actual position of the target detection frame at the next moment; calculating the position difference between the predicted position and the actual position of the target detection frame at the next moment, and the area difference of the target detection frame;

[0154] Specifically, the camera frame rate is 10 Hz, that is, one detection image is obtained every 0.1 seconds. The position of the detection frame after 0.1 seconds is predicted by linear extrapolation using the target detection frame information in the queue (v ′ ,h ′ ) and boundary (L ′ ,R ′ ,T ′ ,B ′ );

[0155] S44: If both the position difference and the area difference are smaller than the corresponding preset values, the target is determined to be matched successfully, and the target is tracked through the target detection frame, and the process returns to step S41; otherwise, the process returns to step S43.

[0156] Specifically, when a target of the same category as the target in the queue is detected in a new frame, the pixel distance between the position of the detection frame and the position of the prediction frame is less than a certain threshold, and the area of ​​the detection frame differs from the size of the detection frame of the previous frame by less than 20%, then the target is considered to be the same target as the target locked in the previous frame. If there is only one frame of detection data in the queue, no prediction is performed and the data of this frame is directly used for threshold matching. The judgment conditions are:

[0157]

[0158] In the above formula, d is the preset distance threshold, (v,h) and (L,R,T,B) are the pixel coordinates and boundaries of the current frame target respectively, (v ′ ,h ′ ) and (L ′ ,R ′ ,T ′ ,B ′ ) are the pixel coordinates and boundaries of the predicted target in the previous frame;

[0159] The coordinates and size of the matched detection frame are stored in a queue to obtain the pixel coordinates of the target in the current frame. After obtaining the pixel coordinates of the target, the LiDAR point cloud within the detection frame is filtered using the target's image detection frame. The coordinates of the detected target can be output through operations such as clustering algorithms.

[0160] Continuously performing steps S43-S44 to match the target between the two frames will continuously obtain the real-time image coordinates of the tracked target. When the tracked target approaches the edge of the image, the difference between the target's pixel coordinates and the screen center coordinates is calculated. This coordinate difference is multiplied by the angular resolution to obtain the required pan / tilt rotation angle, thereby controlling the pan / tilt rotation to re-center the target on the screen and clear the queue. This operation is repeated continuously to achieve continuous tracking of the image target and coordinate output;

[0161] When clicking on a target on the visual camera screen, the system determines whether a target exists in the clicked area. Assume the clicked pixel coordinates are (a, b). The condition for determining whether a target exists in the clicked area is that if a target satisfies L≤a≤R∧T≤b≤B, the target is considered the clicked target and the process returns to step 2 to track and lock onto the target. If there is no target in the clicked area, the gimbal angle is directly calculated, rotating the clicked area to the center of the screen. If there are no subsequent clicks, the gimbal accepts guidance from the AIS and K-band radar until a target is identified or clicked again.

[0162] See Figure 5 , Figure 5 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically comprises: a target tracking device 401 for an escorted ship in an autonomous escort operation, a processor 402 and a storage medium 403.

[0163] A target tracking device 401 for an escorted ship in an autonomous escort operation: The target tracking device 401 for an escorted ship in an autonomous escort operation implements a target tracking method for an escorted ship in an autonomous escort operation.

[0164] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the target tracking method for the escorted ship in the autonomous escort operation.

[0165] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the target tracking method for the escorted ship in the autonomous escort operation.

[0166] It should be noted that, in this document, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0167] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same item of hardware. The use of the terms first, second, and third, etc., does not denote any order and should be construed as identifiers.

[0168] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.< / lf> < / cr>

Claims

1. A target tracking method for an escorted ship in an autonomous escort operation, characterized in that: Including steps: S1: The accompanying vessel obtains the AIS data of the accompanied vessel through the MMSI data of the accompanied vessel. If the acquisition is successful, the process proceeds to step S2; otherwise, the accompanying vessel is prohibited from entering the accompanying state. S2: In this stage, the escort vessel tracks the target through AIS and obtains the TTM data of the escorted vessel through K-band radar. If the AIS data is lost, the process returns to S1. If the AIS data matches the TTM data, the process proceeds to step S3. Otherwise, step S2 is repeated. S3: In this stage, the accompanying ship tracks the target through the fusion of AIS and K-band radar, and controls the visual camera to visually identify the accompanied ship. If the K-band radar loses the target, the process returns to step S2. If the visual recognition is successful, the process proceeds to step S4. Otherwise, step S3 is repeated. S4: The accompanying ship tracks the accompanied ship through the visual camera and laser radar. If the visual camera loses the target, it returns to step S3.

2. The target tracking method for the escorted ship in the autonomous escort operation according to claim 1, characterized in that: In step S2, the escort vessel tracks the target through AIS as follows: S211: The escorting ship obtains the AIS data received at the most recent time t0 from time t, and sets the state of the escorting ship at time t0 t is the current moment; S212: Construct a state transition model to convert the state Substitute into the state transfer model and calculate the predicted state of the escorted ship at time t S213: Obtain the Kalman gain by calculating the error covariance matrix, construct the observation vector, and predict the state by the Kalman gain and the observation vector. Update and get the updated status S214: Updated status The escorted ship is tracked, the error covariance matrix is ​​updated, and then the process returns to step S211.

3. The target tracking method for an escorted vessel in an autonomous escort operation according to claim 2, characterized in that: state Prediction Status Among them, * represents the predicted value, is the predicted AIS longitude at time t, is the predicted AIS latitude at time t, is the AIS longitude at time t0, is the AIS latitude at time t0, CRS AIS is the AIS heading, Vel AIS AIS speed, 1.141×10 -5 and 8.99×10 -6 Scaling factors for converting longitude and latitude to meters respectively.

4. The target tracking method for the escorted ship in the autonomous escort operation according to claim 1, characterized in that: In step S2, the process of matching AIS data with TTM data is as follows: S221: AIS data acquired by AIS includes the AIS longitude Lng of the escorted vessel AIS 、AIS latitude Lat AIS 、AIS heading CRS AIS and AIS speed Vel AIS ; The TTM data obtained by the K-band radar includes the K-wave distance d of the escorted ship K 、K wave azimuth θ K , K-wave heading CRS K and K wave speed Vel K ; S222: Time-synchronize the AIS data and the TTM data by interpolation, convert the TTM data and the AIS data into the same coordinate system, and obtain the target K-wave radar data and AIS data at the same time in the same coordinate system; S223: Calculate the total weighted Euclidean distance between the AIS track and the TTM track using the target K-wave radar data and the AIS data; S224: obtaining the similarity between the AIS trajectory and the TTM trajectory by calculating the total weighted Euclidean distance; S225: If the similarity is greater than or equal to the preset value, it is determined that the AIS data and the TTM data are successfully matched, and the process proceeds to step S3; otherwise, the process returns to step S222.

5. The target tracking method for the escorted vessel in the autonomous escort operation according to claim 4, characterized in that: The weighted Euclidean distance d between the i-th TTM track and the j-th AIS track at time t (i，j，t) The calculation formula is: Among them, w d and w v are the weights of position and velocity, respectively. and are the coordinates of the track points of TTM track and AIS track at time t, and are the velocity vectors of the track points of TTM track and AIS track at time t respectively; The total weighted Euclidean distance D between the i-th TTM track and the j-th AIS track (i，j) The calculation formula is: The calculation formula for the similarity ρ between the i-th TTM trajectory and the j-th AIS trajectory is: Where n is the number of trajectory points involved in the matching calculation, and λ is the regularization parameter.

6. The target tracking method for the escorted ship in the autonomous escort operation according to claim 1, characterized in that: In step S3, the escort ship tracks the target by integrating AIS with K-band radar as follows: S31: After the AIS data and TTM data are successfully matched, the AIS data and TTM data are acquired in real time, and the expected value of the TTM data at the current moment is calculated. If the TTM data at the next moment has the same number as the expected value and the Euclidean distance is less than a preset threshold, the TTM data at the next moment is used for weighted average fusion, and the process proceeds to step S32. If no TTM data at the next moment with the same number is found within the threshold range, the TTM data with the closest Euclidean distance to the expected value is used for weighted average fusion, and the process proceeds to step S32. If there is no new TTM data within the threshold range, it is determined that the target is lost, and the process returns to step S2. S32: Perform weighted average fusion on the AIS data and the TTM data to obtain fused target data Data; When receiving AIS data, the calculation formula of Data is: Among them, Data K For TTM data, Data AIS is AIS data, w K is the TTM data weight, w AIS is the AIS data weight; When no AIS data is received, the interpolated AIS data weight w is used AIS (t)=w0·e -λΔt Calculate Data; Among them, w AIS (t) is the interpolated AIS data weight at time t, w0 is the initial weight, λ is the decay constant, and Δt is the time difference of receiving AIS data; S33: Target tracking is performed using the fused target data.

7. The target tracking method for the escorted ship in the autonomous escort operation according to claim 1, characterized in that: In step S4, the target tracking of the escorted ship is performed by the visual camera and the laser radar as follows: S41: Obtain the actual position of the target detection frame at the current moment through the visual camera and laser radar, and obtain the center pixel of the detection frame; S42: Calculate the pixel difference between the center pixel of the detection frame and the center pixel of the screen, and control the gimbal based on the pixel difference to place the target at the center of the screen; S43: predicting the predicted position of the target detection frame at the next moment based on the actual position of the target detection frame at the current moment, and obtaining the actual position of the target detection frame at the next moment; Calculate the position difference between the predicted position and the actual position of the target detection frame at the next moment, as well as the area difference of the target detection frame; S44: If both the position difference and the area difference are less than the corresponding preset values, the target is determined to be matched successfully, and the target is tracked using the target detection frame, and the process returns to step S41; Otherwise, return to step S43.

8. A storage medium, characterized in that: The storage medium stores instructions and data for implementing the target tracking method for the escorted ship in the autonomous escort operation according to any one of claims 1 to 7.

9. A target tracking device for an escorted vessel in an autonomous escort operation, characterized in that: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the target tracking method for the escorted ship in the autonomous escort operation according to any one of claims 1 to 7.

Images