Ship feature snapshot method based on track prediction and active zooming

Abstract

The invention discloses a ship feature snapshot method based on track prediction and active zooming, which realizes real-time discovery through a lightweight detection model, solves geographic coordinates in combination with POS data and an imaging model, smoothes a track by using Kalman filtering and predicts a future position, drives a holder to adjust Pan / Tilt / Zoom parameters in advance to complete accurate alignment, and finally, determines whether the track is a target track or not. According to the method, the ship name is identified, high-resolution snapshot is triggered at the optimal moment, finally, the ship name and the IMO / MMSI number are identified through OCR, a traceable ship feature record is formed, and automatic comparison and verification with AIS data are supported.

Description

Technical Field

[0001] This invention belongs to the field of dynamic target tracking technology, specifically relating to a method for capturing ship features based on trajectory prediction and active zoom. Background Technology

[0002] Against the backdrop of increasingly stringent maritime law enforcement and water traffic supervision, drone-based vessel surveillance technology is facing a critical bottleneck in its transition from "coarse-grained surveillance" to "refined verification." With increasing vessel density at sea and the growing concealment of illegal operations, the need for real-time and accurate collection of vessel identification information (such as vessel name and MMSI number) has become particularly urgent. However, existing drone surveillance technologies still have significant shortcomings in core areas such as resolution adaptation, target tracking response, environmental robustness, and system coordination, making it difficult to meet the high standards of maritime law enforcement agencies regarding the rigor and traceability of evidence collection. Specific limitations are as follows: 1. The fundamental constraint on recognition accuracy: the disconnect between resolution and zoom. Current drone surveillance systems generally use wide-angle cameras with a single focal length for patrol surveillance. While wide-angle lenses can provide a larger field of view and cover a wider area of ​​sea, their video resolution is often limited by hardware power consumption and transmission bandwidth. In wide-angle mode, key identifying features of a vessel (such as its name, hull number, and MMSI number) occupy a very small proportion of the image, typically only a very low percentage (less than 0.5%) of the total pixels, resulting in blurred character details and insufficient contrast.

[0003] More seriously, existing systems lack an active zoom adaptation mechanism when shooting at long distances. The drone cannot adjust its optical focal length in real time according to the ship's movement and distance changes, resulting in characters in the captured images being too small (less than 10 pixels), completely exceeding the recognition threshold of the OCR (Optical Character Recognition) algorithm. This "blind capture, passive recognition" mode renders a large amount of crucial evidence invalid due to unreadable characters, greatly weakening the integrity of the evidence chain against illegally operating vessels.

[0004] 2. Target tracking response lag: Prediction inaccuracy and control lag The motion characteristics of ships at sea determine the uncertainty of their speed and the nonlinearity of their trajectories. Influenced by waves, wind, and their own maneuvering, ships experience large speed fluctuations and their trajectories are difficult to predict. Existing unmanned aerial vehicle (UAV) monitoring systems exhibit significant lag in target tracking. Gimbal control delay: When shooting in rough seas or in strong winds, the gimbal needs to be constantly fine-tuned to keep the target centered. However, the gimbal's rotation and zoom speeds have physical limits, and the control algorithm lacks deep integration with the ship's state, causing the ship to easily deviate from the optimal angle at the moment of capture.

[0005] Cumulative Delay Misalignment: During the gimbal rotation, optical zoom, and image exposure processes, drones generate a significant cumulative system delay. In the face of a high-speed ship, this delay can cause a substantial spatial misalignment of the target at the moment of capture, meaning that "the image captures the ship's historical position," making it difficult to correlate the captured image with the actual ship's identity.

[0006] 3. Adaptability deficiencies in complex environments: Robustness needs to be improved. The marine environment is extremely complex and changeable, with extreme weather and lighting conditions such as backlighting, nighttime, and rough seas being common.

[0007] Backlighting and Reflection: When shooting against the light, the ship's name and hull number on the side of the vessel are often strongly reflected by the sunlight, causing the character area to be overexposed or completely obscured by light spots. Existing character recognition algorithms are mainly trained for uniform lighting environments and have extremely weak feature extraction capabilities for areas with high light intensity reflection, which easily leads to misjudgments.

[0008] Nighttime and Low Light: Nighttime photography relies on the ship's own navigation lights or the supplemental lighting from drones. Uneven light distribution and frequent light flicker can lead to a surge in image noise, causing existing algorithms to experience a sharp decline in feature extraction and recognition rates in high-noise environments.

[0009] Wind and wave disturbances: Strong winds and waves cause ships to rock violently, making characters blurry and causing drastic changes in tilt angle. Existing algorithms are not adaptable enough and cannot complete accurate recognition within a very short capture window.

[0010] 4. The crisis of system collaboration disruption: lack of closed-loop awareness Perhaps the most fundamental limitation of current technology is that existing drone monitoring solutions are still "puzzle-like" individual operations in terms of system coordination.

[0011] Existing drone monitoring solutions mostly rely on fixed-focus capture or manual remote zoom control, without actively predicting and adapting to the ship's movement status; some character recognition solutions are only optimized for single environmental scenarios and do not cover complex water conditions, making it difficult to meet the rigorous requirements of maritime law enforcement. Summary of the Invention

[0012] The technical problem this invention aims to solve is to overcome the above-mentioned technical defects and provide a ship feature capture method based on trajectory prediction and active zoom. It achieves real-time detection through a lightweight detection model, calculates geographic coordinates by combining POS data and imaging model, smooths the trajectory using Kalman filtering and predicts future position, drives the gimbal to adjust Pan / Tilt / Zoom parameters in advance to achieve precise alignment, and triggers high-resolution capture at the optimal moment. Finally, it identifies the ship name and IMO / MMSI number through OCR to form a traceable ship feature record, which supports automatic comparison and verification with AIS data.

[0013] To address the aforementioned problems, the technical solution of this invention is: a method for capturing ship features based on trajectory prediction and active zoom, comprising the following steps: (a) The drone flies periodically along a preset route to collect real-time video of the monitored waters in wide-angle mode, and a lightweight ship target detection model is used to perform frame-by-frame reasoning on the video frames to obtain the initial pixel bounding box and confidence level of the ship target. (b) Based on the initial pixel bounding box, combined with the longitude, latitude and heading angle of the UAV platform, the intrinsic parameters, heading angle, pitch angle and current focal length of the image acquisition unit, the real-time latitude and longitude position of the ship in the WGS-84 coordinate system is solved by using the pinhole imaging model, and the position of multiple consecutive frames is smoothed by using a Kalman filter to estimate the real-time speed v and heading angle θ of the ship. (c) Based on the speed v, heading angle θ and system delay Δt, calculate the predicted latitude and longitude position P′ of the ship after time Δt, and use it as the optimal capture point; (d) The gimbal control unit calculates the required horizontal rotation angle Pan, pitch angle Tilt and focal length Zoom based on the relative geometric relationship between the predicted position P′ and the camera installation position, and drives the gimbal motor to move in a closed loop with maximum acceleration to the preset position to complete active zoom and alignment. (e) Once the gimbal is in position, immediately switch to telephoto mode to trigger high-resolution capture and obtain a close-up image of the ship including the bow, hull, and ship name characters. (f) The ship name character area in the close-up image is parsed using an OCR text recognition model to extract the ship name; at the same time, the ship identification number is located and identified by prior knowledge of the ship outline and MMSI number painting position, forming a ship feature record containing capture time, latitude and longitude, ship name, identification number and close-up image, and written into the regulatory database.

[0014] Preferably, the lightweight ship target detection model is YOLOv5-s or YOLOv8-n, deployed on an edge computing box, with an inference frame rate of ≥25 FPS.

[0015] Preferably, the state vector of the Kalman filter includes latitude and longitude, speed, heading and acceleration, and the observation noise R and process noise Q are dynamically adjusted according to the waterway grade.

[0016] Preferably, the system delay Δt is obtained through pre-calibration, including: detection delay 10 ms, gimbal rotation delay ≤300 ms, zoom delay ≤200 ms, and exposure delay ≤30 ms.

[0017] Preferably, the focal length (Zoom) is automatically determined by the threshold β, which represents the proportion of ship pixels to the target surface, ensuring that the height of the ship name characters is ≥60 pixels and that β is ≥15%.

[0018] Preferably, the OCR text recognition model adopts the CRNN+CTC architecture, and the training samples include images of ship name characters in inland waterways, coastal areas and backlit nighttime scenes, and data augmentation is used to improve robustness.

[0019] Preferably, after step (f), the ship feature record is further compared with the real-time AIS message received by the AIS base station. If the identification number is consistent and the ship name similarity is ≥95%, it is marked as verified; otherwise, manual review is triggered.

[0020] Preferably, the vessel identification number is an IMO number or an MMSI number.

[0021] The advantages of this invention compared to existing technologies are: Significantly improved capture accuracy: By using a trajectory prediction compensation system to compensate for delays, combined with active gimbal alignment and zoom adaptation, the problem of capture misalignment caused by ship movement is solved, and the clear capture rate of ship names and identification numbers is increased to over 98%.

[0022] High adaptability to complex environments: The optimized OCR model is trained with data from multiple scenarios, and the character recognition accuracy is ≥95% in complex environments such as backlight, night, wind and waves, meeting the regulatory needs of different water areas.

[0023] High data reliability: The introduction of an AIS message automatic matching and verification mechanism enables cross-verification of identification results, effectively reducing the false identification rate and providing reliable evidence for law enforcement.

[0024] Balancing real-time performance and efficiency: Lightweight detection models and edge computing deployment ensure real-time response for target detection and trajectory estimation. The entire capture process (from detection to record generation) takes ≤1 second, meeting the needs of batch ship monitoring.

[0025] Full-process automation: No human intervention is required, realizing full-process automation from target detection and capture control to identification and verification, greatly reducing labor costs and improving the efficiency of waterway supervision. Detailed Implementation

[0026] The specific implementation of the present invention will be further illustrated below with reference to the embodiments.

[0027] Example: A method for capturing ship features based on trajectory prediction and active zoom includes the following steps: (a) The drone flies periodically along a preset route to collect real-time video of the monitored waters in wide-angle mode, and a lightweight ship target detection model is used to perform frame-by-frame reasoning on the video frames to obtain the initial pixel bounding box and confidence level of the ship target. (b) Based on the initial pixel bounding box, combined with the longitude, latitude and heading angle of the UAV platform, the intrinsic parameters, heading angle, pitch angle and current focal length of the image acquisition unit, the real-time latitude and longitude position of the ship in the WGS-84 coordinate system is solved by using the pinhole imaging model, and the position of multiple consecutive frames is smoothed by using a Kalman filter to estimate the real-time speed v and heading angle θ of the ship. (c) Based on the speed v, heading angle θ and system delay Δt, calculate the predicted latitude and longitude position P′ of the ship after time Δt, and use it as the optimal capture point; (d) The gimbal control unit calculates the required horizontal rotation angle Pan, pitch angle Tilt and focal length Zoom based on the relative geometric relationship between the predicted position P′ and the camera installation position, and drives the gimbal motor to move in a closed loop with maximum acceleration to the preset position to complete active zoom and alignment. (e) Once the gimbal is in position, immediately switch to telephoto mode to trigger high-resolution capture and obtain a close-up image of the ship including the bow, hull, and ship name characters. (f) The ship name character area in the close-up image is parsed using an OCR text recognition model to extract the ship name; at the same time, the ship identification number is located and identified by prior knowledge of the ship outline and MMSI number painting position, forming a ship feature record containing capture time, latitude and longitude, ship name, identification number and close-up image, and written into the regulatory database.

[0028] System Architecture and Hardware Composition: The system of this invention mainly consists of the following parts: Unmanned Aerial Vehicle (UAV) Platform: Equipped with a high-resolution camera, GNSS positioning module, and gimbal. The camera supports both wide-angle and telephoto working modes, and the gimbal has dual-axis motion capabilities (pan and tilt) with a motion control accuracy within 0.1 degrees.

[0029] Edge computing box: Deploys a lightweight ship target detection model (such as YOLOv5-s) to perform frame-level processing on real-time video streams captured by drones. This box connects to the camera via a high-speed interface (such as USB 3.0) and uploads feature data to the monitoring center via a wireless network (such as 4G / 5G).

[0030] Gimbal control unit: Receives control commands from the edge computing box, drives the gimbal motor to achieve rapid movement, and controls the camera to switch between wide-angle and telephoto modes.

[0031] Regulatory database: Stores generated vessel characteristic records, including capture time, latitude and longitude, vessel name, identification number, and close-up images.

[0032] Software Algorithm and Core Steps: The method of this invention includes the following key steps: Step (a): Video Acquisition and Target Detection The drone flies periodically along a pre-defined cruise route (such as a rectangle or ellipse). During this process, the camera is in wide-angle mode by default to obtain a larger monitoring field of view. The edge computing box loads a lightweight detection model (YOLOv5-s), which has been quantized and optimized to achieve an inference speed of ≥25 FPS on embedded devices. The model scans each frame of the image and outputs the pixel bounding box of the ship and its confidence score.

[0033] Step (b): Track calculation and smoothing After obtaining the initial pixel bounding box, the system first uses a pinhole imaging model combined with the camera's intrinsic parameter matrix to backproject the pixels from the image coordinate system to the world coordinate system (WGS-84), obtaining the ship's initial latitude and longitude coordinates. Due to the large error in single-frame data, the system further calls a Kalman filter for data fusion. The filter's state vector includes: Location: Longitude, Latitude Speed: Ship speed v (m / s) Direction: Heading angle θ (degrees) Acceleration: Used to predict short-term motion trends The filter's observation noise R and process noise Q are dynamically adjusted according to the current waterway's classification (such as inland waterway, coastal waterway, deep sea) to adapt to the motion characteristics under different environments.

[0034] Step (c): Track prediction After acquiring the real-time speed v and heading θ, the system takes into account the system delay Δt (including calculation delay, command transmission delay, mechanical response delay, etc.) between the detection and the completion of the gimbal movement, and predicts the ship's ideal position P′ after time Δt using the following formula:

[0035] in, This refers to the current latitude and longitude coordinates of the ship.

[0036] Step (d): Active Alignment and Zoom After receiving the predicted position P′, the gimbal control unit first calculates the relative position vector between the UAV's current coordinates and P′. Based on this vector, the system calculates the required horizontal angle Pan and pitch angle Tilt. Simultaneously, based on the ratio threshold β between the current field of view and the target size (recommended to be set to 15%), the system calculates the required zoom level to ensure the ship's name characters occupy at least 60 pixels in height in the close-up image. The control unit then instructs the gimbal motors to perform a closed-loop motion with maximum acceleration to reach the target position in the shortest possible time.

[0037] Step (e): High-resolution capture Once the gimbal is in position, the camera immediately switches from wide-angle mode to telephoto mode and triggers the shutter to capture the image. Because the camera is already aligned, the captured image is extremely clear, preserving the details of the bow, hull, and ship's name.

[0038] Step (f): Feature extraction and data storage The system inputs the captured high-resolution images into an OCR text recognition model deployed on an edge box. This model uses a CRNN+CTC architecture and is able to recognize tilted or blurred characters under irregular lighting conditions. The model first locates the ship name character region (usually located in a conspicuous position on the bow or side of the ship), and then parses out the specific ship name.

[0039] For the vessel identification number (IMO number or MMSI number), the system locates the vessel by detecting standard paint positions on the hull (such as the side or cabin door). The system also cross-compares the parsed vessel name and identification number with the real-time AIS messages received by the AIS base station. If they match and the similarity exceeds 95%, it is directly marked as "verified"; if they do not match, a manual review process is triggered to prevent misjudgment.

[0040] Ultimately, the system packages the capture time, latitude and longitude coordinates, ship name, identification number, and close-up image, and writes them into the database of the monitoring center to form a complete electronic evidence chain.

[0041] Preferred solution Model selection: To adapt to environments with limited edge computing resources, it is preferable to deploy lightweight YOLOv5-s or YOLOv8-n models and accelerate them through TensorRT.

[0042] Delay calibration: Before the system goes online, a detailed calibration experiment must be conducted to measure the detection delay (approximately 10 ms), gimbal rotation delay (≤300 ms), and zoom delay (≤200 ms) to calculate the accurate system delay Δt.

[0043] Image quality control: A pixel ratio threshold β is introduced in the zoom calculation to ensure that the ship name characters remain clear in the image even when the ship is moving at high speed, which facilitates subsequent OCR recognition.

[0044] Data augmentation: The training data for the OCR model needs to cover extreme weather scenarios such as nighttime backlighting, rainy days, and foggy days. Data augmentation is performed through random rotation, scaling, and lighting changes to improve the robustness of the model.

[0045] This invention aims to solve the following key technical problems existing in current ship feature capture based on trajectory prediction and active zoom: 1. The challenge of obtaining real-time ship geolocation from wide-angle video: In wide-angle videos, ships appear only as pixels, making it impossible to directly obtain their geographic coordinates. This invention establishes a mechanism for solving the intersection point of a ray and the sea surface by combining pixel-camera model inverse kinematics with UAV attitude and camera parameters, thereby achieving real-time WGS-84 position calculation for ships.

[0046] 2. Predicting the future position of the ship to compensate for system lag: Due to delays of hundreds of milliseconds in detection, gimbal, and zoom, the ship may shift before the image is captured. This invention utilizes Kalman filtering to smooth the trajectory and calculates the position Δt after speed and heading, achieving delay compensation prediction.

[0047] 3. The challenge of rapid and accurate gimbal alignment in telephoto mode: Telephoto lenses have extremely narrow fields of view, and even slight deviations can lead to target loss. This invention calculates the optimal Pan / Tilt / Zoom ratio based on the spatial relationship between the predicted position and the drone's coordinates, and achieves rapid alignment and capture in a single shot through closed-loop control.

[0048] 4. The challenge of adaptive focal length control for clear imaging of ship name characters: OCR often fails due to insufficient pixel size for distant ship names. This invention automatically adjusts the focal length based on the ship's pixel ratio, ensuring that the character height is no less than 60 pixels, thereby significantly improving the recognition success rate.

[0049] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for capturing ship features based on trajectory prediction and active zoom, characterized in that: Includes the following steps: (a) The drone flies periodically along a preset route to collect real-time video of the monitored waters in wide-angle mode, and a lightweight ship target detection model is used to perform frame-by-frame reasoning on the video frames to obtain the initial pixel bounding box and confidence level of the ship target. (b) Based on the initial pixel bounding box, combined with the longitude, latitude and heading angle of the UAV platform, the intrinsic parameters, heading angle, pitch angle and current focal length of the image acquisition unit, the real-time latitude and longitude position of the ship in the WGS-84 coordinate system is solved by using the pinhole imaging model, and the position of multiple consecutive frames is smoothed by using a Kalman filter to estimate the real-time speed v and heading angle θ of the ship. (c) Based on the speed v, heading angle θ and system delay Δt, calculate the predicted latitude and longitude position P′ of the ship after time Δt, and use it as the optimal capture point; (d) The gimbal control unit calculates the required horizontal rotation angle Pan, pitch angle Tilt and focal length Zoom based on the relative geometric relationship between the predicted position P′ and the camera installation position, and drives the gimbal motor to move in a closed loop with maximum acceleration to the preset position to complete active zoom and alignment. (e) Once the gimbal is in position, immediately switch to telephoto mode to trigger high-resolution capture and obtain a close-up image of the ship including the bow, hull, and ship name characters. (f) The ship name character area in the close-up image is parsed using an OCR text recognition model to extract the ship name; at the same time, the ship identification number is located and identified by prior knowledge of the ship outline and MMSI number painting position, forming a ship feature record containing capture time, latitude and longitude, ship name, identification number and close-up image, and written into the regulatory database.

2. The method for capturing ship features based on trajectory prediction and active zoom according to claim 1, characterized in that: The lightweight ship target detection model is YOLOv5-s or YOLOv8-n, deployed on an edge computing box, with an inference frame rate of ≥25 FPS.

3. The method for capturing ship features based on trajectory prediction and active zoom according to claim 1, characterized in that: The state vector of the Kalman filter includes latitude and longitude, speed, heading, and acceleration. The observation noise R and process noise Q are dynamically adjusted according to the waterway grade.

4. The method for capturing ship features based on trajectory prediction and active zoom according to claim 1, characterized in that: The system delay Δt is obtained through pre-calibration and includes: detection delay 10 ms, gimbal rotation delay ≤300 ms, zoom delay ≤200 ms, and exposure delay ≤30 ms.

5. The method for capturing ship features based on trajectory prediction and active zoom according to claim 1, characterized in that: The focal length (Zoom) is automatically determined by the threshold β, which represents the proportion of ship pixels to the target surface, ensuring that the height of the ship name characters is ≥60 pixels and that β is ≥15%.

6. The method for capturing ship features based on trajectory prediction and active zoom according to claim 1, characterized in that: The OCR text recognition model adopts a CRNN+CTC architecture. The training samples include images of ship name characters in inland waterways, coastal areas, and backlit nighttime scenes, and data augmentation is used to improve robustness.

7. The method for capturing ship features based on trajectory prediction and active zoom according to claim 1, characterized in that: After step (f), the ship feature record is further compared with the real-time AIS message received by the AIS base station. If the identification number is consistent and the ship name similarity is ≥95%, it is marked as verified; otherwise, manual review is triggered.

8. The method for capturing ship features based on trajectory prediction and active zoom according to claim 1, characterized in that: The vessel identification number is either an IMO number or an MMSI number.

Images