Ship tracking method, equipment and medium

Abstract

The invention discloses a ship tracking method and device and a medium, and relates to the technical field of data processing. The method comprises the following steps: through cooperative work of a fixed camera and a PTZ camera, the technical problem that wide-area coverage and high-precision tracking are difficult to consider at the same time under a single PTZ architecture is effectively solved; a deep learning model is utilized to identify ship types, positions and confidence coefficients in real time, and a preset multi-level priority decision strategy is combined, so that high-value targets such as cargo ships and dangerous goods transport ships can be automatically focused, and monitoring resources are prevented from being ineffectively occupied by low-priority targets; furthermore, through PTZ control parameter compensation based on a motion track and a self-adaptive zooming mechanism based on an expected imaging size under pointing after motion compensation, the tracking stability and the picture detail definition are remarkably improved. In conclusion, rapid identification, accurate locking and continuous high-definition tracking of the key target are realized, and the overall tracking efficiency, the intelligent level and the emergency response capability of the water area monitoring system are greatly improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a ship tracking method, device and medium. Background Technology

[0002] With increasingly busy water traffic, the importance of waterway safety monitoring is constantly rising. Existing waterway monitoring systems mostly employ a single PTZ (Pan-Tilt-Zoom) camera architecture, which struggles to simultaneously meet the dual demands of wide-area coverage and high-precision detail tracking. In practical applications, when multiple moving vessel targets appear within the monitored area, traditional systems generally lack effective intelligent discrimination mechanisms. They cannot automatically identify and focus on key targets such as cargo ships and dangerous goods transport vessels based on target type, behavioral characteristics, or task priority. This results in monitoring resources being consumed by low-priority targets, severely impacting overall monitoring and tracking efficiency and emergency response capabilities.

[0003] Therefore, providing a ship tracking method to improve the overall tracking efficiency of existing technologies is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a ship tracking method, device, and medium, with the main objective of providing a ship tracking method to improve the overall tracking efficiency of existing technologies.

[0005] In a first aspect, embodiments of this application provide a vessel tracking method, the method comprising: acquiring wide-area video of a monitored waterway using a fixed camera, and performing real-time analysis of the acquired video frames based on a deep learning detection model, so as to identify in real time the category of each vessel target in the last acquired video frame, the position coordinates of each vessel target in the video frame, and the confidence level of the category; Based on a preset priority rule, a ship target is selected from the plurality of ship targets as the target to be tracked; The position coordinates are mapped and calculated to obtain the PTZ control parameters corresponding to the position coordinates; Motion compensation is performed on the PTZ control parameters based on the motion trajectory of the target to be tracked, and adaptive zoom is performed based on the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation, and PTZ control commands are generated. The PTZ control commands are used to control the PTZ camera to track the target.

[0006] In one implementation of this application, the step of mapping and calculating the position coordinates to obtain the PTZ control parameters corresponding to the position coordinates includes: The position coordinates are normalized to obtain normalized position coordinates; The normalized position coordinates and polynomial coefficients are input into a preset cubic polynomial function, and the horizontal rotation angle and vertical tilt angle of the PTZ camera are calculated by mapping. The PTZ control parameters include the horizontal rotation angle and the vertical tilt angle. The polynomial coefficients are obtained by fitting a preset number of calibration data with a cubic polynomial through a ridge regression model.

[0007] In one implementation of this application, the step of performing motion compensation on the PTZ control parameters based on the motion trajectory of the target to be tracked, and performing adaptive zooming based on the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation, and generating PTZ control commands includes: Predict the position offset of the target at a future preset time based on the movement trajectory of the target; The motion compensation sensitivity coefficient is determined based on the current zoom level of the PTZ camera; The position offset is weighted based on the sensitivity coefficient, and motion compensation is performed on the PTZ control parameters based on the weighting result. Based on the category of the target to be tracked and the expected imaging size, the current zoom level is adaptively adjusted to obtain an adaptive zoom level. The adaptive zoom level is used to ensure that the target to be tracked maintains a preset display ratio in the PTZ camera image after responding to the PTZ control command. The motion-compensated horizontal rotation angle, the motion-compensated vertical tilt angle, and the adaptive zoom level are combined to generate the PTZ control command.

[0008] In one implementation of this application, the step of adaptively adjusting the current zoom level based on the category of the target to be tracked and the expected imaging size to obtain an adaptive zoom level includes: The target imaging size of the target to be tracked in the PTZ camera image is determined according to the category of the target to be tracked and the preset display ratio; Based on the motion-compensated PTZ pointing, the current zoom level, and the distance information between the target to be tracked and the PZT camera, the expected imaging size of the target to be tracked is calculated; The expected imaging size is compared with the target imaging size to generate a zoom adjustment amount, and the current zoom level is updated according to the zoom adjustment amount to obtain the adaptive zoom level.

[0009] In one implementation of this application, the method further includes: Calculate the relative size ratio of the detection box of the target to be tracked in the fixed camera image; Based on the relative size ratio, a basic zoom level is calculated through a linear mapping relationship, wherein the linear mapping relationship is configured such that when the relative size ratio is at its minimum, the basic zoom level takes its maximum value; when the relative size ratio is at its maximum, the basic zoom level takes its minimum value; and the basic zoom level decreases linearly as the relative size ratio increases. The current zoom level of the PTZ camera is replaced with the base zoom level to achieve adaptive zoom.

[0010] In one implementation of this application, the method further includes: The movement trend of the target to be tracked is determined based on the historical location data of the target to be tracked. Based on the motion trend, feedforward compensation is applied to the horizontal rotation angle and / or vertical tilt angle of the PTZ control parameters to perform motion compensation and suppress tracking deviation.

[0011] In one implementation of this application, the preset priority rule includes three priority levels of decision strategies. The first-level priority decision strategy selects a ship target of a specified category from the plurality of ship targets as the target to be tracked. The second-level priority decision strategy continues tracking of the already tracked ship target if no ship target of the specified category exists among the plurality of ship targets. The third-level priority decision strategy selects a ship target from the remaining ship targets according to a default strategy if the already tracked ship target is lost. The first-level priority decision strategy has a higher priority than the second-level priority decision strategy, which in turn has a higher priority than the third-level priority decision strategy. The default strategy includes any one of the area-first strategy, the existence duration strategy, and the confidence-first strategy. The area-first strategy selects the ship target with the largest detection box area, the existence duration strategy selects the ship target that appears in the monitoring screen for the longest time, and the confidence-first strategy selects the ship target with the highest confidence score.

[0012] In one implementation of this application, after controlling the PTZ camera to track the target based on the PTZ control command, the method includes: The position of the target to be tracked in the video stream of the PTZ camera is monitored in real time, and the PTZ control parameters are dynamically updated according to the position changes of the target to form a closed-loop tracking control.

[0013] Secondly, embodiments of this application also provide a vessel tracking device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: acquire wide-area video of a monitored water area using a fixed camera, and perform real-time analysis of the acquired video frames based on a deep learning detection model to identify in real-time the category of each vessel target in the last acquired video frame, the position coordinates of each vessel target in the video frame, and the confidence level of the category; select one vessel target from the plurality of vessel targets as the target to be tracked based on a preset priority rule; perform mapping calculation on the position coordinates to obtain PTZ control parameters corresponding to the position coordinates; perform motion compensation on the PTZ control parameters according to the motion trajectory of the target to be tracked, and perform adaptive zooming according to the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing after motion compensation, generating PTZ control instructions; and control the PTZ camera to track the target to be tracked based on the PTZ control instructions.

[0014] Thirdly, embodiments of this application also provide a non-volatile computer storage medium for ship tracking, storing computer-executable instructions. These instructions are configured to: acquire wide-area video of the monitored waterway using a fixed camera, and perform real-time analysis of the acquired video frames based on a deep learning detection model to identify in real-time the category of each ship target in the last acquired video frame, the position coordinates of each ship target in the video frame, and the confidence level of the category; select one ship target from the plurality of ship targets as the target to be tracked based on a preset priority rule; perform mapping calculation on the position coordinates to obtain PTZ control parameters corresponding to the position coordinates; perform motion compensation on the PTZ control parameters according to the motion trajectory of the target to be tracked, and perform adaptive zooming according to the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation, generating PTZ control instructions; and control the PTZ camera to track the target to be tracked based on the PTZ control instructions.

[0015] This application provides a vessel tracking method, device, and medium, comprising: effectively solving the technical challenge of simultaneously achieving wide-area coverage and high-precision tracking under a single PTZ architecture by using a fixed camera and a PTZ camera in collaborative operation; utilizing a deep learning model to identify vessel type, location, and confidence level in real time, and combining a preset multi-level priority decision strategy to automatically focus on high-value targets such as cargo ships and dangerous goods transport ships, avoiding the ineffective occupation of monitoring resources by low-priority targets; furthermore, significantly improving tracking stability and image detail clarity through PTZ control parameter compensation based on motion trajectory and an adaptive zoom mechanism based on the expected imaging size after motion compensation. In summary, this invention achieves rapid identification, accurate locking, and continuous high-definition tracking of key targets, greatly improving the overall tracking efficiency, intelligence level, and emergency response capability of waterway monitoring systems. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart of a ship tracking method provided in this application embodiment; Figure 2 This is a schematic diagram of the internal structure of a ship tracking device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] To facilitate understanding of the following embodiments of this application, the embodiments of this application provide some conceptual explanations and definitions, including: PTZ refers to the gimbal zoom control system, including horizontal rotation, vertical tilt, and optical zoom functions. YOLO is a real-time object detection algorithm based on deep learning. RTSP is a real-time streaming protocol for transmitting video streams. ONVIF is an interoperability standard between network video devices. IoU is the intersection-over-union ratio metric used in object detection to evaluate the degree of overlap of detection boxes.

[0019] This application provides a ship tracking method, device, and medium to address the following technical problem: how to improve the overall tracking efficiency of existing technologies.

[0020] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0021] Figure 1 This is a flowchart illustrating a ship tracking method provided in an embodiment of this application. Figure 1 As shown in the figure, the ship tracking method provided in this application embodiment specifically includes the following steps: Step 101: Wide-area video is captured on the monitored waters using a fixed camera, and the captured video frames are analyzed in real time based on a deep learning detection model to identify the category of each vessel target in the last captured video frame, the position coordinates of each vessel target in the video frame, and the confidence level of the category.

[0022] In some embodiments, the fixed camera is deployed in a fixed location to cover the entire monitoring area and acquire a panoramic video stream. Fixed cameras typically employ high resolution and wide-angle lenses to ensure coverage.

[0023] In some embodiments, the deep learning detection model includes, but is not limited to, the YOLO model. This deep learning detection model is used for ship detection and classification, performing real-time analysis of video frames captured by a fixed camera to detect ship targets and identify their categories (e.g., cargo ships, passenger ships, etc.). The deep learning detection model is a pre-trained deep learning model and can be fine-tuned on specific datasets to improve accuracy.

[0024] Step 102: Based on the preset priority rules, select one ship target from the multiple ship targets as the target to be tracked.

[0025] Step 103: Perform mapping calculations on the position coordinates to obtain the PTZ control parameters corresponding to the position coordinates.

[0026] To facilitate understanding of step 103, this embodiment provides a detailed explanation, including: establishing a coordinate mapping relationship between the fixed camera and the PTZ camera. Multiple corresponding points are obtained through a calibration process, and a polynomial fitting algorithm is used to convert the target position coordinates in the fixed camera into PTZ control parameters (Pan - horizontal rotation angle, Tilt - vertical rotation angle, Zoom - zoom level) for the PTZ camera. The mapping relationship is obtained by fitting calibration data, ensuring that the PTZ camera can accurately point to the position of the vessel target in the fixed camera. The polynomial fitting algorithm is used to perform mapping calculations on the position coordinates.

[0027] Step 104: Perform motion compensation on the PTZ control parameters according to the motion trajectory of the target to be tracked, and perform adaptive zoom according to the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation, and generate PTZ control commands.

[0028] In some embodiments, the PTZ parameters are compensated based on the motion characteristics of the vessel target. Specifically, this includes analyzing the vessel target's trajectory and direction, predicting the vessel target's next position, and adjusting the PTZ parameters accordingly to offset the errors caused by gimbal response delay and target motion, thereby improving tracking stability.

[0029] Step 105: Control the PTZ camera to track the target based on the PTZ control command.

[0030] In some embodiments, the PTZ camera has PTZ (pan, tilt, zoom) functionality for capturing details of a selected target. It can adjust the viewing angle and focal length according to control commands to track a specific target and acquire high-definition detailed images.

[0031] This application provides a vessel tracking method, comprising: effectively solving the technical challenge of simultaneously achieving wide-area coverage and high-precision tracking under a single PTZ architecture by using a fixed camera and a PTZ camera in collaborative operation; utilizing a deep learning model to identify vessel type, location, and confidence level in real time, and combining this with a preset multi-level priority decision strategy to automatically focus on high-value targets such as cargo ships and dangerous goods transport ships, avoiding the ineffective occupation of monitoring resources by low-priority targets; furthermore, significantly improving tracking stability and image detail clarity through PTZ control parameter compensation based on motion trajectory and an adaptive zoom mechanism based on the expected imaging size after motion compensation. In summary, this invention achieves rapid identification, accurate locking, and continuous high-definition tracking of key targets, significantly improving the overall tracking efficiency, intelligence level, and emergency response capability of waterway monitoring systems.

[0032] Furthermore, in one possible implementation of this application, another method for ship tracking includes the following: establishing a video stream connection between a fixed camera and a PTZ camera. Initialization includes setting the network parameters of the cameras, the video stream format, and the resolution, and testing the connection success. If the connection fails, a retry or alarm will be triggered. The fixed camera captures the video stream in real time, and a YOLO model is used for multi-target ship detection. For each frame, the YOLO model is used to detect ship targets, and the location bounding box, category, and confidence score of each ship target are obtained. A unique ID is assigned to each detected target, and the state information of the ship targets (such as position, size, category, appearance time, last update time, etc.) is maintained. A tracking algorithm is used to associate targets in previous and subsequent frames, update the target state, and remove targets that have not appeared for a long time. A three-level strategy is used to select tracking targets. First, among all detected vessel targets, those of a specified category (e.g., cargo ships) are prioritized. If no vessel target of a specified category is found, the currently tracked vessel target is checked for validity (i.e., still in the frame and not removed). If valid, tracking continues. If invalid, the best vessel target is selected using a default strategy (e.g., selecting the target with the largest area). The target coordinates in the fixed camera are mapped to the PTZ parameters of the detail camera (i.e., the PTZ camera) using a polynomial fitting algorithm. Using pre-calibrated polynomial coefficients, the normalized coordinates of the target (representing the vessel target) in the fixed camera image are converted to the horizontal rotation (P), vertical rotation (T), and zoom (Z) of the detail camera. Image resolution normalization is considered during the calculation to ensure mapping accuracy. The final control commands are generated by combining motion direction compensation and adaptive zoom mechanisms. Motion direction compensation calculates the motion direction based on the target's historical position and compensates for the PTZ parameters (e.g., increasing the horizontal rotation when moving left and decreasing it when moving right). Adaptive zoom adjusts zoom parameters based on target type and size; for example, high zoom is used for cargo ships, while default zoom is used for small vessels. The generated PTZ control commands are sent to the detail camera to aim at the selected target, achieving continuous and stable tracking. Simultaneously, the target position is monitored in real time, and PTZ parameters are updated based on target movement, forming a closed-loop control system.

[0033] In some embodiments, the three-level strategy for selecting tracking targets specifically includes: a three-level priority target selection mechanism: through three-level decision-making based on category priority, target continuity, and default strategy, important targets (such as cargo ships) are prioritized for tracking. This mechanism first filters targets of a specified category, then maintains the continuity of the current target to avoid frequent switching, and finally selects targets according to the default strategy when no target of a specified category is specified, thereby making optimal decisions in complex scenarios. The camera collaborative mapping algorithm includes cubic polynomial fitting based on N-point calibration to establish a high-precision coordinate-PTZ parameter mapping relationship. By collecting multiple corresponding points (fixed camera image coordinates and detail camera PTZ parameters) between the fixed camera and the detail camera, a cubic polynomial is used for fitting to obtain the coefficients of the mapping function. This algorithm can accurately convert the target position in the fixed camera into the control parameters of the detail camera. Motion adaptive compensation includes: by analyzing the target motion trajectory, predicting the motion direction and compensating for the PTZ parameters to improve tracking stability. This technology utilizes the target's historical position data to calculate the motion direction (such as from left to right or from right to left), and compensates for the PTZ parameters according to the motion direction to offset the errors caused by gimbal response delay and target motion. Intelligent zoom control includes automatically adjusting the zoom level based on the target category, balancing the observation range and image detail. For targets requiring detailed observation (such as cargo ships), high-magnification zoom is used to obtain a clear image; for other targets, the default zoom is used to maintain the observation range. Zoom control also considers the size and distance of the target to avoid excessive zooming that could lead to target loss.

[0034] In some embodiments, the method may also employ, but is not limited to, the following implementation: for example, parallel tracking of the multiple ship targets based on a preset tracking algorithm. Specifically, this includes: assigning a unique ID to each detected ship target, recording information such as the ship target's position, speed, size, and category, and using a preset tracking algorithm (such as Kalman filtering, Hungarian algorithm, etc.) to maintain continuous tracking of the target, so that the target can be re-associated even if it is temporarily lost.

[0035] As a refinement of this embodiment, when performing step 103 to map and calculate the position coordinates to obtain the PTZ control parameters corresponding to the position coordinates, the following implementation methods can also be adopted, but are not limited to: normalizing the position coordinates to obtain normalized position coordinates; inputting the normalized position coordinates and polynomial coefficients into a preset cubic polynomial function to map and calculate the horizontal rotation angle and vertical tilt angle of the PTZ camera, wherein the PTZ control parameters include the horizontal rotation angle and the vertical tilt angle, and the polynomial coefficients are obtained by fitting a preset number of calibration data to a cubic polynomial using a ridge regression model. The calculation process of the PTZ control parameters has been described in the above embodiments, and for clarity and conciseness, it will not be repeated in this embodiment.

[0036] As a refinement of the above embodiment, when performing step 104—namely, performing motion compensation on the PTZ control parameters based on the motion trajectory of the target to be tracked, and adaptively zooming based on the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation—and generating PTZ control commands, the following implementation methods can also be adopted, but are not limited to: predicting the position offset of the target to be tracked at a future preset time based on the motion trajectory of the target to be tracked; determining the motion compensation sensitivity coefficient based on the current zoom level of the PTZ camera; weighting the position offset based on the sensitivity coefficient, and performing motion compensation on the PTZ control parameters based on the weighting result; adaptively adjusting the current zoom level according to the category of the target to be tracked and the expected imaging size to obtain an adaptive zoom level, the adaptive zoom level being used to maintain a preset display ratio of the target to be tracked in the PTZ camera image after responding to the PTZ control command; and combining the motion-compensated horizontal rotation angle, the motion-compensated vertical tilt angle, and the adaptive zoom level to generate the PTZ control command.

[0037] As a refinement of the above embodiments, when performing the step of adaptively adjusting the current zoom level according to the category of the target to be tracked and the expected imaging size to obtain an adaptive zoom level, the following implementation methods can also be adopted, but are not limited to: determining the target imaging size of the target to be tracked in the PTZ camera image according to the category of the target to be tracked and the preset display ratio; calculating the expected imaging size of the target to be tracked based on the motion-compensated PTZ pointing, the current zoom level, and the distance information between the target to be tracked and the PZT camera; comparing the expected imaging size with the target imaging size to generate a zoom adjustment amount, and updating the current zoom level according to the zoom adjustment amount to obtain the adaptive zoom level.

[0038] In a specific embodiment of the present invention, the process of performing motion compensation and adaptive zoom is as follows: The system first uses a uniform motion model to predict the position offset of the cargo ship to be tracked in the next 0.5 seconds (preset time) based on the position coordinate sequence of the cargo ship in the past 5 frames of video. The offset is +1.2° in the horizontal direction and +1.2° in the vertical direction. 0.3°; Simultaneously, the system reads that the current zoom level of the PTZ camera is 20x, and determines the motion compensation sensitivity coefficient to be 0.05 based on the preset sensitivity mapping relationship (e.g., sensitivity coefficient = 1 / current zoom level); then, the position offset is multiplied by this sensitivity coefficient to obtain the weighted compensation amount: horizontal compensation angle = 1.2° × 0.05 = 0.06°, vertical compensation angle = 0.3° × 0.05 = The system calculates a 0.015° adjustment and adds this compensation to the base PTZ control parameters obtained from the fixed camera coordinate mapping, forming the motion-compensated horizontal rotation angle and vertical tilt angle. Further, based on the cargo ship type (large bulk carrier) and preset display ratio (target height occupies 60% of the screen), the system determines the target imaging size in the PTZ camera image. Then, based on the motion-compensated PTZ pointing, the current zoom level (20x), and the distance information between the ship and the PTZ camera obtained through AIS (850 meters), the system calculates that the expected imaging size of the cargo ship is only 35% of the screen height. By comparing the expected imaging size with the target imaging size, the system generates a zoom adjustment amount and updates the current zoom level from 20x to 32x accordingly, obtaining an adaptive zoom level. Finally, the system combines the motion-compensated horizontal rotation angle, the motion-compensated vertical tilt angle, and the adaptive zoom level (32x), encapsulates them into a standard ONVIF PTZ control command, and sends it to the PTZ camera via the network, enabling it to accurately align with and clearly present the high-value target at a preset ratio, achieving stable and efficient intelligent tracking.

[0039] In some embodiments, the preset priority rule includes three priority levels of decision strategies. The first-level priority decision strategy selects a ship target of a specified category from the plurality of ship targets as the target to be tracked. The second-level priority decision strategy continues tracking of the already tracked ship target if no ship target of the specified category exists among the plurality of ship targets. The third-level priority decision strategy selects a ship target from the remaining ship targets according to a default strategy if the already tracked ship target is lost. The first-level priority decision strategy has a higher priority than the second-level priority decision strategy, which in turn has a higher priority than the third-level priority decision strategy. The default strategy includes any one of an area-first strategy, a duration-of-existence strategy, or a confidence-first strategy. The area-first strategy selects the ship target with the largest detection box area; the duration-of-existence strategy selects the ship target that has appeared in the monitoring screen for the longest time; and the confidence-first strategy selects the ship target with the highest confidence score.

[0040] As a refinement of the above embodiments, after executing step 105, which involves controlling the PTZ camera to track the target based on the PTZ control command, the method may also adopt, but is not limited to, the following implementation methods: for example, real-time monitoring of the position of the target in the PTZ camera video stream, and dynamic updating of the PTZ control parameters according to the position change of the target to form closed-loop tracking control.

[0041] This application provides a hardware architecture system applying the method of this application, specifically including: a fixed monitoring subsystem comprising a fixed network camera installed at a high point in the water area, responsible for panoramic video acquisition. This camera is connected to the processing system via gigabit Ethernet, providing a high-definition video stream with a resolution of 2560×1440. A PTZ tracking subsystem comprises a high-speed dome network camera with pan-tilt-zoom (PTZ) rotation, lens tilt, and optical zoom functions. This camera receives system control commands and executes precise PTZ actions via an independent network connection. A central processing unit is an industrial control computer equipped with a high-performance GPU processor. This unit connects both the fixed camera and the PTZ camera simultaneously via a switch, enabling the reception of video streams and the transmission of PTZ control commands.

[0042] Corresponding to the aforementioned hardware architecture system, this application provides a detailed description of ship tracking based on the hardware architecture system, including: a hardware detection and connection stage, where the system automatically scans camera devices within the network, acquires camera parameters through standard protocols, and establishes video stream connections. Fixed cameras and PTZ cameras establish independent RTSP connections, setting corresponding resolutions, frame rates, and encoding formats. In the algorithm model loading stage, the system loads the pre-trained YOLOv8 ship detection model into GPU memory and initializes the multi-object tracker parameters. Simultaneously, it configures key parameters such as detection confidence threshold and intersection-over-union (IoU) threshold to prepare for real-time processing. In the calibration parameter verification stage, the system reads coordinate mapping parameters obtained beforehand through multi-point calibration to verify the effectiveness of the polynomial fitting coefficients. This ensures that the mapping relationship between fixed camera coordinates and PTZ parameters is accurate and reliable. The core real-time processing flow includes: (1) Video acquisition and preprocessing stage: A fixed camera continuously acquires video streams, employing an intelligent frame sampling strategy to balance the processing load. The acquired frames undergo color space conversion and size standardization to provide standardized input data for subsequent object detection. (2) Object detection and recognition stage: The system uses a YOLO model to perform inference analysis on the preprocessed video frames. The model identifies ship targets in the scene, outputting the bounding box coordinates, confidence score, and category information for each target, providing basic data for multi-target tracking. The system uses the YOLOv8 model for ship target detection, which is based on a deep convolutional neural network architecture. The forward propagation process can be represented as: P = Model(I_normalized) Where I_normalized represents the normalized input image after color space conversion, size scaling, and pixel normalization; Model represents the pre-trained YOLOv8 deep convolutional neural network model; P The detection results output by the model include the bounding box coordinates, class labels, and detection confidence scores for each ship target, which are used for subsequent tracking and decision-making.

[0043] In the multi-target tracking management phase, the system assigns a unique identifier to each detected target and maintains the target's state information and trajectory. It matches the same target in consecutive frames using a data association algorithm, updates the target's position, and manages the target's lifecycle. In the intelligent target selection decision phase, the system employs a three-level priority strategy to select a tracking object from multiple targets. First, it filters priority targets of specified categories. For example, in some ports, cargo ships are important monitoring targets. When there are multiple types of ships in the port (cargo ships, navigation aids, passenger ships, law enforcement ships, garbage collection ships), cargo ships are selected first for tracking. Second, it considers target continuity and finally selects the most prominent target as the tracking object according to a preset strategy. The three-level priority strategy adopts a configurable multi-strategy architecture, providing multiple preset selection strategies to adapt to the needs of different monitoring scenarios. The system supports dynamic switching of strategies through configuration files or a management interface, ensuring optimal target selection results in different application environments. Specific implementation schemes of the preset selection strategies include: Strategy 1: Area-first strategy based on spatial saliency. This strategy uses the visual saliency of the target in the image space as the core selection criterion, prioritizing the target with the largest detection box area as the tracking object. The technical principle behind this strategy is that larger targets are usually closer to the camera or more prominent in the scene, thus possessing higher observation value and tracking stability. This strategy is suitable for routine monitoring scenarios where prominent targets require focused attention. Strategy Two: A duration-based strategy. This strategy selects targets based on their duration of presence in the monitoring scene, prioritizing those with the longest duration of presence. Its technical advantage lies in the fact that targets with long durations often have more stable trajectories and higher scene relevance, helping to reduce target switching frequency and improve the continuity of system tracking. This strategy is particularly suitable for specific application scenarios requiring stable and continuous observation. Strategy Three: A confidence-based strategy based on recognition reliability. This strategy uses the recognition confidence score of the target detection algorithm as the core selection criterion, prioritizing targets with the highest detection confidence score as tracking targets. Its technical value lies in the fact that high-confidence targets have higher recognition accuracy, effectively reducing the impact of false detections on system performance and improving overall tracking reliability. This strategy is suitable for precision monitoring scenarios with high requirements for target recognition accuracy. In the PTZ parameter intelligent calculation stage (corresponding to step 103 and its detailed explanation), the system maps the coordinates of the selected target to PTZ control parameters using a polynomial fitting algorithm. Combining the target motion direction analysis results, appropriate parameter compensation is applied, and a suitable zoom level is determined based on the target type.

[0044] The system uses a polynomial fitting algorithm to map the target coordinates in a fixed camera to PTZ control parameters. The specific implementation of the polynomial fitting algorithm in PTZ control is as follows: Polynomial basis function construction: The system uses a complete cubic polynomial as the basis function space, defined as follows: The basis function vector includes: Φ(x, y) = [φ0(x,y), φ1(x,y), ..., φ9(x,y)]

[0045] The basis functions are as follows: φ0(x,y) = 1; φ1(x,y) = x; φ2(x,y) = y; φ3(x,y) = x²; φ4(x,y) = x·y; φ5(x,y) = y²; φ6(x,y) = x³; φ7(x,y) = x²·y; φ8(x,y) = x·y²; φ9(x,y) = y³.

[0046] In the PTZ control parameter mapping, x and y Φ represents the normalized coordinates of the target to be tracked in the image from a fixed camera; x , y ) is a complete cubic polynomial basis function vector consisting of 10 basis functions, used to nonlinearly map image coordinates to the PTZ control space; φ 0 to φ 9: These are the 10 monomial basis functions that constitute this cubic polynomial space, corresponding to the constant term, the linear term (x, y), the quadratic cross term (x², xy, y²), and the cubic term (x³, x²y, xy², y³), which together form a complete third-order polynomial expression for the Pan or Tilt angle with respect to the image coordinates. Multiplying this basis function vector with the pre-calibrated coefficient vector will yield the corresponding horizontal rotation angle (Pan) and vertical tilt angle (Tilt).

[0047] Coordinate normalization: Before applying polynomial fitting, the image coordinates are first normalized. x_n=(x-0.5) / W y_n=(y-0.5) / H Where W=2560 and H=1440 are the image dimensions, corresponding to width and height respectively, and the normalized coordinate range is approximately [-0.5, 0.5].

[0048] Ridge Regression Model Establishment: Objective Function:

[0049] Where: X∈R^{m×10} is the feature matrix, m=11 calibration points, y∈R^m is the target PTZ parameter vector (Pan or Tilt), θ∈R 10 Let λ be the coefficient vector to be determined, λ = 10-3 The regularization parameter is used. The objective function achieves a stable and robust estimation of nonlinear mapping relationships by minimizing the weighted sum of the squared error between the predicted and measured values ​​and the L2 norm of the coefficient vector.

[0050] Solution to the normal equation:

[0051] in, λ =10 3 The ridge regression regularization parameter is used to enhance the matrix. X X To ensure reversibility and prevent overfitting; I It is a 10th order identity matrix, ensuring X X + λI Positive definite and invertible; this formula directly obtains the optimal mapping coefficients by analytically solving for the minimum point of the ridge regression objective function. θ This enables high-precision, robust nonlinear mapping from image coordinates to PTZ control parameters.

[0052] Parameter prediction process: For new target coordinates (x, y), the PTZ parameter prediction formula includes horizontal rotation angle prediction and vertical tilt angle prediction. The horizontal rotation angle prediction is as follows:

[0053] Wherein, Pan represents the horizontal rotation control amount of the PTZ camera; θ_p It is a 10-dimensional column vector containing polynomial coefficients for calculating Pan, obtained through ridge regression calibration; θ_p [ i ] indicates the first element in the vector. i One element; ( x_n , y_n ) is derived from normalized image coordinates ( x_n , y_n The vector of complete cubic polynomial basis functions constructed; _i ( x_n , y_n ) is the first in the vector i One basis function; x_n and y_n Original pixel coordinates x , y The value after normalization.

[0054] Vertical tilt angle prediction is:

[0055] Where Tilt represents the vertical tilt control amount of the PTZ camera; θ t It is a 10-dimensional coefficient vector obtained through ridge regression calibration. θ t [ i ] for its first i One element; ( x n , y n ) is derived from normalized coordinates ( x n , y n The vector of complete cubic polynomial basis functions; where x n =( x 0.5) / 2560、 y n =( y 0.5) / 1440 is the original pixel coordinates ( x , y Normalized to approximately [ Values ​​within the range of 0.5, 0.5.

[0056] Complete PTZ parameter calculation process: Pan=θ_p0+θ_p1·x_n+θ_p2·y_n+θ_p3·x_n²+θ_p4·x_n·y_n+θ_p5·y_n²+θ_p6·x_n³+θ_p7·x_n²·y_n+θ_p8·x_n·y_n²+θ_p9·y_n³ Tilt=θ_t0+θ_t1·x_n+θ_t2·y_n+θ_t3·x_n²+θ_t4·x_n·y_n+θ_t5·y_n²+θ_t6·x_n³+θ_t7·x_n²·y_n+θ_t8·x_n·y_n²+θ_t9·y_n³ This polynomial fitting algorithm establishes a high-precision coordinate mapping relationship through a finite number of calibration points, providing a core mathematical foundation for dual-camera collaborative tracking.

[0057] In one optional implementation of this application embodiment, the adaptive zoom can also be achieved using, but is not limited to, the following methods: calculating the relative size ratio of the detection box of the target to be tracked in the fixed camera image; calculating a base zoom level based on the relative size ratio using a linear mapping relationship, wherein the linear mapping relationship is configured such that the base zoom level is at its maximum value when the relative size ratio is minimum, and at its minimum value when the relative size ratio is maximum; the base zoom level decreases linearly as the relative size ratio increases; and the current zoom level of the PTZ camera is replaced with the base zoom level to achieve the adaptive zoom.

[0058] In one possible implementation of the embodiments of this application, the motion compensation can also be achieved in the following ways, but not limited to: determining the motion trend of the target to be tracked based on the historical position data of the target; applying feedforward compensation to the horizontal rotation angle and / or vertical tilt angle of the PTZ control parameters according to the motion trend to perform the motion compensation and suppress tracking deviation.

[0059] To facilitate understanding of the aforementioned adaptive zoom and motion compensation process, this embodiment provides further explanation in conjunction with the aforementioned hardware architecture system, including: The basic principle of zoom control is as follows: the system dynamically adjusts the zoom level based on the relative size of the target detection box in the image to achieve adaptive magnification observation. The core idea of ​​zoom control is: the smaller the target size, the higher the zoom level, to ensure that important targets obtain sufficient observation details in the detail camera. The system first calculates the relative size ratio of the target detection box in the entire image: r = (target width / image width + target height / image height) / 2 This formula obtains a normalized size index between 0 and 1 by taking the average of the relative proportions of width and height, where 0 represents the smallest target and 1 represents the largest target.

[0060] Magnification level mapping: Based on relative size indices, the system calculates the basic magnification level through linear mapping. Z = 1 + 3 × (1 - r) This relationship ensures that: when a very small target is detected (r close to 0), the system automatically uses 4x high zoom for detail capture; when the target size occupies half the image (r=0.5), the system uses 2.5x medium zoom; and when the target size is large (r close to 1), the system uses 1x base zoom to maintain a wide field of view. During the control command execution and feedback phase, the system encapsulates the calculated PTZ parameters into standard control commands and sends them to the PTZ camera via HTTP. The system monitors the command execution status to ensure the camera is accurately aimed at the selected target.

[0061] In some embodiments, the system includes a health monitoring and maintenance mechanism, specifically including: a health status monitoring mechanism that continuously monitors the operational status of each system component, including camera connection status, processing performance, and resource usage; the system periodically generates operational reports to provide data support for maintenance management; an anomaly handling mechanism that provides automatic recovery and degradation solutions for potential issues such as video stream interruptions and algorithm failures; a reconnection mechanism and backup algorithms to ensure continuous operation under abnormal conditions; and a data recording and analysis mechanism that records detailed system operation logs, target detection data, and performance statistics; and the system saves key video clips as needed, providing a data foundation for post-event analysis and system optimization.

[0062] In summary, the embodiments of this application can achieve the following effects, including: 1. By integrating a dual-camera collaborative architecture with intelligent algorithms, significant performance improvements were achieved in multi-target vessel tracking scenarios. The system can automatically identify and prioritize tracking important vessel targets, greatly improving the utilization efficiency of monitoring resources. Through high-precision mapping algorithms and motion compensation mechanisms, the accuracy of PTZ control and the tracking stability of moving targets were effectively improved, significantly reducing the target loss rate.

[0063] 2. The adaptive zoom control mechanism intelligently adjusts the observation details based on the target type, ensuring high-definition detail capture of key targets while expanding the monitoring range. The method boasts a high degree of automation, significantly reducing the need for manual intervention and achieving long-term continuous and stable operation. The dual-camera architecture successfully resolves the contradiction between wide-area monitoring and detailed tracking in traditional single-camera systems, providing a comprehensive and effective technical solution for water area monitoring.

[0064] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a ship tracking device, the structure of which is as follows: Figure 2 As shown.

[0065] Figure 2 This is a schematic diagram of the internal structure of a ship tracking device provided in an embodiment of this application. Figure 2 As shown, the device includes: At least one processor 201; And a memory 202 that is communicatively connected to at least one processor; The memory 202 stores instructions executable by at least one processor. These instructions are executed by at least one processor 201 to enable the processor 201 to: acquire wide-area video of the monitored water area using a fixed camera; analyze the acquired video frames in real time based on a deep learning detection model to identify the category of each vessel target in the last acquired video frame, the position coordinates of each vessel target in the video frame, and the confidence level of the category; select one vessel target from the plurality of vessel targets as the target to be tracked based on a preset priority rule; perform mapping calculation on the position coordinates to obtain the PTZ control parameters corresponding to the position coordinates; perform motion compensation on the PTZ control parameters according to the motion trajectory of the target to be tracked, and perform adaptive zooming according to the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation, generating PTZ control instructions; and control the PTZ camera to track the target to be tracked based on the PTZ control instructions.

[0066] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium stores computer-executable instructions, which are configured to: acquire wide-area video of a monitored water area using a fixed camera, and analyze the acquired video frames in real time based on a deep learning detection model to identify the category of each vessel target in the last acquired video frame, the position coordinates of each vessel target in the video frame, and the confidence level of the category; select one vessel target from the plurality of vessel targets as the target to be tracked based on a preset priority rule; perform mapping calculation on the position coordinates to obtain PTZ control parameters corresponding to the position coordinates; perform motion compensation on the PTZ control parameters according to the motion trajectory of the target to be tracked, and perform adaptive zooming according to the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing after motion compensation, thereby generating PTZ control instructions; and control the PTZ camera to track the target to be tracked based on the PTZ control instructions.

[0067] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0068] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0069] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0071] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0073] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0074] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0075] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0076] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0077] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A ship tracking method, characterized in that, The method includes: Wide-area video is captured by fixed cameras over the monitored waters, and the captured video frames are analyzed in real time based on a deep learning detection model to identify the category of each vessel target in the last captured video frame, the position coordinates of each vessel target in the video frame, and the confidence level of the category. Based on a preset priority rule, a ship target is selected from the plurality of ship targets as the target to be tracked; The position coordinates are mapped and calculated to obtain the PTZ control parameters corresponding to the position coordinates; Motion compensation is performed on the PTZ control parameters based on the motion trajectory of the target to be tracked, and adaptive zoom is performed based on the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation, and PTZ control commands are generated. The PTZ control commands are used to control the PTZ camera to track the target.

2. The ship tracking method according to claim 1, characterized in that, The mapping calculation of the position coordinates to obtain the PTZ control parameters corresponding to the position coordinates includes: The position coordinates are normalized to obtain normalized position coordinates; The normalized position coordinates and polynomial coefficients are input into a preset cubic polynomial function, and the horizontal rotation angle and vertical tilt angle of the PTZ camera are calculated by mapping. The PTZ control parameters include the horizontal rotation angle and the vertical tilt angle. The polynomial coefficients are obtained by fitting a preset number of calibration data with a cubic polynomial through a ridge regression model.

3. The ship tracking method according to claim 2, characterized in that, The step of performing motion compensation on the PTZ control parameters based on the motion trajectory of the target to be tracked, and performing adaptive zooming based on the category of the target to be tracked and the expected imaging size of the target to be tracked under the PTZ pointing direction after motion compensation, and generating PTZ control commands includes: Predict the position offset of the target at a future preset time based on the movement trajectory of the target; The motion compensation sensitivity coefficient is determined based on the current zoom level of the PTZ camera; The position offset is weighted based on the sensitivity coefficient, and motion compensation is performed on the PTZ control parameters based on the weighting result. Based on the category of the target to be tracked and the expected imaging size, the current zoom level is adaptively adjusted to obtain an adaptive zoom level. The adaptive zoom level is used to ensure that the target to be tracked maintains a preset display ratio in the PTZ camera image after responding to the PTZ control command. The motion-compensated horizontal rotation angle, the motion-compensated vertical tilt angle, and the adaptive zoom level are combined to generate the PTZ control command.

4. The ship tracking method according to claim 3, characterized in that, The step of adaptively adjusting the current zoom level based on the category of the target to be tracked and the expected imaging size to obtain an adaptive zoom level includes: The target imaging size of the target to be tracked in the PTZ camera image is determined according to the category of the target to be tracked and the preset display ratio; Based on the motion-compensated PTZ pointing, the current zoom level, and the distance information between the target to be tracked and the PZT camera, the expected imaging size of the target to be tracked is calculated; The expected imaging size is compared with the target imaging size to generate a zoom adjustment amount, and the current zoom level is updated according to the zoom adjustment amount to obtain the adaptive zoom level.

5. The ship tracking method according to claim 1, characterized in that, The method further includes: Calculate the relative size ratio of the detection box of the target to be tracked in the fixed camera image; Based on the relative size ratio, a basic zoom level is calculated through a linear mapping relationship, wherein the linear mapping relationship is configured such that when the relative size ratio is at its minimum, the basic zoom level takes its maximum value; when the relative size ratio is at its maximum, the basic zoom level takes its minimum value; and the basic zoom level decreases linearly as the relative size ratio increases. The current zoom level of the PTZ camera is replaced with the base zoom level to achieve adaptive zoom.

6. The ship tracking method according to claim 1, characterized in that, Its features are, The method further includes: The movement trend of the target to be tracked is determined based on the historical location data of the target to be tracked. Based on the motion trend, feedforward compensation is applied to the horizontal rotation angle and / or vertical tilt angle of the PTZ control parameters to perform motion compensation and suppress tracking deviation.

7. A ship tracking method according to any one of claims 1-6, characterized in that, The preset priority rules include three priority levels of decision-making strategies. The first-level priority decision-making strategy selects a ship target of a specified category from the plurality of ship targets as the target to be tracked. The second-level priority decision-making strategy continues to track the already tracked ship target if no ship target of the specified category exists among the plurality of ship targets. The third-level priority decision-making strategy selects a ship target from the remaining ship targets according to the default strategy if the already tracked ship target is lost. The priority of the first-level priority decision-making strategy is higher than that of the second-level priority decision-making strategy, which is higher than that of the third-level priority decision-making strategy. The default strategy includes any one of the area-first strategy, the existence duration strategy, and the confidence-first strategy. The area-first strategy selects the ship target with the largest detection box area, the existence duration strategy selects the ship target that appears in the monitoring screen for the longest time, and the confidence-first strategy selects the ship target with the highest confidence score.

8. A ship tracking method according to claim 7, characterized in that, After controlling the PTZ camera to track the target based on the PTZ control command, the method includes: The position of the target to be tracked in the video stream of the PTZ camera is monitored in real time, and the PTZ control parameters are dynamically updated according to the position changes of the target to form a closed-loop tracking control.

9. A ship tracking device, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a ship tracking method as described in any one of claims 1-8.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement a ship tracking method as described in any one of claims 1-8.

Images