Ship Mobility Tag Tracking Method Based on Video Recognition and AIS Data Fusion

Abstract

The application discloses a ship mobile label tracking method based on video recognition and AIS data fusion, which comprises the following steps: 1, converting the AIS latitude and longitude uploaded by a ship into Cartesian coordinate data; 2, transmitting the Cartesian coordinate data to a video recognition device; 3, predicting the position of the ship; 4, converting the predicted coordinate position into image coordinates; 5, converting the image coordinates into screen coordinates; 6, obtaining the position of the ship in the screen; and 7, binding the mobile label with the screen coordinates of the ship to realize the tracking and hanging of the mobile label. The application provides a ship mobile label tracking method based on video recognition and AIS data fusion, which can not only monitor the near and far distance ships on the sea, but also can simplify the monitoring tracking and realization difficulty and reduce the cost through the fusion of video recognition and AIS data, so as to realize the tracking and monitoring of the passing ships in a mobile label automatic tracking mode.

Description

Technical Field

[0001] This invention relates to the field of ship monitoring technology, and in particular to a ship movement tag tracking method based on video recognition and AIS data fusion. Background Technology

[0002] Currently, there are generally two methods for monitoring ships at sea. One is a maritime ship detection method based on machine vision and multi-source data fusion. This method combines the Yolov5 target detection algorithm, radar, ordinary monitoring equipment, and AIS system to achieve maritime ship monitoring.

[0003] Another method for maritime vessel monitoring is an adaptive weighted data fusion method based on vision and multi-source radar. Similar to the first method, this approach mainly utilizes the position information of the target vessel obtained from image information acquired by visual sensors and the position information of the target vessel obtained from data information acquired by radar sensors, and then fuses the two to improve the accuracy of fusion between visual information and radar / AIS position information.

[0004] Both of the above-mentioned methods for monitoring ships at sea have the following drawbacks:

[0005] First, the above solution requires the integration of data from multiple devices, making system deployment complex and costly.

[0006] Second, the video surveillance equipment in the above-mentioned scheme can only monitor ships at close range. The equipment has poor monitoring capabilities and cannot obtain information about distant targets.

[0007] Third, because radar cannot determine the type of obstacle, it is also unable to track and locate ships at a distance.

[0008] Fourth, the accuracy of the above method is difficult to guarantee. When ships are close to each other, it is easy to identify ships that are close to each other as one target.

[0009] 5. AIS data updates slowly, making it easy for ship tracking to drift. Summary of the Invention

[0010] The main objective of this invention is to overcome the aforementioned technical deficiencies and provide a ship mobile tag tracking method based on video recognition and AIS data fusion, which can not only monitor ships at near and far distances, but also simplify monitoring and tracking and reduce implementation difficulty and costs by integrating video recognition with AIS data, thereby achieving automatic tracking and monitoring of passing ships.

[0011] To achieve the above-mentioned objectives, this invention proposes a ship movement tag tracking method based on video recognition and AIS data fusion, comprising the following methods:

[0012] Step 1) Convert the AIS latitude and longitude data uploaded by the ship into Cartesian coordinate data in the coordinate system of the gimbal installation location;

[0013] Step 2) Transmit the Cartesian coordinate data to the coordinate system of the video recognition device;

[0014] Step 3) Predict the location of the ship based on the coordinate data from the video recognition device;

[0015] Step 4) Convert the predicted coordinates into image coordinates;

[0016] Step 5) Convert the image coordinates to screen coordinates, and calculate the ship's position on the screen using the ship's AIS latitude and longitude data;

[0017] Step 6) Perform linear fusion calculation on the ship's position on the screen calculated from AIS data and the ship's position on the screen obtained from the video recognition device to determine the position where the ship should be on the screen;

[0018] Step 7) After obtaining the real-time ship screen coordinates, bind the moving tag to the ship screen coordinates to realize the tracking and tagging of the moving tag.

[0019] The video recognition device includes an AR eagle eye recognition device.

[0020] In step 1), the ship uploads AIS data, and the AR Eagle Eye recognition device identifies the ship entering the monitoring area.

[0021] Step 3) includes predicting the ship's coordinate position in the AR Eagle Eye camera coordinate system at the next moment, thus predicting the ship's camera coordinate position at the next moment.

[0022] Step 6) includes fusing the coordinates of the ship on the screen detected by the AR Eagle Eye recognition device with the coordinates of the ship on the screen calculated by the latitude and longitude coordinates of AIS to obtain the real-time position of the ship on the screen.

[0023] Step 6) includes fusing the ship's latitude and longitude coordinates detected by the AR Eagle Eye recognition device with the latitude and longitude coordinates of the AIS to avoid abnormalities or drifting of the tags.

[0024] After calculating the ship's screen coordinates in step 7), the mobile tag is bound to the ship by limiting the Euclidean distance between the mobile tag and the ship at the screen coordinates. The ship's attributes are then displayed in the mobile tag by obtaining the ship's attribute data from the AIS data.

[0025] The beneficial effects of the technical solution provided by this invention are as follows: This invention monitors passing ships at sea based on video recognition equipment. By integrating the AIS data information of the ships and the video data of the AR Eagle Eye device, it automatically marks the ship number and other information of passing ships in the monitoring video, and enables the marked tags to continuously track and locate the ships. This invention can monitor ships at both near and far distances at sea, simplifying the monitoring and tracking process and reducing the difficulty of implementation. It achieves the tracking and monitoring of passing ships through automatic tracking of moving tags. The method of this invention is easy to operate, reduces costs, and is suitable for widespread promotion. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a ship movement tag tracking method based on video recognition and AIS data fusion according to an embodiment of the present invention;

[0028] Figure 2 This is another flowchart of a ship movement tag tracking method based on video recognition and AIS data fusion according to an embodiment of the present invention;

[0029] Figure 3 This is a block diagram illustrating the data acquisition and processing of a ship motion tag tracking method based on video recognition and AIS data fusion, according to an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram illustrating the latitude and longitude calculation of a ship movement tag tracking method based on video recognition and AIS data fusion according to an embodiment of the present invention.

[0031] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0033] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0034] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0035] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0036] This invention proposes a ship movement tag tracking method based on the fusion of video recognition and AIS data.

[0037] Reference Figure 1 In one embodiment of the present invention, the ship movement tag tracking method based on video recognition and AIS data fusion includes the following method:

[0038] Step 1) Convert the AIS latitude and longitude uploaded by the ship into Cartesian coordinate data S100 in the coordinate system of the gimbal installation location;

[0039] Step 2) Transmit the Cartesian coordinate data to the coordinate system of the video recognition device;

[0040] In this embodiment, preferably, the video recognition device includes an AR eagle eye recognition device; the AR eagle eye recognition device includes an AR eagle eye gimbal.

[0041] Step 2) includes performing coordinate transformation on the Cartesian coordinates under the AR Eagle Eye gimbal S200; and transmitting the transformed coordinate data to the AR Eagle Eye gimbal coordinates S300.

[0042] Step 3) Based on the coordinate data from the video recognition device, predict the location of the ship's appearance (S400).

[0043] Step 4) Convert the predicted coordinates to image coordinates S500;

[0044] Step 5) Convert the image coordinates to screen coordinates S600;

[0045] Step 5) Calculate the position of the ship on the screen using the ship's AIS latitude and longitude data, that is, calculate the coordinates S700 of the ship on the screen;

[0046] Step 6) Perform linear fusion calculation on the ship's position on the screen calculated from AIS data and the ship's position on the screen obtained from the video recognition device to obtain the ship's position on the screen S800.

[0047] Step 7) After obtaining the real-time ship screen coordinates, bind the moving tag to the ship screen coordinates to realize the tracking and tagging of the moving tag S900.

[0048] In this embodiment, preferably, in step 1), the ship uploads AIS data, and the AR Eagle Eye recognition device identifies the ship entering the monitoring area.

[0049] In this embodiment, AIS refers to Automatic Identification System (AIS); the latitude and longitude coordinates of AIS are the latitude and longitude coordinates of Automatic Identification System.

[0050] In this embodiment, the mobile tag monitoring video automatically labels the ship number and other information of passing ships; preferably, it is displayed in a window on the upper right of the ship.

[0051] In this embodiment, preferably, step 3) includes predicting the coordinate position of the ship in the AR Eagle Eye camera coordinate system at the next moment, and predicting the camera coordinate position of the ship at the next moment.

[0052] Step 3) includes predicting the ship's future coordinates in the AR Eagle Eye camera coordinate system, thus predicting the ship's camera coordinates for the next moment.

[0053] In this embodiment, preferably, step 6) involves fusing the coordinates of the ship on the screen detected by the AR Eagle Eye recognition device with the coordinates of the ship on the screen calculated by the latitude and longitude coordinates of AIS to obtain the real-time position of the ship on the screen.

[0054] See Figure 4 In this embodiment, the fusion calculation includes:

[0055] (1) The position of the ship in the coordinate system with AR Eagle Eye as the origin is calculated from the AIS coordinates:

[0056] Latitude eccentricity coefficient:

[0057] ;

[0058] Distance in the latitudinal direction:

[0059] ;

[0060] Distance along the longitude direction:

[0061] ;

[0062] in:

[0063] D is the latitudinal eccentricity coefficient;

[0064] ER is the Earth's equatorial radius;

[0065] PR is the Earth's polar radius;

[0066] Lat represents latitude;

[0067] Lon represents longitude;

[0068] AIS indicates the ship's position;

[0069] IL indicates the gimbal mounting location.

[0070] (2) There is a coordinate transformation relationship between the AR Eagle Eye and the gimbal installation position. Assume the transformation relationship is R and the translation relationship is T.

[0071] The ship's coordinates relative to the AR Hawkeye are:

[0072] ;

[0073] (3) Using the Holt index to predict the ship's next position:

[0074]

[0075]

[0076]

[0077] ;

[0078] in,

[0079] T represents the frequency at which the ship uploads AIS data to AR Eagle Eye;

[0080] Sf(t) Let X be the time series at period t. AR Y AR Smoothing value;

[0081] A f(t) The actual value of the time series at period t (

[0082] );

[0083] b f(t) For the time series X at period t AR Y AR The trend value;

[0084] F f(t+m) To predict the ship's position (X,Y) for period m based on the current time series at period t.

[0085] (4) Through the above process, the real-time predicted position of the ship in the AR Eagle Eye coordinate system can be obtained through the ship's AIS information, and the predicted coordinates can be further converted into image coordinates:

[0086] ;

[0087] (5) Convert image coordinates to pixel coordinates:

[0088] ;

[0089] In this embodiment, the linear fusion calculation includes obtaining the ship's position on the screen calculated from the ship's AIS data. The ship's position on the screen calculated by the SuperBrain AI (denoted as u) is then used... AI ,v AI The position of the ship on the screen as predicted by AIS (set as u). AIS ,v AIS Perform linear fusion, and the final result (let's call it u, v) is:

[0090]

[0091] ;

[0092] In this embodiment, preferably, step 6) includes fusing the ship's latitude and longitude coordinates detected by the AR Eagle Eye recognition device with the latitude and longitude coordinates of the AIS to avoid abnormalities or drifting of the tags.

[0093] Step 6) includes fusing the ship's latitude and longitude coordinates detected by the AR Eagle Eye recognition device with the latitude and longitude coordinates of AIS to avoid situations where multiple ships are identified as one when there are many ships, or where the tags are abnormal due to ships being too close together (a ship following multiple moving tags, tag attributes not matching the ship, etc.), or where the tags drift.

[0094] In this embodiment, preferably, after calculating the screen coordinates of the ship in step 7), the mobile tag is bound to the ship by limiting the Euclidean distance between the mobile tag and the ship at the screen coordinates, and the ship attribute data is displayed in the mobile tag by obtaining the ship attribute data from the AIS data.

[0095] In this embodiment, the Euclidean distance includes:

[0096] In this embodiment, the UV coordinates of the moving tag on the screen are:

[0097] ,

[0098] The ship's UV coordinates on the screen are (u, v). Let p be the Euclidean distance between the two coordinates. The process of binding the moving tag to the ship is the process of restricting p within a certain range, where C is an arbitrary constant. The calculation formula is as follows:

[0099] .

[0100] See Figure 3 The data acquisition in this invention is mainly based on obtaining the ship's AIS data through network communication and obtaining real-time video streams of passing ships through AR eagle-eye cameras.

[0101] The data processing and display section mainly involves the AR Eagle Eye recognition device using the acquired ship AIS data and real-time ship monitoring video stream. Through coordinate transformation algorithms, it fuses the ship's AIS coordinate data and AR camera coordinate data to calculate the real-time screen coordinates of the ship tag.

[0102] See Figure 2 The method for acquiring data according to the present invention includes the following steps:

[0103] Step 1), the AIS base station sends data S800;

[0104] Step II) Receive data S810 through the AIS data receiving module;

[0105] Step III) Filter the AIS data using S820;

[0106] Step IV) The AIS data is analyzed and processed by the business data management module S830;

[0107] Step V), the AIS data is used by the tag management module to generate tag information S840;

[0108] Step VI) Identify the tag information using the AR Eagle Eye recognition device S850; further, perform AI recognition of the ship using the identified tag information S851;

[0109] Step VII) Calculate the ship's position S860 using the ship tag position calculation module;

[0110] Step VIII) Display the ship's position through the ship display module.

[0111] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Various changes and improvements can be made to the present invention without departing from the spirit and scope of the present invention. All equivalent structural transformations made using the contents of the present invention specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for tracking ship movement tags based on video recognition and AIS data fusion, characterized in that: Including the following methods: Step 1) Convert the AIS latitude and longitude uploaded by the ship into Cartesian coordinate data in the coordinate system of the gimbal installation location; the video recognition device includes AR eagle eye recognition equipment; In step 1), the ship uploads AIS data, and the AR Eagle Eye recognition device identifies the ship entering the monitoring area. Step 2) Transmit the Cartesian coordinate data to the coordinate system of the video recognition device; Step 3) Predict the location of the ship based on the coordinate data from the video recognition device; Step 3) includes predicting the ship's coordinate position in the AR Eagle Eye camera coordinate system at the next moment, and predicting the ship's camera coordinate position at the next moment. Step 4) Convert the predicted coordinates into image coordinates; Step 5) Convert the image coordinates to screen coordinates, and calculate the ship's position on the screen using the ship's AIS latitude and longitude data; Step 6) Perform linear fusion calculation on the ship's position on the screen calculated from AIS data and the ship's position on the screen obtained from the video recognition device to determine the position where the ship should be on the screen; Step 6) includes fusing the coordinates of the ship on the screen detected by the AR Eagle Eye recognition device with the coordinates of the ship on the screen calculated by the latitude and longitude coordinates of AIS to obtain the real-time position of the ship on the screen. Fusion computing includes steps A, B, C, D, and E: Step A: Calculate the ship's position in a coordinate system with AR Eagle Eye as the origin using AIS coordinates: Latitude eccentricity coefficient: ； Distance in the latitudinal direction: ； Distance along the longitude direction: ； in: D is the latitudinal eccentricity coefficient; ER is the Earth's equatorial radius; PR is the Earth's polar radius; Lat represents latitude; Lon represents longitude; AIS indicates the ship's position; IL indicates the installation location of the pan-tilt unit; Step B: There is a coordinate transformation relationship between the AR Eagle Eye and the gimbal mounting position. Let the transformation relationship be R and the translation relationship be T. The ship's coordinates relative to the AR Hawkeye are: ； Step C: Predict the ship's next position using the Holt index: ； in, T' uploads data from the ship to the AR Eagle Eye system. Data frequency; For the time series at period t Smoothing value; The actual value of the time series at period t ( ); For the time series at period t The trend value; To predict the ship position for period m based on the current time series at period t. ; Step D: After the above process, the real-time predicted position of the ship in the AR Eagle Eye coordinate system can be obtained through the ship's AIS information, and the predicted coordinates can be further converted into image coordinates: ； Step E: Convert image coordinates to pixel coordinates: ； The linear fusion calculation includes obtaining the ship's position on the screen calculated from the ship's AIS data; setting the ship's position on the screen calculated by the SuperBrain AI as ( The positions of the ships on the screen as predicted by AIS are set to ( Linear fusion was performed, and the final result was... : ； Step 6) includes fusing the ship's latitude and longitude coordinates detected by the AR Eagle Eye recognition device with the latitude and longitude coordinates of the AIS to avoid abnormalities or drifting of the tags. Step 7) After obtaining the real-time ship screen coordinates, bind the moving tag to the ship screen coordinates to realize the tracking and tagging of the moving tag.

2. The ship movement tag tracking method based on video recognition and AIS data fusion according to claim 1, characterized in that: After calculating the ship's screen coordinates in step 7), the mobile tag is bound to the ship by limiting the Euclidean distance between the mobile tag and the ship at the screen coordinates. The ship's attributes are then displayed in the mobile tag by obtaining the ship's attribute data from the AIS data.

Images