Ship navigation trajectory sensing method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a ship navigation track sensing method and system, electronic equipment and a storage medium, and belongs to the technical field of ship sensing enhancement. According to the method, first navigation tracks of multiple ships in a detectable range are obtained through a distance detection sensor to obtain a first data set, and second navigation tracks of the multiple ships are obtained through an automatic ship identification system to obtain a second data set; and then carrying out correlation analysis on the tracks of the first data set and the second data set based on a track point matching algorithm, and correspondingly removing the correlated first navigation track and second navigation track from the data sets according to a correlation result so as to reduce the subsequent calculation amount. The trajectory similarity between every two navigation trajectories in the first data set and the second data set is calculated, the first navigation trajectory and the second navigation trajectory which are matched with each other are determined according to the trajectory similarity, the first navigation trajectory which is matched with each other is corrected according to the second navigation trajectory, and therefore the accuracy of ship navigation trajectory detection is improved.

Description

Technical Field

[0001] The present application relates to the field of ship perception enhancement technology, and in particular to a ship navigation track perception method, system, electronic device and storage medium. Background Art

[0002] Video images can provide ships with real-time, intuitive environmental information, helping crew members to more accurately understand the surrounding environment and make more reasonable decisions. In ship applications, cameras or radars installed on the ship can be used for distance detection to determine the tracks of other ships on the sea, thereby preventing collisions. However, natural factors such as wind, waves and currents can cause the hull to shake violently, making it difficult for cameras or radars installed on the ship to stably detect targets, which may affect the availability of data such as images, resulting in low accuracy in ship track recognition. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a ship navigation track perception method, system, electronic device and storage medium, aiming to improve the accuracy of ship navigation track detection.

[0004] To achieve the above purpose, an embodiment of the present application provides a method for sensing a ship's navigation track, comprising the following steps:

[0005] Acquire first navigation tracks of multiple ships within a detectable range through a distance detection sensor to obtain a first data set;

[0006] Acquire the second navigation tracks of the plurality of ships through a ship automatic identification system to obtain a second data set;

[0007] Based on a track point matching algorithm, performing an association analysis on a first navigation track in the first data set and a second navigation track in the second data set to obtain an association result, wherein the association result is used to associate the first navigation track and the second navigation track belonging to the same ship;

[0008] According to the association result, the associated first navigation track and the associated second navigation track are correspondingly removed from the first data set and the second data set;

[0009] Calculating the trajectory similarity between the first flight trajectory in the first data set and the second flight trajectory in the second data set;

[0010] A first navigation trajectory and a second navigation trajectory that match each other are determined according to the trajectory similarity, and the first navigation trajectory that matches each other is corrected according to the second navigation trajectory.

[0011] In some embodiments, the distance detection sensor is a binocular camera module, and obtaining the first navigation tracks of multiple ships within the detectable range through the distance detection sensor to obtain the first data set includes the following steps:

[0012] Acquire a first image and a second image for the same ship through the binocular camera module;

[0013] Determine a first navigation track of the target ship according to a first pixel position of the target ship in the first image, a second pixel position of the target ship in the second image, and a position parameter of the binocular camera module;

[0014] A first data set is formed according to first navigation tracks of a plurality of ships.

[0015] In some embodiments, the ship navigation track perception method further includes the following steps:

[0016] Determine a data set to be processed according to the track point acquisition frequency of the distance detection sensor and the track point acquisition frequency of the ship automatic identification system, wherein the data to be processed is a data set corresponding to the smaller track point acquisition frequency of the first data set and the second data set;

[0017] The navigation track in the data to be processed is interpolated by using the Lagrange interpolation method, so that the time interval between track points of the interpolated navigation track is the same as the time interval between track points of the navigation track in another data set.

[0018] In some embodiments, the method of performing an association analysis on the first navigation track in the first data set and the second navigation track in the second data set based on a track point matching algorithm to obtain an association result includes the following steps:

[0019] extracting any first navigation track in the first data set and any second navigation track in the second data set for correlation analysis;

[0020] Aligning the track points of the first navigation track with the track points of the second navigation track based on the track point collection time;

[0021] Calculate the distance between each pair of aligned trajectory points to obtain the trajectory pair distance;

[0022] When the distances of the track pairs with the expected number of proportions are all less than the first expected threshold, it is determined that the association result between the first navigation track and the second navigation track is mutually associated.

[0023] In some embodiments, the step of calculating the trajectory similarity between the first flight trajectory in the first data set and the second flight trajectory in the second data set comprises the following steps:

[0024] Calculating the similarity of movement direction, movement speed and trajectory position between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set;

[0025] The moving direction similarity, the moving speed similarity and the trajectory position similarity are weightedly calculated to obtain the trajectory similarity between the first navigation trajectory and the second navigation trajectory.

[0026] In some embodiments, determining the first navigation trajectory and the second navigation trajectory that match each other according to the trajectory similarity includes the following steps:

[0027] constructing a cost matrix according to the trajectory similarities between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set;

[0028] The cost matrix is ​​solved by using the Hungarian algorithm to obtain an optimal trajectory matching strategy for the first data set and the second data set, wherein the optimal trajectory matching strategy is used to determine a first navigation trajectory and a second navigation trajectory that match each other.

[0029] In some embodiments, the step of correcting the first navigation track that matches each other according to the second navigation track comprises the following steps:

[0030] Calculating the distance between each pair of mutually aligned track points in the first navigation track and the second navigation track that match each other to obtain a track pair distance;

[0031] Selecting a trajectory pair distance greater than a second expected threshold from all the trajectory pair distances as an abnormal trajectory pair;

[0032] A fused trajectory point is determined according to two trajectory points in the abnormal trajectory pair respectively from the first navigation trajectory and the second navigation trajectory, and the corresponding trajectory point in the first navigation trajectory is replaced by the fused trajectory point.

[0033] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a ship navigation track perception system, including:

[0034] The first module is used to obtain first navigation tracks of multiple ships within a detectable range through a distance detection sensor to obtain a first data set;

[0035] The second module is used to obtain the second navigation tracks of the plurality of ships through the ship automatic identification system to obtain a second data set;

[0036] a third module, configured to perform association analysis on the first navigation track in the first data set and the second navigation track in the second data set based on a track point matching algorithm to obtain an association result, wherein the association result is used to associate the first navigation track and the second navigation track belonging to the same ship;

[0037] A fourth module, configured to remove the associated first navigation track and the associated second navigation track from the first data set and the second data set respectively according to the association result;

[0038] A fifth module, configured to calculate the trajectory similarity between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set;

[0039] The sixth module is used to determine the first navigation trajectory and the second navigation trajectory that match each other according to the trajectory similarity, and to correct the first navigation trajectory that matches each other according to the second navigation trajectory.

[0040] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program implements the method described in the above embodiment when executed by the processor.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the above embodiment.

[0042] The ship navigation track perception method, system, electronic device and storage medium proposed in the present application obtain the first navigation tracks of multiple ships within the detectable range through a distance detection sensor to obtain a first data set, obtain the second navigation tracks of multiple ships through a ship automatic identification system to obtain a second data set, and then, based on a track point matching algorithm, perform correlation analysis on the first navigation track in the first data set and the second navigation track in the second data set to obtain a correlation result, and remove the correlated first navigation track and second navigation track from the first data set and the second data set respectively according to the correlation result, so as to reduce the calculation amount of subsequent similarity analysis and track matching. Calculate the track similarity between the first navigation track in the first data set and the second navigation track in the second data set, determine the mutually matching first navigation track and second navigation track according to the track similarity, and correct the mutually matching first navigation track according to the second navigation track, so as to improve the accuracy of ship navigation track detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a method for sensing a ship's navigation trajectory provided by an embodiment of the present application;

[0044] Figure 2 It is a schematic diagram of the components and their interaction relationships in the multi-source perception data comprehensive acquisition device provided in an embodiment of the present application;

[0045] Figure 3 is a schematic diagram of the appearance of a perception module provided in an embodiment of the present application;

[0046] Figure 4 It is an interface design and interface circuit diagram of a multi-source perception data comprehensive acquisition device provided in an embodiment of the present application;

[0047] Figure 5 It is a schematic diagram of the binocular ranging principle provided in an embodiment of the present application;

[0048] Figure 6 is a schematic diagram of navigation trajectory interpolation provided in an embodiment of the present application;

[0049] Figure 7 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] It should be noted that, although the functional modules are divided in the system and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0053] The following describes the nouns involved in the embodiments of the present application:

[0054] AIS data refers to the data collected and provided by the Automatic Identification System (AIS). AIS data mainly comes from ship-borne or aircraft-borne AIS equipment, which collects and transmits relevant information through satellites and ground base stations. In addition, AIS data can also be integrated with other navigation-related data, such as meteorological information, ocean flow information, etc., to provide more comprehensive navigation intelligence. AIS data includes real-time dynamic information and static information such as the position, speed, heading, ship name, call sign, MMSI (Maritime Mobile Service Identity), IMO (International Maritime Organization) number, ship type, draft, length, and ship width of various ships or aircraft.

[0055] The Hungarian Algorithm, also known as the Kuhn-Munkres algorithm, is a combinatorial optimization algorithm used to solve the optimal matching problem on a bipartite graph. It is usually applied to assignment problems, such as task assignment and job scheduling, where a matching strategy with the lowest cost needs to be found.

[0056] The embodiments of the present application provide a ship navigation track perception method, system, electronic device and storage medium, aiming to improve the accuracy of ship navigation track detection.

[0057] The ship navigation track perception method, system, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the ship navigation track perception method in the embodiments of the present application is described.

[0058] The ship navigation track perception method provided in the embodiment of the present application relates to the field of ship perception enhancement technology. The ship navigation track perception method provided in the embodiment of the present application can be applied to the terminal or to the server. In some embodiments, the terminal can be a device with computing functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, but is not limited to the above forms.

[0059] According to some embodiments of the present application, the ship navigation trajectory perception method of the embodiment of the present application can also be applied to a multi-source perception data comprehensive acquisition device. The components and interaction relationships in the multi-source perception data comprehensive acquisition device are as follows: Figure 2As shown, the device includes a perception module 100, a data fusion module 500, a storage module 600, a monitor module 700, a central controller module 800 and a data preprocessing module 900. The perception module includes a camera data acquisition module 120, a radar data acquisition module 130, and an AIS data acquisition module 110. The data preprocessing module 900 includes devices such as a PC and a router, and the storage module 600 includes a cloud storage module 610 and a local storage module 620.

[0060] Among them, the appearance of the perception module is as follows Figure 3 As shown, the camera data acquisition module 120 in the perception module is located below the radar data acquisition module 130, and the compass 140 is located below the camera. The three are connected by a metal bracket, and the AIS is stored inside the device.

[0061] Interface design and interface circuit of multi-source sensing data comprehensive acquisition device Figure 4 As shown, the markings are as follows: camera 3 connected to router 801, camera 2 connected to router 802, AIS antenna 803, GPS antenna 804, camera 1 connected to router 805, radar network perforation 806, radar power perforation 807 (24V), compass perforation 808, socket power perforation 809 (220V) and AIS host 810.

[0062] The data collection process of the multi-source sensing data comprehensive collection device is that the raw data collected by the AIS data collection module is first pre-processed in the data pre-processing module to remove duplicate, erroneous or invalid data, and the AIS data is parsed to extract key information to obtain static data and dynamic data. The data collected by AIS can be divided into two categories: static data and dynamic data: static data includes the basic characteristics of the ship, such as the length and width of the ship. The dynamic data includes the real-time changing information such as the current longitude and latitude, distance, speed, etc. of the ship. The compass can also be used to determine the ship's heading.

[0063] The radar data acquisition module obtains the original image through the received radar signal data. First, the data preprocessing module performs signal preprocessing through filtering and denoising. After that, the target can be identified in the image through methods such as image segmentation and feature extraction. Then the target is tracked, that is, the position and movement trajectory of the target are tracked between consecutive frames to identify targets such as ships. Finally, the target or image features are further analyzed to extract more useful information, such as the length, width, and movement status of the target.

[0064] The radar data acquisition module obtains the original image through the received radar signal data. First, the data preprocessing module performs signal preprocessing through filtering and denoising. After that, the target can be identified in the image through methods such as image segmentation and feature extraction. Then the target is tracked, that is, the position and movement trajectory of the target are tracked between consecutive frames to identify targets such as ships. Finally, the target or image features are further analyzed to extract more useful information, such as the length, width, and movement status of the target.

[0065] The portable multi-source perception data comprehensive acquisition device realizes the precise ranging function through three identical cameras. By locking the same target with multiple cameras, it can realize the precise ranging of the ship target, so as to integrate it with the AIS data more efficiently and accurately. By identifying the ship target through video, it can automatically modify the wrong information (ship type, length, width, etc.) in the AIS static information of the target ship. In order to integrate high-quality AIS data and visual target data, it is necessary to unify the different source data into the same coordinate system.

[0066] For example, in order to realize that the AIS track data and the image track data are in the same coordinate system, the pinhole model can be used to project the AIS data world coordinate system (WCS) to the pixel coordinate system (PCS). Before coordinate transformation, Mercator projection is first performed to realize forward geodetic calculation.

[0067] The projection coordinates (x, y) of the original position (U, V, W) of the AIS information in the PCS are calculated by formula (1):

[0068]

[0069] Among them, Z is the scale factor, K in is the internal parameter matrix of the camera, K ex is the external parameter matrix.

[0070] Since the camera position is fixed, K ex Set to the identity matrix, the internal parameter matrix K in This can be estimated using the pinhole model.

[0071] In another example, both the AIS track data and the image track data may be converted to the polar coordinate system, as follows:

[0072] Convert the AIS track data to the polar coordinate system. Assume that the longitude and latitude of the ship in the AIS is (lon s ,lat s ), the latitude and longitude position of the camera is (lon c ,lat c ), the distance r of the ship relative to the camera a :

[0073]

[0074] c=2*sain(sqrt(a)); (3)

[0075] r a =c*R*1000; (4)

[0076] Where R is the radius of the Earth.

[0077] The angle θ between the line formed by the ship and the camera and the true north direction a , after conversion, it is expressed as follows:

[0078] y=sin(lon s -lon c )*cos(lat s ); (5)

[0079] x=sin(lat s )*cos(lat c )-sin(lat c )*cos(lat s )*cos(lon s -lon c ); (6)

[0080]

[0081] After unifying the AIS tracks and CCTV tracks into the polar coordinate system, the CCTV track set is expressed as follows:

[0082]

[0083] The AIS track set is represented as follows:

[0084]

[0085] Among them, t ij represents the time of the jth track point of the ith track, r ij represents the distance of the jth track point of the ith track, θ ij represents the azimuth of the j-th track point of the ith track.

[0086] The ship navigation trajectory perception method of the embodiment of the present application does not impose any specific restrictions on the type of coordinate system in which the track data is located, as long as the two types of track data are located in the same coordinate system.

[0087] Figure 1 is an optional flow chart of the ship navigation track perception method provided in the embodiment of the present application, Figure 1 The method may include but is not limited to steps S101 to S106.

[0088] Step S101, obtaining first navigation tracks of multiple ships within a detectable range through a distance detection sensor to obtain a first data set;

[0089] Step S102, obtaining second navigation tracks of multiple ships through a ship automatic identification system to obtain a second data set;

[0090] Step S103, based on a track point matching algorithm, performing correlation analysis on the first navigation track in the first data set and the second navigation track in the second data set to obtain a correlation result, wherein the correlation result is used to correlate the first navigation track and the second navigation track belonging to the same ship;

[0091] Step S104, according to the association result, the associated first navigation track and the second navigation track are correspondingly removed from the first data set and the second data set;

[0092] Step S105, calculating the trajectory similarity between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set;

[0093] Step S106, determining the first navigation trajectory and the second navigation trajectory that match each other according to the trajectory similarity, and correcting the first navigation trajectory that matches each other according to the second navigation trajectory.

[0094] In step S101 of some embodiments, the distance detection sensor is mainly used to measure the distance of multiple ships within the detectable range. The distance detection sensor can be a coarse distance measurement sensor such as a visual camera (for example, a binocular camera module), or a radar. The distance detection sensor can be used to determine the distance of the target ship from the distance detection sensor. On this basis, the location of the target ship can be determined in combination with the longitude and latitude of the distance detection sensor itself, thereby determining the navigation track of the target ship. The navigation track obtained in this way is located in the world coordinate system.

[0095] In some embodiments, the distance detection sensor is a binocular camera module, and step S101 may include but is not limited to steps S201 to S203:

[0096] Step S201, acquiring a first image and a second image of the same ship through a binocular camera module;

[0097] Step S202, determining a first navigation track of the target ship according to a first pixel position of the target ship in the first image, a second pixel position of the target ship in the second image, and a position parameter of the binocular camera module;

[0098] Step S203: compose a first data set according to the first navigation tracks of the plurality of ships.

[0099] In this embodiment, the camera maps the objects in the objective world to a two-dimensional plane through rigid body transformation, perspective transformation, and affine transformation. First, the world coordinate system (X w , Y w , Z w ) is transformed into the camera coordinate system (X c , Y c , Z c ), then transformed into the image coordinate system (x, y) by perspective transformation, and finally transformed into the pixel coordinate system (u, v) in the two-dimensional plane by affine transformation.

[0100] This embodiment uses binocular camera module ranging technology to determine the trajectory point of the target ship, and then determine its navigation trajectory. Two cameras at different positions take the first image and the second image of the target point. There will be a parallax x between the first image and the second image. lelf -x right , find the parallax and then combine it with the principle of similar triangles to find the distance between the target point and the camera. Let the imaging planes of the two cameras be parallel to each other and on the same horizontal plane, and the main optical axes be parallel. Let the optical center distance be B, and the focal length of the camera be f, c keft is the central principal point of the left camera, c right is the principal point of the right camera center. The projection coordinates of a point M in the real world on the left and right imaging planes are the first pixel position M left (x lelf ,y left ) and the second pixel position M right (x right ,y right ), y left =y right , then the following formula is obtained from the trigonometric relationship:

[0101]

[0102] M left M right =B-(x lelf -x right ); (12)

[0103]

[0104] The camera's main optical axis divides the camera's horizontal field of view angle θ into two equal parts, such as Figure 5 As shown, the angle α between the ship and the main optical axis of the camera is calculated according to the following formula:

[0105]

[0106] The dual cameras and the center point of the ship form a triangle. These cameras are of the same model, with a pitch angle of 0 and their main optical axes parallel to each other.

[0107] Then, let p1(u1, v1) and p2(u2, v2) be the coordinates of the center point of the target detection frame of the same ship captured by the left and right cameras at the same time, which can be obtained from the target detection and tracking model. δ, ε and γ are the three internal angles of ΔABS formed by the left and right cameras and the center point of the ship. α1 and α2 are the angles between the line formed by the center point of the target ship and the two cameras and the main optical axes of the two cameras, respectively. ρ is the angle between the line formed by the two cameras and the main optical axes of the two cameras. The position parameter B of the binocular camera module is the distance between the left and right cameras.

[0108] δ=ρ-α1; (15)

[0109] ε=π-ρ+α2; (16)

[0110] γ=π-δ-ε; (17)

[0111] ρ can be calculated by following the steps below: Assuming that the longitude and latitude of the left camera and the right camera are (lon1, lat1) and (lon2, lat2) respectively, the angle τ between the line formed by the two cameras and the north direction is calculated as shown in the following formula:

[0112] x=cos(lat1)*sin(lat2)-sin(lat1)*cos(lat2)*cos(lon2-lon1); (18)

[0113] y=cos(lat2)*sin(lon2-lon1); (19)

[0114]

[0115] If the angle between the camera's main optical axis and the north direction is μ, then

[0116] ρ=|τ-μ|; (21)

[0117] Because the two cameras are of exactly the same model, L and f are the same, so the difference between α1 and α2 is the difference in the horizontal coordinates of the center point of the detection frame of the same ship in the two cameras. Therefore, the distance between the ship and the two cameras can be calculated by the trigonometric sine theorem,

[0118]

[0119] In some embodiments of step S102, the ship automatic identification system is A1S, and the data obtained by the ship automatic identification system is AIS data. AIS data includes GPS positioning data uploaded by each ship. By processing A1S data, the second navigation track of multiple ships within the detectable range obtained by the distance detection sensor can be obtained. Exemplarily, the AIS information of the target ship several seconds or even minutes ago is received through the AIS device of the ship. The information is text information including the movement state of the target ship at that time, mainly including the time at that time, the precise position of the target ship, the speed over the ground, the heading and other information, forming the initial second navigation track several seconds or even minutes ago; according to the initial second navigation track information, according to the ship hydrodynamic model, the target ship information at the current moment is calculated to obtain the second navigation track of the target ship; the second data set is formed according to the second navigation tracks of multiple ships. Further, after obtaining the AIS data, it can be pre-processed by deleting the data with obvious wrong information, incomplete, repeated, unreasonable stay points and abnormal drift points in the AIS data.

[0120] In some embodiments, the ship navigation track perception method of the embodiment of the present application may also include but is not limited to steps S301 to S302:

[0121] Step S301, determining a data set to be processed according to the trajectory point acquisition frequency of the distance detection sensor and the trajectory point acquisition frequency of the ship automatic identification system, wherein the data to be processed is a data set corresponding to the smaller trajectory point acquisition frequency of the first data set and the second data set;

[0122] Step S302, interpolating the navigation track in the to-be-processed data using the Lagrange interpolation method, so that the time interval between the navigation track points after the interpolation is the same as the time interval between the navigation track points of another data set.

[0123] In this embodiment, the trajectory point collection frequency of the distance detection sensor and the trajectory point collection frequency of the ship automatic identification system may be different, resulting in the first navigation trajectory and the second navigation trajectory being unable to be aligned in the time dimension. Therefore, this embodiment can perform interpolation processing on the navigation trajectory in which the trajectory point collection frequency is lower, that is, the trajectory points are sparse, to facilitate subsequent processing.

[0124] Taking the case of the low frequency of track point collection of the ship automatic identification system as an example, please refer to Figure 6, the track point time interval corresponding to the track point acquisition frequency of the distance detection sensor is selected as the AIS track point update interval. For the video track, it is only necessary to record the track point information at the corresponding time. For the AIS navigation track data, the Lagrange interpolation method can be used to extrapolate the AIS data (i.e., the AIS navigation track data) to obtain the AIS track points at any time point, so as to align the recording time with the CCTV data (i.e., the navigation track data obtained by the camera ranging technology).

[0125] The Lagrange interpolation method uses a polynomial function to fit the existing AIS track points, and then calculates the interpolation data based on the fitted function. i-1 to i There are k AIS track points between t j is the time when the trajectory point is emitted, t i-1 <t j <t i ,j=0,1,2,…,k. p j Finger j The trajectory point information at the moment, the Lagrange polynomial function L(t) is as follows:

[0126]

[0127] In step S103 of some embodiments, the trajectory point matching algorithm is an algorithm for determining whether two navigation trajectories are the same trajectory by matching the trajectory points of the two navigation trajectories one by one.

[0128] In some embodiments, step S103 may include but is not limited to steps S401 to S404:

[0129] Step S401, extracting any first navigation track in the first data set and any second navigation track in the second data set for correlation analysis;

[0130] Step S402, aligning the track points of the first navigation track with the track points of the second navigation track based on the track point collection time;

[0131] Step S403, calculating the distance between each pair of mutually aligned trajectory points to obtain the trajectory pair distance;

[0132] Step S404: When the distances of the track pairs with the expected number of proportions are all less than the first expected threshold, it is determined that the association result between the first navigation track and the second navigation track is mutually associated.

[0133] In this embodiment, the first data set T is defined VIS and the second dataset T AIS They are respectively expressed as follows:

[0134]

[0135] Among them, X ai and Y vj They represent the navigation tracks of the i-th AIS target ai and the j-th visual target vj respectively; I and J represent the number of the first navigation tracks in the first data set and the number of the second navigation tracks in the second data set respectively.

[0136] Any first navigation trajectory in the first data set and any second navigation trajectory in the second data set are extracted for correlation analysis. Based on the trajectory point acquisition time, the trajectory points of the first navigation trajectory and the trajectory points of the second navigation trajectory are aligned, and then the distance between each pair of aligned trajectory points is calculated to obtain the trajectory pair distance. When the trajectory pair distances accounting for an expected number of percentages are all less than the first expected threshold, for example, 95% of the trajectory pair distances are less than the first expected threshold, it means that most of the trajectory points match each other. Therefore, it is considered that the correlation result between the first navigation trajectory and the second navigation trajectory is mutually correlated.

[0137] Exemplarily, the number of matches and the association result of the AIS navigation track data (i.e., the second navigation track) or the video navigation track data (i.e., the first navigation track) at the previous moment are taken as input, as follows:

[0138]

[0139] Among them, Z ai Yes X ai and Y v1 The number of successful matches, [a i , v j ] means a i , v j Already linked together.

[0140] If, finally, two tracks are recorded in B, then by default, the two tracks are directly matched without similarity measurement with other tracks. Otherwise, the enhanced fast dynamic time warping algorithm considering direction can be used to calculate the similarity of AIS navigation track data or video navigation track data.

[0141] In some embodiments of step S104, after determining the association results between the first navigation track in the first data set and the second navigation track in the second data set, the first navigation track and the second navigation track that are already associated with each other are correspondingly removed from the first data set and the second data set according to the association results. Since the amount of computation required by the trajectory point matching algorithm is less than the track similarity calculation, the amount of computer computation can be reduced by first using the trajectory point matching algorithm to associate some of the navigation tracks and then performing similarity analysis on the remaining navigation tracks.

[0142] In some embodiments of step S105, the trajectory similarity between the first navigation trajectory in the updated first data set and the second navigation trajectory in the updated second data set is calculated, and the trajectory similarity can be used to determine whether the first navigation trajectory and the second navigation trajectory are the same trajectory.

[0143] In some embodiments, step S105 may also include but is not limited to steps S501 to S502:

[0144] Step S501, calculating the movement direction similarity, movement speed similarity and trajectory position similarity between the first navigation track in the first data set and the second navigation track in the second data set;

[0145] Step S502: performing weighted calculation on the movement direction similarity, the movement speed similarity and the trajectory position similarity to obtain the trajectory similarity between the first navigation trajectory and the second navigation trajectory.

[0146] In this embodiment, the trajectory similarity of two navigation trajectories is composed of the similarity of movement direction, movement speed and trajectory position. g It is expressed as follows:

[0147] C g =λ1*C c +λ2*C d +λ3*C s ; (28) Among them, C c Indicates the similarity of motion direction, C s Indicates the similarity of motion speed, C d represents the similarity of trajectory positions (calculated by the distance of trajectory points), and λ1, λ2, and λ3 represent the weights corresponding to each similarity. For example, λ1 can be set to 1000 to avoid fusion of data with too large angle deviation, and λ2 and λ3 can be set to 0.5 respectively.

[0148] In some embodiments, after similarity measurement is performed between all navigation trajectories, a similarity matrix M of size I×J is constructed. s , M s (i, j) represents X ai and Y v1 The similarity value of ai and Y v1 When the Euclidean distance between the last trajectory points exceeds the maximum matching distance, the two trajectories are considered to be completely different.

[0149] The specific steps are as follows:

[0150] (1) For two trajectories X and Y, the lengths are P and Q respectively, expressed as:

[0151]

[0152] (2) Constructing the alignment matrix

[0153] Use Euclidean distance to calculate the distance between each point in X and Y, and construct a P*Q alignment matrix d, where d(P, Q) represents point m F and n q The Euclidean distance between .

[0154] (3) Dynamic Time Warping (DTW) Path Construction:

[0155] Define a path W to construct a mapping between X and Y that satisfies constraints ① to ③.

[0156] Constraint ①: W1 = (1, 1), W C =(P,Q);

[0157] Constraint ②: The adjacent elements of path W can only contain adjacent coordinate points, including diagonally adjacent points. Therefore, W C-1 Can only be one of {(P-1, q), (p, q-1), (p-1, q-1);

[0158] Constraint ③: The elements of path W increase monotonically over time, that is, p′≤p, q′≤q.

[0159] (4) According to the dynamic programming strategy, the cumulative distance D(p, q) is calculated. Where D(p, q) is the minimum cumulative distance of the previous three possible warp path elements and m p and n q The sum of the Euclidean distances between is as follows:

[0160] D(p, q)=d(p, q)+min{D(p-1, q), D(p-1, q-1)}; (30)

[0161] (5) Find the optimal path: According to the dynamic programming strategy of DTW, find the path with the smallest cumulative distance, that is, the optimal warp path.

[0162] (6) Calculate similarity: Calculate the similarity value S(X, Y) between X and Y as follows:

[0163]

[0164] Where Dis(W) represents the sum of all corresponding Euclidean distances in the warp path W, and represents the angle between the starting point and the end point of X and Y.

[0165] In some embodiments, determining the first navigation trajectory and the second navigation trajectory that match each other according to the trajectory similarity in step S106 may include, but is not limited to, steps S601 to S602:

[0166] Step S601, constructing a cost matrix according to the trajectory similarity between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set;

[0167] Step S602, using the Hungarian algorithm to solve the cost matrix, to obtain the optimal trajectory matching strategy of the first data set and the second data set, wherein the optimal trajectory matching strategy is used to determine the first navigation trajectory and the second navigation trajectory that match each other.

[0168] In this embodiment, the cost matrix is ​​a two-dimensional array, in which each element represents the cost of matching an element in the left set with an element in the right set, that is, the trajectory similarity. After constructing the cost matrix, the optimal trajectory matching strategy with the total cost (maximum similarity) can be determined based on the Hungarian algorithm, so that the first navigation trajectory and the second navigation trajectory that match each other can be determined, and then the valid AIS data corresponding to multiple ships can be integrated into the video surveillance data.

[0169] In some embodiments, the step S106 of correcting the first navigation track that matches each other according to the second navigation track may include, but is not limited to, steps S701 to S703:

[0170] Step S701, calculating the distance between each pair of mutually aligned track points in the first and second matching navigation tracks to obtain a track pair distance;

[0171] Step S702, selecting a trajectory pair distance greater than a second expected threshold from all trajectory pair distances as an abnormal trajectory pair;

[0172] Step S703, determining a fused trajectory point according to two trajectory points from the first navigation trajectory and the second navigation trajectory respectively in the abnormal trajectory, and replacing the corresponding trajectory point in the first navigation trajectory with the fused trajectory point.

[0173] In this embodiment, after determining the mutually matching first navigation track and second navigation track through similarity analysis, the cosine distance formula can be used to calculate the distance between each pair of mutually aligned track points in the mutually matching first navigation track and second navigation track to obtain the track pair distance, and then select the track pair distance greater than the second expected threshold from all track pair distances as an abnormal track pair, that is, the track points with a large difference between the two mutually matching video detection navigation tracks and AIS navigation tracks are abnormal track pairs. Determine the fused track point based on the two track points in the abnormal track pair, and the calculation method of the fused track point can be weighted calculation of the positions of the two track points. The weight coefficient of the track point can be set based on the accuracy of binocular ranging positioning or GPS positioning. The higher the accuracy of the positioning method, the higher the weight coefficient of the track point of the corresponding method. It is also possible to directly use the average weighted calculation of the two track points to determine the fused track point. Replace the corresponding track point in the first navigation track with the fused track point, so as to correct the visual recognition navigation track and improve the accuracy of ship navigation track detection.

[0174] The present application also provides a ship navigation track perception system, including:

[0175] The first module is used to obtain first navigation tracks of multiple ships within a detectable range through a distance detection sensor to obtain a first data set;

[0176] The second module is used to obtain the second navigation tracks of multiple ships through the ship automatic identification system to obtain a second data set;

[0177] A third module is used to perform correlation analysis on the first navigation track in the first data set and the second navigation track in the second data set based on a track point matching algorithm to obtain a correlation result, wherein the correlation result is used to correlate the first navigation track and the second navigation track belonging to the same ship;

[0178] A fourth module is used to remove the associated first navigation track and the associated second navigation track from the first data set and the second data set respectively according to the association result;

[0179] A fifth module is used to calculate the trajectory similarity between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set;

[0180] The sixth module is used to determine the first navigation trajectory and the second navigation trajectory that match each other according to the trajectory similarity, and to correct the first navigation trajectory that matches each other according to the second navigation trajectory.

[0181] It can be understood that the contents of the above-mentioned ship navigation track perception method embodiment are applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above-mentioned ship navigation track perception method embodiment, and the beneficial effects achieved are also the same as the beneficial effects achieved by the above-mentioned ship navigation track perception method embodiment.

[0182] The embodiment of the present application also provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned ship navigation track perception method. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0183] See also Figure 7 , Figure 7 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0184] The processor 701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0185] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 702, and the processor 701 calls and executes the ship navigation track perception method of the embodiment of the present application;

[0186] Input / output interface 703, used to implement information input and output;

[0187] Communication interface 704, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0188] A bus 705 that transmits information between the various components of the device (e.g., the processor 701, the memory 702, the input / output interface 703, and the communication interface 704);

[0189] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via a bus 705 .

[0190] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned ship navigation trajectory perception method.

[0191] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0192] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0193] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0194] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0195] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0196] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0197] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0198] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0199] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0201] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0202] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for sensing a ship's navigation trajectory, characterized in that: The following steps are involved: Acquire first navigation tracks of multiple ships within a detectable range through a distance detection sensor to obtain a first data set; Acquire the second navigation tracks of the plurality of ships through a ship automatic identification system to obtain a second data set; Based on a track point matching algorithm, performing an association analysis on a first navigation track in the first data set and a second navigation track in the second data set to obtain an association result, wherein the association result is used to associate the first navigation track and the second navigation track belonging to the same ship; According to the association result, the associated first navigation track and the associated second navigation track are correspondingly removed from the first data set and the second data set; Calculating the trajectory similarity between the first flight trajectory in the first data set and the second flight trajectory in the second data set; A first navigation track and a second navigation track that match each other are determined according to the track similarity, and the first navigation track that matches each other is corrected according to the second navigation track.

2. The ship navigation track perception method according to claim 1, characterized in that: The distance detection sensor is a binocular camera module. The method of acquiring the first navigation tracks of multiple ships within the detectable range through the distance detection sensor to obtain the first data set includes the following steps: Acquire a first image and a second image for the same ship through the binocular camera module; Determine a first navigation track of the target ship according to a first pixel position of the target ship in the first image, a second pixel position of the target ship in the second image, and a position parameter of the binocular camera module; A first data set is formed according to first navigation tracks of a plurality of ships.

3. The method for sensing a ship's navigation track according to claim 1, characterized in that: The ship navigation track perception method also includes the following steps: Determine a data set to be processed according to the track point acquisition frequency of the distance detection sensor and the track point acquisition frequency of the ship automatic identification system, wherein the data to be processed is a data set corresponding to the smaller track point acquisition frequency of the first data set and the second data set; The navigation track in the data to be processed is interpolated by using the Lagrange interpolation method, so that the time interval between track points of the interpolated navigation track is the same as the time interval between track points of the navigation track in another data set.

4. The method for sensing a ship's navigation track according to claim 1, characterized in that: The method of performing association analysis on the first navigation track in the first data set and the second navigation track in the second data set based on the track point matching algorithm to obtain an association result includes the following steps: extracting any first navigation track in the first data set and any second navigation track in the second data set for correlation analysis; Aligning the track points of the first navigation track with the track points of the second navigation track based on the track point collection time; Calculate the distance between each pair of aligned trajectory points to obtain the trajectory pair distance; When the distances of the track pairs with the expected number of proportions are all less than the first expected threshold, it is determined that the association result between the first navigation track and the second navigation track is mutually associated.

5. The method for sensing a ship's navigation track according to claim 1, characterized in that: The calculating of the trajectory similarity between the first flight trajectory in the first data set and the second flight trajectory in the second data set comprises the following steps: Calculating the similarity of movement direction, movement speed and trajectory position between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set; The moving direction similarity, the moving speed similarity and the trajectory position similarity are weightedly calculated to obtain the trajectory similarity between the first navigation trajectory and the second navigation trajectory.

6. The ship navigation track perception method according to claim 1, characterized in that: The step of determining the first navigation trajectory and the second navigation trajectory that match each other according to the trajectory similarity comprises the following steps: constructing a cost matrix according to the trajectory similarities between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set; The cost matrix is ​​solved by using the Hungarian algorithm to obtain an optimal trajectory matching strategy for the first data set and the second data set, wherein the optimal trajectory matching strategy is used to determine a first navigation trajectory and a second navigation trajectory that match each other.

7. The method for sensing a ship's navigation track according to claim 1, characterized in that: The method of correcting the first navigation track that matches each other according to the second navigation track comprises the following steps: Calculating the distance between each pair of mutually aligned track points in the first navigation track and the second navigation track that match each other to obtain a track pair distance; Selecting a trajectory pair distance greater than a second expected threshold from all the trajectory pair distances as an abnormal trajectory pair; A fused trajectory point is determined according to two trajectory points in the abnormal trajectory pair respectively from the first navigation trajectory and the second navigation trajectory, and the corresponding trajectory point in the first navigation trajectory is replaced by the fused trajectory point.

8. A ship navigation track perception system, characterized in that: include: The first module is used to obtain first navigation tracks of multiple ships within a detectable range through a distance detection sensor to obtain a first data set; The second module is used to obtain the second navigation tracks of the plurality of ships through the ship automatic identification system to obtain a second data set; a third module, configured to perform association analysis on the first navigation track in the first data set and the second navigation track in the second data set based on a track point matching algorithm to obtain an association result, wherein the association result is used to associate the first navigation track and the second navigation track belonging to the same ship; A fourth module, configured to remove the associated first navigation track and the associated second navigation track from the first data set and the second data set respectively according to the association result; A fifth module, configured to calculate the trajectory similarity between the first navigation trajectory in the first data set and the second navigation trajectory in the second data set; The sixth module is used to determine the first navigation trajectory and the second navigation trajectory that match each other according to the trajectory similarity, and to correct the first navigation trajectory that matches each other according to the second navigation trajectory.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the method described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of claims 1 to 7.

Images