AIS data track denoising method

Abstract

The present invention discloses an AIS data track denoising method, which includes: obtaining AIS track data after conditional screening processing; performing small time scale discontinuity judgment processing on the AIS track data to obtain a track segmentation mark set; using the track segmentation mark set to segment the AIS track data to obtain a sub-track set; based on the sub-track set and the ship's ground speed corresponding to the AIS track data, performing speed anomaly denoising processing on the AIS track data to obtain first AIS track data with data points with speed anomalies removed; based on the ship length corresponding to the first AIS track data, performing heading range anomaly denoising processing on the first AIS track data to obtain second AIS track data with data points with heading range anomalies removed. The AIS data track denoising method of the present invention can effectively weaken the influence of weak continuity of satellite-based AIS data on data denoising, significantly improve the data denoising effect, has a wide range of applications, and can be operated online.

Description

Technical Field

[0001] The present invention relates to the technical field of AIS data processing, and in particular to an AIS data track denoising method. Background Art

[0002] The Automatic Identification System (AIS) is an auxiliary navigation device that the International Maritime Organization (IMO) requires to be installed on ships. It uses GPS technology, very high frequency (VHF) technology, and self-organizing time division multiple access (SOTDMA) technology. It has the advantages of working around the clock and in all weather conditions and is widely distributed around the world. The AIS system uses open broadcast technology to periodically broadcast a wealth of ship data information. AIS data contains dynamic information, static information, voyage information, and safety information about ship operations. By analyzing AIS data, important information such as ship behavior characteristics and maritime traffic patterns can be obtained. Satellite-based AIS equipment can achieve continuous detection and reception of AIS data worldwide. However, due to various reasons, some errors exist in the raw AIS data. If track denoising is not performed, the accuracy of the data analysis will be reduced.

[0003] Existing AIS data track denoising methods mainly include conditional filtering and motion state judgment. Conditional filtering filters and judges data fields according to AIS message specifications. Motion state judgment generally determines outliers by manually setting thresholds for ship speed or turning rate, and then removes them.

[0004] However, conditional screening can only exclude data with obvious numerical abnormalities, ignoring the relevance of AIS data as time series data; the setting of speed and turning rate thresholds for motion state judgment is not data-driven, and does not take into account the discontinuous characteristics of current satellite-based AIS data. It is not suitable for satellite-based AIS data with complex situations and global coverage. Summary of the Invention

[0005] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a method for denoising AIS data tracks.

[0006] The technical solutions of the present invention are as follows:

[0007] A method for denoising AIS data tracks is provided, the method comprising:

[0008] Obtain AIS track data after conditional screening;

[0009] Performing small time scale discontinuity determination processing on the AIS track data to obtain a track segmentation mark set;

[0010] Segmenting the AIS track data using the track segmentation mark set to obtain a sub-track set;

[0011] performing speed anomaly denoising on the AIS track data based on the sub-track set and the ship ground speed corresponding to the AIS track data, to obtain first AIS track data without speed anomaly data points;

[0012] Based on the ship length corresponding to the first AIS track data, heading range anomaly denoising processing is performed on the first AIS track data to obtain second AIS track data without the data points of the heading range anomaly.

[0013] In some possible implementations, it is defined that the AIS track data after conditional screening and arranged in chronological order is Data point x l The corresponding time is t l , l=1,2,…,n, time series Data point x l with x l+1 The time difference is δt l =t l+1 -t l ;

[0014] Performing small time scale discontinuity determination processing on the AIS track data to obtain a track segmentation mark set includes:

[0015] Calculate the time difference sequence corresponding to the AIS track data δt={δt1,…,δt n-1};

[0016] Calculate the time difference sequence δt={δt1,…,δt n-1 The upper quartile of and interquartile range (IQR) δt ;

[0017] Establish a track segmentation mark set C which is initially an empty set;

[0018] Judge in sequence Is it satisfied? If so, then {x i ,x i+1} is judged as discontinuous on a small time scale, and x i Add to set C;

[0019] judge Is it satisfied? If so, add x1 to the set C and make it the first element;

[0020] judge Is it satisfied? If so, then x n Add it to set C as the last element.

[0021] In some possible implementations, segmenting the AIS track data using the track segmentation mark set to obtain a sub-track set includes:

[0022] Two adjacent data points in the track segmentation mark set are selected in sequence, and a sub-track is formed by the two selected data points and data points in the AIS track data that are within a time span of the two data points, thereby obtaining a sub-track set.

[0023] In some possible implementations, performing speed anomaly denoising on the AIS track data based on the sub-track set and the ship ground speed corresponding to the AIS track data includes:

[0024] Select one sub-track from the sub-track set in turn;

[0025] For each selected sub-track, one data point in the sub-track is selected in turn, and whether the selected data point is a speed anomaly point is determined based on the ship's ground speed corresponding to the AIS track data. If so, the current data point is deleted from the AIS track data.

[0026] In some possible implementations, the following method is used to determine whether a selected data point is a velocity anomaly point:

[0027] Calculate the spherical distance between the currently selected data point and the next data point in the sub-track;

[0028] Calculate the first average speed v of the ship from the current selected data point to the next data point according to the spherical distance calc ;

[0029] Calculate the second average speed v of the ship at the currently selected data point and the next data point ave ;

[0030] Judge v calc >v0 or Is it satisfied? If so, the currently selected data point is judged to be a speed anomaly point;

[0031] Wherein, v0 represents a preset speed threshold, k is a preset threshold, and k>0.

[0032] In some possible implementations, the first average speed v calc Calculated using the following formula:

[0033]

[0034] Among them, D q,q+1 Indicates the spherical distance between the currently selected data point and the next data point, t q Indicates the time corresponding to the currently selected data point, t q+1 Indicates the time corresponding to the next data point.

[0035] In some possible implementations, the second average speed v ave Calculated using the following formula:

[0036]

[0037] Among them, v q Indicates the speed corresponding to the currently selected data point, v q+1 Indicates the speed corresponding to the next data point.

[0038] In some possible implementations, performing heading range anomaly denoising on the first AIS track data based on the ship length corresponding to the first AIS track data includes:

[0039] Select the first three time-continuous data points in the first AIS track data;

[0040] Taking the plane where the ship is located as the reference, calculate the distance between the first data point and the second data point, and the distance between the second data point and the third data point among the three selected data points;

[0041] Calculate the angle between the line connecting the first data point to the second data point and the line connecting the second data point to the third data point;

[0042] judge If not, determining that the heading range at the second data point is abnormal, and deleting the current second data point from the first AIS track data;

[0043] Determine whether the second data point is deleted. If so, use the third data point and two subsequent time-consecutive data points as the selected three data points and continue to perform heading range anomaly denoising until all data points in the first AIS track data are traversed. If not, use the second data point and two subsequent time-consecutive data points as the selected three data points and continue to perform heading range anomaly denoising until all data points in the first AIS track data are traversed.

[0044] Among them, δθ represents the angle between the line connecting the first data point to the second data point and the line connecting the second data point to the third data point. represents the distance between the first data point and the second data point, represents the distance between the second data point and the third data point, and L represents the length of the ship corresponding to the first AIS track data.

[0045] In some possible implementations, when the method is run online, a small time-scale discontinuity determination threshold is calculated based on all second AIS track data acquired in a historical period, and the acquired small time-scale discontinuity determination threshold is used in the small time-scale discontinuity determination processing process of AIS track data acquired in the next period.

[0046] The main advantages of the technical solution of the present invention are as follows:

[0047] The AIS data track denoising method of the present invention can effectively weaken the impact of the weak continuity of satellite-based AIS data on data denoising by performing small time-scale discontinuity judgment processing, thereby improving the data denoising effect. By adopting data-driven speed anomaly denoising processing and heading range anomaly denoising processing, and considering the ship size characteristics in the heading range anomaly denoising processing to make the description of the ship's motion state more accurate, the data denoising effect can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0049] Figure 1 Flowchart of a method for denoising AIS data tracks according to an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a steady motion phase of a ship turning according to an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of an original track is given as an example of the present invention;

[0052] Figure 4 To utilize the method of one embodiment of the present invention to Figure 3 The schematic diagram of the original track after denoising is shown. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] The technical solutions provided by the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0055] refer to Figure 1 An embodiment of the present invention provides an AIS data track denoising method, which includes the following steps S1-S5:

[0056] Step S1, obtaining AIS track data after conditional screening.

[0057] In one embodiment of the present invention, the conditional screening process may be a conventional conditional screening process, specifically, screening the AIS data fields according to the AIS message specification to remove data with obviously abnormal values.

[0058] Step S2: performing small time scale discontinuity determination processing on the AIS track data to obtain a track segmentation mark set.

[0059] Definition: The AIS track data after conditional screening and arranged in chronological order is Data point x l The corresponding time is t l , l=1,2,…,n, time series Data point x l with x l+1 The time difference is δt l =t l+1 -t l .

[0060] In one embodiment of the present invention, based on the above definition, a small time scale discontinuity determination process is performed on the AIS track data to obtain a track segmentation mark set, further comprising the following steps:

[0061] Step S21, calculate the time difference sequence δt corresponding to the AIS track data = {δt1,…,δt n-1};

[0062] Step S22, calculate the time difference sequence δt={δt1,…,δt n-1 The upper quartile of and interquartile range (IQR) δt ;

[0063] Step S23, establishing a track segmentation mark set C which is initially an empty set;

[0064] Step S24, determine Is it satisfied? If so, then {x i ,x i+1} is judged as discontinuous on a small time scale, and x i Add to set C;

[0065] Step S25, determine Is it satisfied? If so, add x1 to the set C and make it the first element;

[0066] Step S26, determine Is it satisfied? If so, then x n Add it to set C as the last element.

[0067] Through the above processing, the track segmentation mark set corresponding to the AIS track data is obtained.

[0068] Step S3: segment the AIS track data using the track segmentation mark set to obtain a sub-track set.

[0069] In one embodiment of the present invention, the AIS track data is segmented using a track segmentation mark set to obtain a sub-track set, specifically including:

[0070] Two adjacent data points in the track segmentation mark set are selected in sequence, and a sub-track is formed by the two selected data points and the data points in the AIS track data that are within the time span of the two data points to obtain a sub-track set.

[0071] Specifically, the track segmentation mark set is The two data points selected are For example, the sub-track obtained based on the two selected data points can be expressed as in,

[0072] In one embodiment of the present invention, any two adjacent data points in the track segmentation marker set yield a corresponding sub-track, and all sub-tracks constitute a sub-track set. Sub-tracks maintain internal small-time-scale continuity, and in subsequent denoising, different sub-tracks are treated as distinct tracks.

[0073] Step S4: Based on the sub-track set and the ship ground speed corresponding to the AIS track data, the AIS track data is subjected to speed anomaly denoising processing to obtain first AIS track data with data points with speed anomalies removed.

[0074] In one embodiment of the present invention, based on the sub-track set and the ship ground speed corresponding to the AIS track data, speed anomaly denoising is performed on the AIS track data, specifically including:

[0075] Select one sub-track from the sub-track set in turn;

[0076] For each selected sub-track, one data point in the sub-track is selected in turn. Based on the ship's ground speed corresponding to the AIS track data, it is determined whether the selected data point is a speed anomaly point. If so, the current data point is deleted from the AIS track data.

[0077] Furthermore, in one embodiment of the present invention, determining whether a selected data point is a speed anomaly point based on the ship's ground speed corresponding to the AIS track data specifically includes the following steps:

[0078] Step S41, calculating the spherical distance between the currently selected data point and the next data point in the sub-track;

[0079] Step S42, calculate the first average speed v of the ship from the current selected data point to the next data point according to the spherical distance calc ;

[0080] Step S43, calculate the second average speed v of the ship at the currently selected data point and the next data point ave ;

[0081] Step S44, determine v calc >v0 or Is it satisfied? If so, the currently selected data point is judged to be a speed anomaly point;

[0082] Wherein, v0 represents a preset speed threshold, k is a preset threshold, and k>0.

[0083] In one embodiment of the present invention, the spherical distance between two data points is calculated using the Haversine formula.

[0084] Furthermore, set: the currently selected data point is x q , the next data point is x q+1 , the currently selected data point x q and the next data point x q+1 The corresponding spherical distance is D q,q+1 , the currently selected data point x q The corresponding time is t q , the next data point x q+1 The corresponding time is t q+1 , the currently selected data point x q The corresponding speed is v q , the next data point x q+1 The corresponding speed is v q+1 , where the time and speed corresponding to the data point can be determined through AIS data.

[0085] Based on the above settings, the first average speed v calc Calculated using the following formula:

[0086]

[0087] The second average speed v ave Calculated using the following formula:

[0088]

[0089] In one embodiment of the present invention, formula v calc >v0 is used to determine whether the minimum speed of the ship is too large. When the formula v calc > When v0 is satisfied, it means that the minimum speed of the ship is too large, and the corresponding current data point is judged as noise, that is, the current data point is judged as a speed abnormal point.

[0090] The speed threshold v0 is set according to the actual situation. In one embodiment of the present invention, v0 is the 0.97th percentile of the ship's ground speed corresponding to the AIS track data.

[0091] In one embodiment of the present invention, the formula Used to determine the ship's q to t q+1 Is there an acceleration during the time? When the formula When it is satisfied, it means that the ship speed change is inconsistent with the time span, and the corresponding current data point is judged as noise, that is, the current data point is judged as a speed abnormal point.

[0092] The threshold k is set according to actual conditions. In one embodiment of the present invention, k is set to 0.7.

[0093] Step S5: Based on the ship length corresponding to the first AIS track data, the first AIS track data is subjected to heading range anomaly denoising processing to obtain second AIS track data with the data points of the heading range anomaly removed.

[0094] refer to Figure 2 , Figure 2 This is a diagram of the steady motion phase of the ship's rotation, assuming that: x a 、x b 、x c The first AIS track data S V Three consecutive data points in time, the ship is at x a 、x b 、x c Based on the above assumptions, the ship's motion parameters are relatively stable in this stage, and the rotation trajectory is approximately a circular arc. In this case, the radius of the turning circle R is the steady-state turning radius.

[0095] Further, refer to Figure 2 , assuming: Then we have:

[0096]

[0097] ∠1 represents the distance from the center of the circle O to the first data point x a The line connecting the center of the circle O to the second data point x b The angle between the line connecting the center O of the circle and the second data point x is represented by ∠2. b The line connecting the center of the circle O to the third data point x c The angle between the lines of Represents the first data point x a With the second data point x b distance, Represents the second data point x b With the third data point x c The distance, ∠α represents the first data point x a To the second data point x b The line connecting the center of the circle O to the second data point x b The angle between the lines of ∠β represents the second data point x b To the third data point x c The line connecting the center of the circle O to the third data point x c The angle between the lines, δθ represents the first data point x a To the second data point x b The line connecting the second data point x b To the third data point x c The angle between the lines.

[0098] Furthermore, according to the technical literature: Principles of Ships, Sheng Zhenbang, Liu Yingzhong et al., pp. 289-291, Shanghai Jiaotong University Press, 2003, the minimum turning diameter D' of a ship and the length L of the ship satisfy D'=k*L(5≤k≤7).

[0099] Based on the above analysis, in one embodiment of the present invention, the first AIS track data is subjected to heading range anomaly denoising based on the ship length corresponding to the first AIS track data, further comprising the following steps:

[0100] Step S51, selecting the first three time-continuous data points in the first AIS track data;

[0101] Step S52, calculating the distance between the first data point and the second data point, and the distance between the second data point and the third data point, of the three selected data points, based on the plane on which the ship is located;

[0102] Step S53, calculating the angle between the line connecting the first data point to the second data point and the line connecting the second data point to the third data point;

[0103] Step S54, determine If not, it is determined that the heading range at the second data point is abnormal, and the current second data point is deleted from the first AIS track data;

[0104] In step S55, it is determined whether the second data point is deleted. If so, the third data point and the two subsequent time-consecutive data points are used as the three selected data points, and the process returns to step S52 to continue the heading range anomaly denoising process until all data points in the first AIS track data are traversed. If not, the second data point and the two subsequent time-consecutive data points are used as the three selected data points, and the process returns to step S52 to continue the heading range anomaly denoising process until all data points in the first AIS track data are traversed.

[0105] Through the above processing, the second AIS track data finally obtained is the AIS track data with abnormal speed data points and abnormal heading range data points removed.

[0106] Furthermore, in one embodiment of the present invention, the method can also be used in the case of online operation. When the method is run online, the AIS track data obtained in different time periods are denoised separately, and when performing the denoising process, the small time scale discontinuity determination threshold is calculated based on the second AIS track data obtained in the historical time period, and the obtained small time scale discontinuity determination threshold is used in the small time scale discontinuity determination processing process of the AIS track data obtained in the next time period.

[0107] Specifically, the upper quartile is calculated based on the second AIS track data obtained in the historical period before the next period. and interquartile range (IQR) δt , and then calculate the small time scale discontinuity judgment threshold The obtained small time scale discontinuity determination threshold The small time scale discontinuity determination and processing process for AIS track data acquired in the next period is used.

[0108] The AIS data track denoising method provided by one embodiment of the present invention can effectively weaken the impact of the weak continuity of satellite-based AIS data on data denoising by performing small-time-scale discontinuity determination processing, thereby improving the data denoising effect. By adopting data-driven speed anomaly denoising processing and heading range anomaly denoising processing, and considering the ship size characteristics in the heading range anomaly denoising processing to make the description of the ship's motion state more accurate, the data denoising effect can be significantly improved.

[0109] The AIS data track denoising method provided by an embodiment of the present invention can be applied to AIS data denoising processing within a specified research range, and can also be directly applied to AIS data denoising processing worldwide, and has a wide range of applicability.

[0110] The AIS data track denoising method provided in one embodiment of the present invention can also be run online, thereby promptly detecting abnormal data and achieving real-time monitoring of the ocean situation.

[0111] The following describes the beneficial effects of the AIS data track denoising method provided by an embodiment of the present invention with reference to specific examples:

[0112] refer to Figure 3-Figure 4 , Figure 3 A schematic diagram of an original track is given as an example of the present invention. Figure 4 To utilize the method of one embodiment of the present invention to Figure 3 The schematic diagram of the original track after denoising is shown in the figure. Figure 3-4 It can be seen that compared with the original track, the track after denoising using the AIS data track denoising method provided by an embodiment of the present invention is smoother, and the spiky track and abrupt turning points are removed.

[0113] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for denoising AIS data tracks, characterized in that: include: Obtain AIS track data after conditional screening; Performing small time scale discontinuity determination processing on the AIS track data to obtain a track segmentation mark set; Segmenting the AIS track data using the track segmentation mark set to obtain a sub-track set; performing speed anomaly denoising on the AIS track data based on the sub-track set and the ship ground speed corresponding to the AIS track data, to obtain first AIS track data without speed anomaly data points; performing heading range anomaly denoising processing on the first AIS track data based on a ship length corresponding to the first AIS track data, and obtaining second AIS track data from which data points with heading range anomalies are removed; Definition: The AIS track data after conditional screening and arranged in chronological order is , data points The corresponding time is , , time series , data points and The time difference is ; Performing small time scale discontinuity determination processing on the AIS track data to obtain a track segmentation mark set includes: Calculate the time difference sequence corresponding to AIS track data ; Calculate time difference series Upper quartile of and interquartile range ; Establish a track segmentation mark set that is initially empty ; Judge in sequence Is it satisfied? If so, The small time scale discontinuity of is judged as yes, Join Collection ; judge Is it satisfied? If so, Add it to set C as the first element; judge Is it satisfied? If so, Add it to set C as the last element.

2. The AIS data track denoising method according to claim 1, characterized in that: The AIS track data is segmented using the track segmentation mark set to obtain a sub-track set, including: Two adjacent data points in the track segmentation mark set are selected in sequence, and a sub-track is formed by the two selected data points and data points in the AIS track data that are within a time span of the two data points, thereby obtaining a sub-track set.

3. The AIS data track denoising method according to claim 2, characterized in that: Based on the sub-track set and the ship ground speed corresponding to the AIS track data, speed anomaly denoising processing is performed on the AIS track data, including: Select one sub-track from the sub-track set in turn; For each selected sub-track, one data point in the sub-track is selected in turn, and whether the selected data point is a speed anomaly point is determined based on the ship's ground speed corresponding to the AIS track data. If so, the current data point is deleted from the AIS track data.

4. The AIS data track denoising method according to claim 3, characterized in that: Use the following method to determine whether the selected data point is a speed anomaly point: Calculate the spherical distance between the currently selected data point and the next data point in the sub-track; Calculate the first average speed of the ship from the current selected data point to the next data point according to the spherical distance ; Calculate the second average speed of the ship at the currently selected data point and the next data point ; judge or Is it satisfied? If so, the currently selected data point is judged to be a speed anomaly point; in, Indicates the preset speed threshold, is the preset threshold, and .

5. The AIS data track denoising method according to claim 4, characterized in that: First average speed Calculated using the following formula: ； in, Indicates the spherical distance between the currently selected data point and the next data point. Indicates the time corresponding to the currently selected data point, Indicates the time corresponding to the next data point.

6. The AIS data track denoising method according to claim 4, characterized in that: Second average speed Calculated using the following formula: ； in, Indicates the speed corresponding to the currently selected data point, Indicates the speed corresponding to the next data point.

7. The AIS data track denoising method according to claim 1, characterized in that: Based on the ship length corresponding to the first AIS track data, performing heading range anomaly denoising processing on the first AIS track data, including: Select the first three time-continuous data points in the first AIS track data; Taking the plane where the ship is located as the reference, calculate the distance between the first data point and the second data point, and the distance between the second data point and the third data point among the three selected data points; Calculate the angle between the line connecting the first data point to the second data point and the line connecting the second data point to the third data point; judge If not, determining that the heading range at the second data point is abnormal, and deleting the current second data point from the first AIS track data; Determine whether the second data point is deleted. If so, use the third data point and two subsequent time-consecutive data points as the selected three data points and continue to perform heading range anomaly denoising until all data points in the first AIS track data are traversed. If not, use the second data point and two subsequent time-consecutive data points as the selected three data points and continue to perform heading range anomaly denoising until all data points in the first AIS track data are traversed. in, It represents the angle between the line connecting the first data point to the second data point and the line connecting the second data point to the third data point. represents the distance between the first data point and the second data point, represents the distance between the second data point and the third data point, Indicates the length of the ship corresponding to the first AIS track data.

8. The AIS data track denoising method according to any one of claims 1 to 7, characterized in that: When the method is run online, a small time scale discontinuity determination threshold is calculated based on all second AIS track data acquired in a historical period, and the acquired small time scale discontinuity determination threshold is used in a small time scale discontinuity determination processing process of AIS track data acquired in a next period.

Images