Fishing boat trajectory abnormal point identification and data cleaning method

Abstract

The invention relates to a fishing boat trajectory abnormal point identification and data cleaning method. The method comprises the following steps: acquiring and preprocessing trajectory data of a fishing boat; generating a Geohash code of the corresponding ship location point according to the longitude and latitude of each ship location point; according to the time interval between the current ship location point and the next ship location point and the ship speed of the current ship location point, calculating to obtain a travelable distance, and further analyzing to obtain Geohash codes of all reachable areas of the fishing boat in the time interval as a range code set of the current ship location point; and comparing the range code set of the current ship location point with the range code set of the previous ship location point and the range code set of the next ship location point, and if no intersection exists, identifying that the current ship location point is an abnormal point and removing the abnormal point. According to the method, the abnormal points in the trajectory data can be effectively identified, efficient and effective trajectory data cleaning is realized, and the accuracy and reliability of the data are improved.

Description

Technical Field

[0001] The present invention relates to the fields of fishing vessel monitoring, spatial analysis and data processing, and in particular to a method for identifying abnormal points on fishing vessel trajectories and cleaning data. Background Art

[0002] In modern fishery management, the trajectory data of fishing vessels is widely used to monitor and analyze fishing vessel activities. However, trajectory data is often affected by environmental factors, equipment failure, signal loss and human error, resulting in a large number of anomalies in the data. These anomalies not only reduce the credibility of the data, but also have a negative impact on subsequent data analysis and decision-making.

[0003] Existing techniques for identifying outliers in trajectory data are usually based on simple statistical analysis, and data cleaning often uses a simple threshold method, which fails to effectively utilize the spatial characteristics of navigation data, resulting in inaccurate and inefficient identification of outliers. In addition, these techniques do not take into account the impact of ship speed and travel time interval on trajectory analysis accuracy, thus limiting the effectiveness and reliability of their application. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for identifying abnormal points in the trajectory of a fishing vessel and cleaning the data, which can effectively identify the abnormal points in the trajectory data and remove the abnormal points in the trajectory data, thereby achieving efficient and effective trajectory data cleaning, and improving the accuracy and reliability of the trajectory data to ensure subsequent analysis and application.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a method for identifying abnormal points of fishing vessel tracks and cleaning data, comprising the following steps:

[0006] Obtaining and preprocessing the trajectory data of the fishing boat, wherein the trajectory data includes the time, longitude, latitude and speed of each boat position;

[0007] Generate the corresponding Geohash code according to the longitude and latitude of each ship location;

[0008] The drivable distance is calculated based on the time interval between the current ship position and the next ship position and the ship speed, and then the Geohash codes of all reachable areas of the fishing boat within the time interval are analyzed and obtained as the range code set of the current ship position according to the drivable distance and the Geohash code of the current ship position;

[0009] The range code set of the current ship position is compared with the range code set of the previous ship position and the range code set of the next ship position. If there is no intersection, the current ship position is identified as an abnormal point and deleted from the trajectory data.

[0010] Furthermore, the Geohash codes of all reachable areas of the fishing boat within the time interval are analyzed and obtained as a range code set of the current ship position according to the drivable distance and the Geohash code of the current ship position, including:

[0011] Calculate the grid cell side length according to the accuracy of Geohash coding;

[0012] The result obtained by dividing the drivable distance by the side length of the grid unit is rounded up to obtain a GeoHash level;

[0013] According to the Geohash code and GeoHash level of the current ship position, several grid cells constituting its domain range are determined, and the Geohash codes corresponding to the several grid cells are used as the range code set of the current ship position.

[0014] Furthermore, the plurality of grid cells constituting the scope of the current ship position are determined according to the Geohash code and GeoHash level of the current ship position, including:

[0015] Taking the grid cell corresponding to the Geohash code of the current ship position as the center, expand outward by a set number of grid cells along the four sides, and the set number is equal to the GeoHash level of the current ship position;

[0016] The rectangular area composed of the central grid unit and the extended grid unit is taken as the scope of the current ship position.

[0017] Furthermore, the precision of Geohash encoding is not less than 7.

[0018] Furthermore, if the current ship position is the last ship position, the drivable distance is calculated according to the time interval with the previous ship position and the ship speed.

[0019] Furthermore, if the range code set of the current ship point does not have the same Geohash code as the range code set of the previous ship point, and the range code set of the current ship point does not have the same Geohash code as the range code set of the next ship point, then it is considered that the range code set of the current ship point has no intersection with the range code set of the previous ship point and the range code set of the next ship point.

[0020] Furthermore, the drivable distance is calculated by the following formula:

[0021] Driving distance = ship speed × time interval × speed unit conversion factor.

[0022] Furthermore, the step of preprocessing the trajectory data includes:

[0023] Standardize time formats;

[0024] Removing missing values ​​and duplicate data from the trajectory data;

[0025] Delete the ship position data whose speed exceeds the maximum speed of the fishing vessel.

[0026] Beneficial Effects

[0027] Due to the adoption of the above technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention dynamically calculates the drivable distance according to the ship speed and the time interval, and calculates the corresponding Geohash level, and then obtains the domain range of the ship position by dynamically adjusting the Geohash level, and finally performs intersection detection on the multi-layer Geohash code set of the current point and the previous and subsequent points to bidirectionally verify the validity of the ship position, thereby improving the accuracy of outlier detection; the present invention makes full use of the regional representation and spatial characteristics of Geohash coding, and can adapt to the fishing vessel trajectory data under different navigation conditions, and whether it is high-speed navigation or low-speed cruising, the algorithm can adjust the domain range of the trajectory point according to the specific situation, thereby improving the adaptability and application scope of the algorithm, and effectively improving the accuracy of outlier detection and the efficiency of data cleaning. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of the Geohash hierarchy and domain scope of an embodiment of the present invention;

[0030] Figure 3 It is a schematic diagram of the multi-layer domain intersection of the fishing vessel's position according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0032] The embodiment of the present invention relates to a method for identifying abnormal points of fishing boat tracks and cleaning data based on Geohash coding, such as Figure 1 As shown, the following steps are included:

[0033] Obtain the trajectory data of fishing vessels and perform preprocessing;

[0034] Generate the corresponding Geohash code according to the longitude and latitude of each ship location;

[0035] The drivable distance is calculated based on the time interval between the current ship position and the next ship position and the ship speed;

[0036] According to the drivable distance and Geohash code of the current ship position, the Geohash codes of all reachable areas of the fishing boat within the time interval are analyzed and obtained as the range code set of the current ship position;

[0037] It is determined whether the range code set of the current ship position has an intersection with the range code sets of the previous and next ship positions. If there is no intersection, the current ship position is identified as an abnormal trajectory point and deleted from the trajectory data. Otherwise, the current ship position is considered to be a normal trajectory point.

[0038] More specifically, the following steps are included:

[0039] (1) Obtain the Beidou ship position data of the fishing vessel, which includes timestamp, longitude, latitude, speed (in knots), and other information. Preprocess the collected data to ensure the consistency of the time format, remove missing values ​​and duplicate data, and delete the ship position data whose speed exceeds the maximum speed of the fishing vessel to ensure the accuracy of subsequent analysis;

[0040] (2) Perform Geohash encoding on the current Beidou ship location data according to its longitude and latitude to generate the corresponding GeoHash string. Use 7-bit precision to obtain the appropriate spatial resolution. The range formed by the 7-bit precision GeoHash code near the latitude of 30° is approximately 132.2 meters wide and 152.7 meters long. A higher precision GeoHash code can also be used as needed;

[0041] (3) Based on the ship speed (in knots) of the current ship position and the time interval (in seconds) with the next point, the distance range that the current ship position can travel within the time interval with the ship position as the center can be calculated: drivable distance = ship speed × time interval × conversion factor. The conversion factor converts knots to meters per second (1 knot ≈ 0.51444 meters per second). The last ship position uses the time interval with the previous ship position;

[0042] (4) According to the calculated drivable distance, the Geohash level of each ship site and the domain range including the central grid are dynamically adjusted. The drivable distance is divided by 132.2, and the result is rounded up to get the number of GeoHash levels. Level 1 is a 9-domain range including the central grid, level 2 is a 25-domain range including the central grid, level 3 is a 49-domain range including the central grid, and so on. The maximum number of levels is 8, which is a 289-domain range.

[0043] (5) According to the multi-layer domain scope involved in the current ship position, all GeoHash codes of the multi-layer domain scope involved in the GeoHash code of the current ship position are obtained to form a set;

[0044] (6) Perform intersection detection on the multi-layered domain range GeoHash code of the current ship position with the multi-layered domain range GeoHash code of the previous ship position and the next ship position to determine whether there is intersection, that is, determine whether a link is formed between the ship positions by determining whether there is the same GeoHash code. If there is the same code, it is identified as a normal trajectory point. If the multi-layered domain range Geohash code of a certain ship position has no intersection with the multi-layered domain range of the previous and next ship positions, then the point is identified as an abnormal point;

[0045] (7) The points with intersections with the ship positions are considered normal trajectory points and retained, while the points with no intersections with the preceding and following ship positions are considered abnormal trajectory points and deleted.

[0046] The following takes the trajectory data of the fishing boat named Zhelingyu00028 equipped with Beidou equipment as an example to further illustrate this implementation mode:

[0047] 1. Data preprocessing:

[0048] Taking the trajectory data of the fishing boat named Zhelingyu00028 equipped with Beidou equipment as an example, the original csv format data is converted into an xls format file. The data includes timestamp, longitude, latitude, speed (in knots) and other information. The collected data is preprocessed to ensure the consistency of the time format. The time format is "yyyy / m / d hh:mm:ss", and the missing values ​​and duplicate data are removed. The ship position data whose speed exceeds the maximum speed of the fishing boat is deleted to ensure the accuracy of subsequent analysis. The preprocessed part of the Zhelingyu00028 data is shown in Table 1;

[0049] Table 1 is the pre-processed ship position data of this embodiment - Zhelingyu00028 is taken as an example

[0050]

[0051]

[0052] 2. GeoHash code generation:

[0053] Perform GeoHash encoding on the latitude and longitude of the current ship position, and convert the geographic coordinates into a string form. Select 7-bit GeoHash encoding for precision. This precision is sufficient to balance spatial resolution and processing efficiency, and is suitable for the analysis needs of most fishing vessel trajectories. Use Python's geohash library to perform 7-bit GeoHash encoding on the longitude and latitude data of the ship position in xls format. You can also use GeoHash encoding with higher precision as needed. The corresponding area size of GeoHash encoding length is shown in Table 2.

[0054] Table 2 GeoHash code length corresponding to the area size near latitude 30°

[0055] GeoHash encoding length width length 1 4604.5km 5003.8km 2 1072.5km 625.5km 3 135.1km 156.4km 4 33.9km 19.5km 5 4.2km 4.9km 6 1.1km 610.8m 7 132.2m 152.7m 8 33.1m 19.1m 9 4.1m 4.8m 10 1.0m 0.596m 11 129.1mm 149mm 12 32.3mm 18.6mm

[0056] 3. Calculate the drivable distance based on the ship speed and time interval:

[0057] Use Python to first calculate the time difference between the current ship position and the next ship position in seconds, then multiply the time difference by the speed of the fishing boat at the current ship position. The speed unit is knots, and knots need to be converted to meters per second. Finally, the distance range that the current ship position can travel at the current speed is obtained. For the drivable distance range of the last ship position, the time difference with the previous ship position is used to calculate.

[0058] 4. Geohash level and domain range adjustment:

[0059] Determine the level: Dynamically adjust the Geohash level of each ship site based on the drivable distance. The calculation method is: divide the drivable distance by 132.2 (the width / length of the Geohash with 7-bit precision), and round up the result to get the GeoHash level number.

[0060] Domain range: 1 layer represents 9 domain ranges (3×3 grids), 2 layers represent 25 domain ranges (5×5 grids), 3 layers represent 49 domain ranges (7×7 grids), and so on, covering a larger spatial range. The maximum number of layers is 8, covering 289 domain ranges. If there are more than 8 layers, the maximum number of layers is used for calculation. Figure 2 shown.

[0061] 5. Get the Geohash encoding set of multiple domains:

[0062] According to the GeoHash code of the current ship's location, and the calculated level and domain range of the point, Python's geohash library is used to obtain all the domain GeoHash codes within the multi-layer domain range of the ship's location, forming a Geohash code set for the domain range of the ship's location, representing all possible spatial locations where the ship's location may travel.

[0063] 6. Intersection detection:

[0064] For each ship site, obtain the code set of its GeoHash multi-layer domain range, and use Python to implement the intersection detection of the GeoHash code set of the current ship site with the GeoHash set of the previous ship site and the GeoHash set of the next ship site. If there is an identical GeoHash code (that is, the domain range overlaps), it is considered that the current point and the adjacent point form a valid trajectory link. Use Python to visualize the multi-layer domain range of each ship site. If there is a domain overlap, it is marked in blue. If there is no intersection with the previous and next ship sites, the point is identified as an abnormal point and marked in red, such as Figure 3 shown.

Claims

1. A method for identifying abnormal points in fishing vessel trajectories and cleaning data, characterized in that: The following steps are involved: Obtaining and preprocessing the trajectory data of the fishing boat, wherein the trajectory data includes the time, longitude, latitude and speed of each boat position; Generate the corresponding Geohash code according to the longitude and latitude of each ship location; The drivable distance is calculated based on the time interval between the current ship position and the next ship position and the ship speed, and then the Geohash codes of all reachable areas of the fishing boat within the time interval are analyzed and obtained as the range code set of the current ship position according to the drivable distance and the Geohash code of the current ship position; The range code set of the current ship position is compared with the range code set of the previous ship position and the range code set of the next ship position. If there is no intersection, the current ship position is identified as an abnormal point and deleted from the trajectory data.

2. The method according to claim 1, characterized in that The Geohash codes of all reachable areas of the fishing boat within the time interval are analyzed and obtained as a range code set of the current ship position according to the drivable distance and the Geohash code of the current ship position, including: Calculate the grid cell side length according to the accuracy of Geohash coding; The result obtained by dividing the drivable distance by the side length of the grid unit is rounded up to obtain the GeoHash level; according to the Geohash code and GeoHash level of the current ship position, a number of grid units constituting its scope are determined, and the Geohash codes corresponding to the several grid units are used as the range code set of the current ship position.

3. The method according to claim 2, characterized in that The plurality of grid cells constituting the scope of the current ship position are determined according to the Geohash code and GeoHash level, including: Taking the grid cell corresponding to the Geohash code of the current ship position as the center, expand outward by a set number of grid cells along the four sides, and the set number is equal to the GeoHash level of the current ship position; The rectangular area composed of the central grid unit and the extended grid unit is taken as the scope of the current ship position.

4. The method according to claim 2, characterized in that: The precision of Geohash encoding is not less than 7.

5. The method according to claim 1, characterized in that If the current ship position is the last ship position, the drivable distance is calculated based on the time interval between the current ship position and the previous ship position and the ship speed.

6. The method according to claim 1, characterized in that If the range code set of the current ship point does not have the same Geohash code as the range code set of the previous ship point, and the range code set of the current ship point does not have the same Geohash code as the range code set of the next ship point, then it is considered that the range code set of the current ship point has no intersection with the range code set of the previous ship point and the range code set of the next ship point.

7. The method according to claim 1, characterized in that The drivable distance is calculated by the following formula Driving distance = ship speed × time interval × speed unit conversion factor.

8. The method according to claim 1, characterized in that: The steps of preprocessing the trajectory data include: standardizing the time format; Removing missing values ​​and duplicate data from the trajectory data; Delete the ship position data whose speed exceeds the maximum speed of the fishing vessel.

Images