A fishing vessel trajectory anomaly detection method based on deep learning

Abstract

A method for detecting abnormal fishing vessel trajectories based on deep learning. It mainly solves the problems of poor applicability and low detection rate in detecting abnormal fishing vessel trajectories. The present invention integrates the features of fishing vessel trajectory data, fishing vessel operation mode and marine environment data, and proposes a real-time abnormality detection scheme for fishing vessel trajectories based on deep learning. The scheme first improves the Seq2Seq structure by using a dual attention mechanism, designs a fishing vessel trajectory prediction model (DA‑Seq2Seq), and then analyzes the predicted value and the true value and compares them according to the set threshold to detect abnormal trajectories. Experimental analysis shows that the algorithm's detection rate of abnormal fishing vessel trajectories can reach 86.6%, and the false detection rate is only 2.5%, indicating that the algorithm can detect fishing vessels with abnormal trajectories more timely and effectively.

Description

Technical Field

[0001] The present invention relates to the field of safe production in marine fishing industry, and in particular to a method for detecting abnormal fishing vessel trajectories based on deep learning. Background Art

[0002] Marine fishing refers to a production business that obtains marine animal and plant products with economic value through different types of fishing boats. However, in the actual development of the marine fishing industry, due to the complex and changeable marine environment, the harsh working environment of fishermen at sea, frequent dangerous situations such as fishing boat collisions and anchoring, and the existence of many no-fishing areas and specific restricted areas in other countries' waters. Fishing boats may mistakenly enter no-fishing areas or cross-border fishing during operations, which will pose a serious threat to the safety of fishermen's lives and property in the production process of the marine fishing industry. Therefore, how to establish a reasonable and effective way to supervise fishing vessels, strengthen the supervision of fishing vessels, and protect the safety of fishermen's lives and property has become one of the important research contents to promote the healthy and sustainable development of the marine fishing industry.

[0003] Real-time anomaly detection of ship tracks is an important part of early warning and monitoring. It aims to quickly and accurately detect abnormal trajectories that do not match the expected trajectory, so as to detect abnormal ships in time and avoid dangerous problems such as collisions between commercial and fishing vessels, deviation from the route, and entering restricted areas. As a special type of ship, fishing boats are also very important to select appropriate methods to detect abnormal situations in fishing boat trajectories in order to protect the lives and property of fishermen. In order to realize the abnormal detection of target ship trajectories, Du Zhiqiang from Wuhan University proposed a fast detection method for abnormal behavior of ship trajectories based on Kalman filtering, which realized the rapid detection and alarm of abnormal behavior of ship AIS trajectories. The correctness and time consumption of the experimental results were analyzed. The results showed that the model can meet the application requirements of instant abnormal discovery and instant processing. Han Zhaorong et al. from the University of Chinese Academy of Sciences proposed a trajectory anomaly point detection algorithm based on a bidirectional long short-term memory network model. The trajectory anomaly detection problem was converted into a supervised classification problem by using its ability to automatically learn normal points and abnormal points. The recall rate can reach 0.902, which verifies the effectiveness of this method. Zhao Liangbin and others from Dalian Maritime University used the ship track data in the clustering results as the training data of LSTM and trained it. Based on this, they built a ship track anomaly detector and performed real-time marine anomaly detection. The results showed that this method can also quickly detect abnormal behavior of ships in terms of speed, heading and route, but it only used the trajectory data of cruise ships and cargo ships for experiments, and did not verify its effectiveness on fishing vessel trajectory data. From the above research, it can be seen that most of the ship track anomaly detection methods based on prediction and neural networks have problems such as only being able to detect a single trajectory point and not being able to process large-scale online data in a timely manner. However, they also have the advantages of relatively fast and accurate detection results, the ability to predict danger in advance, and strong scalability. Therefore, this application will also further study the important content of real-time anomaly detection of fishing vessel trajectories based on fishing vessel data on this basis. Summary of the invention

[0004] In order to overcome the shortcomings of the background technology, the present invention provides a method for detecting fishing vessel trajectory anomalies based on deep learning, which mainly solves the problems of poor applicability and low detection rate in detecting abnormal fishing vessel trajectories.

[0005] The technical solution adopted by the present invention is: a method for detecting abnormal fishing vessel trajectory based on deep learning, comprising the following steps:

[0006] Step 1: Prepare the data set, which is the spatiotemporal trajectory points reported by the Beidou shipborne terminal on the fishing boat every 10 to 20 minutes. The data includes the Beidou ID, longitude and latitude, and speed dynamic data of the fishing boat;

[0007] Step 2: Design of DA-Seq2Seq trajectory prediction model.

[0008] The DA-Seq2Seq trajectory prediction model is improved by using the encoder-decoder and dual attention; it includes the following sub-steps:

[0009] Sub-step 1: Take N single time step feature sequences as input, recorded as: in That is, it represents the nth feature variable at the current time step t; at this time, the attention weight of each feature at the current moment can be obtained by feature attention calculation, and the calculation formula is as follows:

[0010]

[0011] Where V e and W e is the trainable weight matrix of the neuron, b e represents the bias coefficient, Represents the attention weight coefficient corresponding to the nth feature variable at time t, and s indicates that the weight coefficient is mapped to the interval (0,1) using the sigmoid activation function;

[0012] Then use the softmax function to Perform normalization processing as shown in formula (4-2); then obtain the characteristic correlation coefficient after normalization The weighted feature vectors can be obtained by calculating the corresponding features that affect the prediction results. As shown in formula (4-3); finally As the input of the encoder layer, the operation shown in formula (4-4) can be performed to obtain the hidden layer state value h at time t t ;

[0013]

[0014] Sub-step 2: First, the hidden state h output by the encoder layer t (t=1,2,…T) is used as the input of temporal attention. The hidden state s of the decoder at time step t'-1 can be calculated by formula (4-5) t'-1 The correlation l with the hidden state of the encoder at time step t t't , and normalize it as well, as shown in formula (4-6), we can get the attention β t't ,

[0015] l t't =a(s t'-1 ,h t ) (4-5)

[0016]

[0017] At this time, the attention β t'tThe background vector c related to the predicted state of the decoder at time t' is obtained by weighted summing of all hidden state values ​​of the encoder t ', the calculation formula is shown in (4-7); then in the decoder stage, use formula (4-8) to convert the output y of the previous step of the decoder t'-1 , background vector c t' And the hidden state s of the previous step t'-1 Transformed into the hidden state s at the current time t' t' ; Finally, the predicted value y of the trajectory point at time t' can be output through the Dense layer t' , as shown in formula (4-9), the predicted values ​​of the trajectory points at each moment are combined to obtain the predicted fishing boat trajectory sequence;

[0018]

[0019] s t' =BiGRU dec (y t'-1 ,c t' ,s t'-1 ) (4-8)

[0020] y t' =β(Vs t' +b) (4-9)

[0021] Step 3: Fishing boat trajectory anomaly detection,

[0022] By selecting a reasonable threshold in the subsequent experiment, each time-step trajectory point predicted by the prediction model is compared and analyzed with the newly arrived actual trajectory point according to the set threshold, and the trajectory point exceeding the threshold is determined as an abnormal trajectory point and an early warning is issued;

[0023] Or when four consecutive track points are abnormal, it is determined as a track segment abnormality and an alarm is issued;

[0024] Or use the ray algorithm to calculate whether the trajectory point of the last time step predicted by the prediction model is located in the no-fishing area, so as to detect the abnormal situation that the fishing boat is about to enter the no-fishing area and issue an early warning;

[0025] Or use the ray algorithm to calculate whether the latest actual trajectory point is located in the no-fishing area, so as to detect the abnormal situation of fishing boats entering the no-fishing area and issue an alarm.

[0026] The encoder-decoder is BiGRU.

[0027] Step 1 also includes using the Spark Streaming streaming processing framework to receive source data in real time, perform feature fusion, and extract trajectory segments.

[0028] The beneficial effect of the present invention is that the present invention integrates the features of fishing vessel trajectory data, fishing vessel operation mode and marine environment data, and proposes a real-time anomaly detection scheme for fishing vessel trajectories based on deep learning. The scheme first improves the Seq2Seq structure by using a dual attention mechanism, designs a fishing vessel trajectory prediction model (DA-Seq2Seq), and then analyzes the predicted value and the true value and compares them according to the set threshold to detect abnormal trajectories. Experimental analysis shows that the algorithm's fishing vessel trajectory anomaly detection rate can reach 86.6%, and the false detection rate is only 2.5%, indicating that the algorithm can detect fishing vessels with abnormal trajectories more timely and effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the DA-Seq2Seq trajectory prediction model.

[0030] Figure 2 Schematic diagram of the changes in MSE for different models.

[0031] Figure 3 Schematic diagram of the actual prediction effect of different model trajectories.

[0032] Figure 4 Schematic diagram of trajectory point anomaly detection experimental results.

[0033] Figure 5 Flowchart of real-time anomaly detection for fishing vessel trajectories. DETAILED DESCRIPTION

[0034] The present invention is further described below in conjunction with the accompanying drawings: A fishing boat trajectory anomaly detection method based on deep learning comprises the following steps:

[0035] Step 1: Prepare the data set, which is the spatiotemporal trajectory points reported by the Beidou shipborne terminal on the fishing boat every 10 to 20 minutes. The data includes the Beidou ID, longitude and latitude, and speed dynamic data of the fishing boat;

[0036] Step 2: Design of DA-Seq2Seq trajectory prediction model.

[0037] The DA-Seq2Seq trajectory prediction model is improved by using the encoder-decoder and dual attention; it includes the following sub-steps:

[0038] Sub-step 1: Take N single time step feature sequences as input, recorded as: in That is, it represents the nth feature variable at the current time step t; at this time, the attention weight of each feature at the current moment can be obtained by feature attention calculation, and the calculation formula is as follows:

[0039]

[0040] Where V e and W e is the trainable weight matrix of the neuron, b e represents the bias coefficient, Represents the attention weight coefficient corresponding to the nth feature variable at time t, and s indicates that the weight coefficient is mapped to the interval (0,1) using the sigmoid activation function;

[0041] Then use the softmax function to Perform normalization processing as shown in formula (4-2); then obtain the characteristic correlation coefficient after normalization The weighted feature vectors can be obtained by calculating the corresponding features that affect the prediction results. As shown in formula (4-3); finally As the input of the encoder layer, the operation shown in formula (4-4) can be performed to obtain the hidden layer state value h at time t t ;

[0042]

[0043] Sub-step 2: First, the hidden state h output by the encoder layer t (t=1,2,…T) is used as the input of temporal attention. The hidden state s of the decoder at time step t'-1 can be calculated by formula (4-5) t'-1 The correlation l with the hidden state of the encoder at time step t t't , and normalize it as well, as shown in formula (4-6), we can get the attention β t't ,

[0044] l t't =a(s t'-1 ,h t ) (4-5)

[0045]

[0046] At this time, the attention β t't The background vector c related to the predicted state of the decoder at time t' is obtained by weighted summing of all hidden state values ​​of the encoder t' , the calculation formula is shown in (4-7); then in the decoder stage, use formula (4-8) to convert the output y of the previous step of the decoder t'-1 , background vector c t' And the hidden state s of the previous step t'-1 Transformed into the hidden state s at the current time t' t' ; Finally, the predicted value y of the trajectory point at time t' can be output through the Dense layer t', as shown in formula (4-9), the predicted values ​​of the trajectory points at each moment are combined to obtain the predicted fishing boat trajectory sequence;

[0047]

[0048] s t' =BiGRU dec (y t'-1 ,c t' ,s t'-1 ) (4-8)

[0049] y t' =β(Vs t' +b) (4-9)

[0050] Step 3: Fishing boat trajectory anomaly detection,

[0051] By selecting a reasonable threshold in the subsequent experiment, each time-step trajectory point predicted by the prediction model is compared and analyzed with the newly arrived actual trajectory point according to the set threshold, and the trajectory point exceeding the threshold is determined as an abnormal trajectory point and an early warning is issued;

[0052] Or when four consecutive track points are abnormal, it is determined as a track segment abnormality and an alarm is issued;

[0053] Or use the ray algorithm to calculate whether the trajectory point of the last time step predicted by the prediction model is located in the no-fishing area, so as to detect the abnormal situation that the fishing boat is about to enter the no-fishing area and issue an early warning;

[0054] Or use the ray algorithm to calculate whether the latest actual trajectory point is located in the no-fishing area, so as to detect the abnormal situation of fishing boats entering the no-fishing area and issue an alarm.

[0055] The encoder-decoder is BiGRU.

[0056] Step 1 also includes using the Spark Streaming streaming processing framework to receive source data in real time, perform feature fusion, and extract trajectory segments.

[0057] The fishing boat trajectory anomaly detection method proposed in this application is based on trajectory prediction, so a reliable trajectory prediction model needs to be designed. Considering that a single trajectory point caused by equipment failure and other reasons does not match the predicted trajectory point and cannot directly determine that the fishing boat trajectory is abnormal, the designed trajectory prediction model must be able to quickly and accurately predict multiple trajectory points to form a trajectory sequence. Therefore, this application selects the sequence-to-sequence (Seq2Seq) model as the base model for trajectory prediction, and designs and uses a dual attention mechanism for improvement on this basis.

[0058] The basic principle of the improved Seq2Seq model is to design a feature attention mechanism on the encoder side, give more weight to important features, and reduce or even ignore the impact of unimportant features on the prediction results. At the same time, the temporal attention mechanism is integrated on the decoder side to solve the problem in the original Seq2Seq sequence model that the amount of information increases due to the increase in time steps, which leads to the weakening of the decoder's ability to extract information at key moments.

[0059] The DA-Seq2Seq trajectory prediction model is improved by using the encoder-decoder and dual attention; it includes the following sub-steps:

[0060] Step 3: Fishing boat trajectory anomaly detection,

[0061] This application mainly detects four abnormal situations: abnormalities of about to enter a prohibited fishing area, abnormalities of being in a prohibited fishing area, abnormalities of track points, and abnormalities of track segments. The details are as follows

[0062] By selecting a reasonable threshold in the subsequent experiment, each time-step trajectory point predicted by the prediction model is compared and analyzed with the newly arrived actual trajectory point according to the set threshold, and the trajectory point exceeding the threshold is determined as an abnormal trajectory point and an early warning is issued;

[0063] Or when four consecutive track points are abnormal, it is determined as a track segment abnormality and an alarm is issued;

[0064] Or use the ray algorithm to calculate whether the trajectory point of the last time step predicted by the prediction model is located in the no-fishing area, so as to detect the abnormal situation that the fishing boat is about to enter the no-fishing area and issue an early warning;

[0065] Or use the ray algorithm to calculate whether the latest actual trajectory point is located in the no-fishing area, so as to detect the abnormal situation of fishing boats entering the no-fishing area and issue an alarm.

[0066] The encoder-decoder is BiGRU.

[0067] Step 1 also includes using the Spark Streaming streaming processing framework to receive source data in real time, perform feature fusion, and extract trajectory segments.

[0068] In order to verify the effectiveness of the DA-Seq2Seq fishing vessel trajectory prediction model proposed in this application, a comparative experiment was designed to compare it with four models, including a trajectory sequence prediction model based on traditional Seq2Seq and a trajectory prediction model based on GRU. The experimental results are as follows: Figure 2 and 3As shown. It can be seen from the figure that when predicting the first time step trajectory point, the errors of the DA-Seq2Seq model and the traditional Seq2Seq model proposed in this application are both greater than the GRU model, but the difference is not large; however, when the number of predicted trajectory points is two or more, the error of the DA-Seq2Seq model proposed in this application is significantly smaller than that of the other four comparison models, and the fit between its predicted value and the true trajectory is also better. On the one hand, this is because the GRU trajectory prediction model needs to recursively predict multiple times to output a complete prediction sequence, but since there are errors in the prediction of each step, bringing the erroneous prediction value into the prediction often causes a larger error, resulting in error accumulation and transmission. Therefore, when outputting the predicted value of the fourth time step, its error is much greater than that of the DA-Seq2Seq model; on the other hand, this is because this article uses feature attention and temporal attention mechanisms to improve the traditional sequence-to-sequence model. Therefore, it can be seen from the figure that no matter how many time steps of predicted values ​​are output, the DA-Seq2Seq trajectory prediction model proposed in this application has lower error values ​​than the traditional sequence-to-sequence model and better stability. It can also be seen from the figure that the prediction error value of the LSTM model at each time step is not much different from that of the GRU model. This is because the LSTM model has a gating mechanism similar to the GRU model, but its structure is more complex and requires higher computing power and time cost. The BP neural network has the worst prediction effect in the figure. This is because its essence is the approximation of the linear weight function, which cannot fully mine the internal pattern of the time series data, while the recurrent neural network and its various variant structures can meet this demand. The above experimental results and analysis show that the DA-Seq2Seq fishing vessel trajectory prediction model proposed in this application, which uses the GRU model as the codec and uses the dual attention mechanism to improve the trajectory prediction effect.

[0069] The previous experiments have proved that the trajectory prediction model proposed in this application can well mine the information of normal trajectories. Therefore, this group of experiments will take the mean square error of each time step of the DA-Seq2Seq prediction model as the threshold (U1), and take the mean square error of each time step of the GRU rolling prediction as the threshold (U2), and conduct anomaly detection experiments on fishing boat trajectory points to verify the effectiveness of the fishing boat trajectory real-time anomaly detection algorithm (FVTRAD) proposed in this paper. The experimental results are shown in the figure. Figure 4As shown, according to the calculation formula of detection rate and false alarm rate, it can be known that when (U1) and (U2) are used as the thresholds for trajectory point anomaly detection, the false detection rates FAR are 2.5% and 3.3% respectively, which are not much different. However, the detection rate DR of the former is 86.6%, while that of the latter is only 75.5%. This is because the single-step rolling prediction model represented by GRU has the phenomenon of error transmission when performing multi-step prediction, so that it is more likely to fail to detect abnormal trajectory points due to unreasonable threshold setting. The fishing boat trajectory anomaly detection algorithm designed by this application can avoid this phenomenon, improve the detection rate of trajectory point anomaly detection, and reduce the false alarm rate. This group of experiments proves the accuracy of the fishing boat abnormal trajectory detection algorithm proposed in this application.

[0070] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention. This specific implementation method is described by taking the spatiotemporal trajectory point data reported by the Beidou ship-borne terminal carried by a Shandong-registered fishing boat every 10 minutes or so as an example.

[0071] (1) Data processing: Use the Spark Streaming streaming processing framework to receive source data in real time, perform feature fusion, and extract trajectory segments.

[0072] (2) Trajectory prediction. The DA-Seq2Seq trajectory prediction model was written and trained in the pytorch environment. Then, the online prediction code was written, the model was embedded into the Spark UDF function and the task was submitted to achieve model parallelization. Finally, the extracted trajectory segments were input into the prediction model to complete the prediction of the fishing boat trajectory in the next time period.

[0073] (3) Anomaly detection. The main abnormalities are the following: the abnormality of about to enter the no-fishing area, the abnormality of being in the no-fishing area, the abnormality of the trajectory point, and the abnormality of the trajectory segment. ① By selecting a reasonable threshold in the subsequent experiment, the trajectory points predicted by the prediction model at each time step are compared and analyzed one by one with the newly arrived actual trajectory points according to the set threshold, and the trajectory points exceeding the threshold are determined as abnormal trajectory points and an alarm is issued. ② When four consecutive trajectory points are abnormal, it is determined that the trajectory segment is abnormal and an alarm is issued. ③ Use the ray algorithm to calculate whether the trajectory point predicted by the prediction model at the last time step is located in the no-fishing area, so as to detect the abnormal situation of the fishing boat about to enter the no-fishing area and issue an alarm. ④ Use the ray algorithm to calculate whether the latest actual trajectory point is located in the no-fishing area, so as to detect the abnormal situation of the fishing boat entering the no-fishing area and issue an alarm.

[0074] The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention. The embodiments should not be regarded as limiting the present invention, but any improvements made based on the spirit of the present invention should be within the scope of protection of the present invention.

Claims

1. A fishing vessel trajectory anomaly detection method based on deep learning, characterized by: The following steps are included: Step 1: Prepare the data set, which is the spatiotemporal trajectory points reported by the Beidou shipborne terminal on the fishing boat every 10 to 20 minutes. The data includes the Beidou ID, longitude and latitude, and speed dynamic data of the fishing boat; Step 2: Design of DA-Seq2Seq trajectory prediction model. Using encoder-decoder and dual attention to improve the DA-Seq2Seq trajectory prediction model; It includes the following sub-steps: Sub-step 1: Take N single time step feature sequences as input, recorded as: in That is, it represents the nth feature variable at the current time step t; at this time, the attention weight of each feature at the current moment can be obtained by feature attention calculation, and the calculation formula is as follows: Where V e and W e is the trainable weight matrix of the neuron, b e represents the bias coefficient, Represents the attention weight coefficient corresponding to the nth feature variable at time t, and s indicates that the weight coefficient is mapped to the interval (0,1) using the sigmoid activation function; Then use the softmax function to Perform normalization processing as shown in formula (4-2); then obtain the characteristic correlation coefficient after normalization The weighted feature vectors can be obtained by calculating the corresponding features that affect the prediction results. As shown in formula (4-3); finally As the input of the encoder layer, the operation shown in formula (4-4) can be performed to obtain the hidden layer state value h at time t t ; Sub-step 2: First, the hidden state h output by the encoder layer t , t=1,2,..T is used as the input of temporal attention. The hidden state s of the decoder at time step t'-1 can be calculated by formula (4-5) t'-1 The correlation l with the hidden state of the encoder at time step t t ' t , and normalize it as well, as shown in formula (4-6), we can get the attention β t't , l t't =a(s t'-1 ,h t ) (4-5) At this time, the attention β t't The background vector c related to the predicted state of the decoder at time t' is obtained by weighted summing of all hidden state values ​​of the encoder t' , the calculation formula is shown in (4-7); then in the decoder stage, use formula (4-8) to convert the output y of the previous step of the decoder t'-1 , background vector c t' And the hidden state s of the previous step t'-1 Transformed into the hidden state s at the current time t' t' ; Finally, the predicted value y of the trajectory point at time t' can be output through the Dense layer t' , as shown in formula (4-9), the predicted values ​​of the trajectory points at each moment are combined to obtain the predicted fishing boat trajectory sequence; s t' =BiGRU dec (y t'-1 ,c t' ,s t'-1 ) (4-8) y t' =β(Vs t' +b) (4-9) Step 3: Fishing boat trajectory anomaly detection, By selecting a reasonable threshold in the subsequent experiment, each time-step trajectory point predicted by the prediction model is compared and analyzed with the newly arrived actual trajectory point according to the set threshold, and the trajectory point exceeding the threshold is determined as an abnormal trajectory point and an early warning is issued; Or when four consecutive track points are abnormal, it is determined as a track segment abnormality and an alarm is issued; Or use the ray algorithm to calculate whether the trajectory point of the last time step predicted by the prediction model is located in the no-fishing area, so as to detect the abnormal situation that the fishing boat is about to enter the no-fishing area and issue an early warning; Or use the ray algorithm to calculate whether the latest actual trajectory point is located in the no-fishing area, so as to detect the abnormal situation of fishing boats entering the no-fishing area and issue an alarm.

2. The method for detecting abnormal fishing vessel trajectories based on deep learning according to claim 1, characterized in that: The encoder-decoder is BiGRU.

3. The method for detecting abnormal fishing vessel trajectories based on deep learning according to claim 1, characterized in that: Step 1 also includes using the Spark Streaming streaming processing framework to receive source data in real time, perform feature fusion, and extract trajectory segments.

Images