Method and device for identifying double-dragging behavior of fishing boat

Abstract

The embodiment of the invention discloses a method and a device for identifying the double-towing behavior of a fishing boat, and the method comprises the steps: obtaining the trajectory data of the fishing boat, and carrying out the preprocessing, similarity calculation and data annotation; the trajectory data comprises MMSI, longitude, dimension, speed, course, time, ship type, destination and country to which the ship belongs; constructing a dynamic equilibrium loss learning framework; the dynamic equilibrium loss learning framework determines a common time interval of the first trajectory data and the second trajectory data of the fishing boat; resampling the first trajectory data and the second trajectory data in the same time interval by taking delta t as a sampling period to obtain trajectory data consisting of L sample points; determining timestamps corresponding to each sample point a'i and b'i in the trajectory data, and arranging the timestamps according to a time sequence to form time sequence data; performing fast Fourier transform, two-dimensional tensor conversion, feature extraction and splicing on the time sequence data through a constructed TimeBlock network to obtain final feature representation; weighted summation is carried out on the extracted feature representations to obtain fusion features; through the fusion feature training model, judging whether the fishing boat has a double-dragging behavior or not; in the training process, the model is optimized through a DEL imbalance loss function, so that the difference between a prediction result and an actual label is minimized.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning text data detection, and in particular to a method and device for identifying double-towed behavior of a fishing vessel. Background Art

[0002] Double tug fishing is a fishing method that can catch marine life over a large area. When double tug fishing is in operation, two fishing boats will drop large fishing nets into the deep sea to increase fishing efficiency and coverage. However, this fishing method will damage important marine life such as coral reefs and seagrass beds, destroy the seabed ecosystem, and may lead to a reduction in fishery resources and an imbalance in the ecosystem. Therefore, timely and effective identification of double tug fishing and taking corresponding punishment and governance measures can prevent illegal fishing, maintain the health of the marine ecological environment and the sustainable use of fishery resources.

[0003] The trajectory data of the double-tugboat is continuously updated over time, forming long-range, continuous time series data. In recent years, a large number of Transformer-based solutions have emerged for long-range time series tasks. Informer uses a self-attention distillation mechanism to highlight the dominant attention by halving the cascade layer input to handle ultra-long input sequences. Autoformer breaks the time series data preprocessing method of serial decomposition and designs an automatic correlation mechanism to decompose complex long-range time series patterns.

[0004] Pyraformer introduces a pyramid attention module to explore the multi-resolution representation of time series data and model the temporal dependencies of different ranges. FEDformer proposes Fourier enhancement blocks and wavelet enhancement blocks to capture important structures in time series through frequency domain mapping. Zeng explores the impact of various components of Transformer on extracting temporal dependencies and proposes a DLinear model to solve the long-range time series prediction task, providing a competitive new baseline.

[0005] Moreover, compared with other ship behaviors, the behavior of double-tug fishing boats accounts for only a small proportion. For the recognition model based on deep learning, a small number of double-tug fishing boat behaviors are small categories, and a large number of other ship behaviors are large categories. That is, there is a serious imbalance problem between large and small class samples. In response to imbalanced learning, Tan proposed Equalization Loss (EQL) to alleviate the gradient suppression of large classes on small classes. EQLv2 is an upgraded version of EQL, which uses a new gradient guidance mechanism to re-measure the loss contribution of each class. Seesawloss dynamically rebalances the gradients of large and small class samples, reducing the penalty for small class samples. The Equalized Focal loss function starts from the imbalance degree of large and small class samples and rebalances according to the contribution of different classes to the loss. He proposed Focal loss, which distributes the weights of difficult and easy samples through a dynamic scaling factor, and quickly focuses on those difficult-to-distinguish samples. Leng proposed Polyfocal loss after Taylor expansion of cross entropy loss. Polyfocal loss is a linear combination of polynomial functions, and the gain is obtained by adjusting the first polynomial coefficient. Cui uses the effective number of samples for each category to rebalance the loss function.

[0006] Based on the above observations, it can be found that during the double towing operation, the two fishing vessels have highly similar speeds and headings, e.g. Figure 1 (a) and 1(b). The navigation tracks of the two fishing boats maintain a relatively fixed distance and longitude and latitude position relationship, showing significant time series synchronization characteristics, such as Figure 1 Moreover, in real-world scenarios, compared with other ship behaviors, the probability of double-tow behavior is extremely low, and its rarity leads to serious class imbalance in the training dataset. Summary of the invention

[0007] Based on this, it is necessary to propose a method and device for identifying the double-towed behavior of fishing vessels in response to the above problems.

[0008] An embodiment of the present invention provides a method for identifying double-tow behavior of a fishing vessel, the method comprising:

[0009] Obtain the trajectory data of the fishing boat, and perform preprocessing, similarity calculation, and data annotation; the trajectory data includes MMSI, longitude, latitude, speed, heading, time, ship type, destination, and country; build a dynamic balanced loss learning framework;

[0010] The dynamic balanced loss learning framework determines a common time interval of the first trajectory data and the second trajectory data of the fishing vessel;

[0011] Resample the first trajectory data and the second trajectory data in the common time interval respectively with Δt as the sampling period to obtain trajectory data consisting of L sample points;

[0012] Determine each sample point a in the trajectory data i ′ and b i ′The corresponding timestamps are arranged in chronological order to form time series data;

[0013] The time series data is subjected to Fast Fourier Transform (FFT), two-dimensional tensor conversion, feature extraction and splicing through the constructed TimesBlock network to obtain the final feature representation;

[0014] Perform weighted summation on the extracted feature representations to obtain fusion features;

[0015] The model is trained by the fusion feature to determine whether the fishing vessel has double towing behavior;

[0016] During the training process, the model is optimized through the DEL imbalance loss function to minimize the difference between its predicted results and the actual labels.

[0017] In the above scheme, the dynamic equalized loss learning framework includes a synchronous time series module (Synchronous Time series Module, STM), a TimesBlocks feature extraction module, and a dynamic equalized loss (DynamicEqualized Loss, DEL).

[0018] In the above scheme, the dynamic balanced loss learning framework determines the common time interval of the first trajectory data and the second trajectory data of the fishing boat, specifically including: when the time intervals of the first trajectory and the second trajectory are T1 and T2 respectively, and satisfy The synchronized time series module extracts the common time interval of the two trajectories

[0019] In the above scheme, the first trajectory data and the second trajectory data in the common time interval are resampled with Δt as the sampling period to obtain trajectory data composed of L sample points, which specifically includes: resampling the two trajectories according to the sampling period Δt=120s, and then standardizing the resampled trajectory length to L; if the resampled sample length is less than L, filling with 0, and finally obtaining trajectory data composed of L sample points, A L ={a′0,a′1,...,a′ l′ ,0,...,0},B L ={b′0,b′1,...,b′l′ ,0,...,0}; where A L is the first trajectory data, B L is the first trajectory data, a′ i and b′ i is the sample point.

[0020] In the above scheme, the determination of each sample point a′ in the trajectory data i and b′ i The corresponding timestamps are arranged in chronological order to form time series data, including: each sample point a′ i and b′ i The corresponding timestamp must satisfy t0<t1<...<t n , each track point of the two fishing boats is arranged in time sequence, forming an ordered time series data.

[0021] In the above scheme, the TimesBlock network constructed performs FFT transformation, two-dimensional tensor conversion, feature extraction and splicing on the time series data to obtain the final feature representation, which specifically includes:

[0022] Perform FFT transformation on the input time series data, expressed as Where X[k] represents the value of the frequency domain signal at frequency index k; x[n] represents the value of the time domain signal at time index n; -j(2π / N)kn is the rotation factor used to map the time domain signal to the frequency domain;

[0023] The time series data after FFT processing is converted into a two-dimensional tensor, and the Inception block is used to extract features through parallel convolution kernels of different sizes in two-dimensional space, and the extracted features are spliced ​​to form the final feature representation. Among them, C1, C(82), C3 represent the outputs of different convolution kernels, and P represents the output of the maximum pooling. Represents feature concatenation operation;

[0024] The final feature representation is converted from two dimensions to one dimension, and useful feature representations are extracted through an adaptive aggregation method to complete the classification of the double-drag behavior.

[0025] In the above scheme, during the training process, the model is optimized by the DEL imbalance loss function to minimize the difference between the predicted result and the actual label, specifically including: the DEL loss function is composed of the mitigation factor M ij , compensation factor C ij and feedback factor g i constitute;

[0026] During the training process, DEL dynamically counts the number of small class samples N i and the number of samples in the large category N j , and construct the mitigation factor, for a trajectory sequence of length N, expressed as: M ij =(N i / N j ) p ,if N j >N i ; p is a hyperparameter for adjusting the degree of mitigation. j ≤N i When M ij =1, that is, the gradient ratio of small-class samples and large-class samples during training is not changed. When N j >N i When , large-class samples appear more frequently than small-class samples, M ij It can alleviate the gradient suppression of a large number of large-class samples on small-class samples;

[0027] During the training process, p i is the probability that the sample is predicted to be a small class, p j is the probability that the sample is predicted to be a large category. DEL dynamically constructs the compensation factor, C ij =(p i / p j ) q ,if p i >p j ; Among them, q is a hyperparameter for adjusting the degree of compensation; for samples in category j, if its prediction appears p i >p j (i≠j), using the compensation factor C ij Increase the penalty for false positive samples;

[0028] In each round of training, assume that the total number of small class samples is s, denoted by T s , the number of correctly classified small class samples is T i , the total number of misclassified small class samples is T s -T i , g i is a dynamic indicator g i =(T s -T i ) / T s .

[0029] The embodiment of the present invention also provides a device for identifying double-tow behavior of a fishing vessel, the device comprising: a data acquisition and processing module, a network construction module, and a training module;

[0030] The data acquisition and processing module is used to obtain the trajectory data of the fishing vessel, and perform preprocessing, similarity calculation, and data labeling; the trajectory data includes MMSI, longitude, latitude, speed, heading, time, vessel type, destination, and country;

[0031] The network construction module is used to construct a dynamic balanced loss learning framework; it is also used to determine the common time interval of the first trajectory data and the second trajectory data of the fishing boat; resample the first trajectory data and the second trajectory data in the common time interval with Δt as the sampling period to obtain trajectory data composed of L sample points; determine each sample point a in the trajectory data i ′ and b i ′ and arrange the corresponding timestamps in chronological order to form time series data; and also used to perform FFT transformation, two-dimensional tensor conversion, feature extraction and splicing on the time series data through the constructed TimesBlock network to obtain the final feature representation;

[0032] The training module is used to perform weighted summation on the extracted feature representations to obtain fused features; train the model through the fused features to determine whether the fishing vessel has double towing behavior; and optimize the model through the DEL imbalance loss function during the training process to minimize the difference between its predicted results and the actual labels.

[0033] The embodiments of the present invention have the following beneficial effects:

[0034] The present invention constructs a dynamic-equalized-lossbased learning framework to ensure the high temporal synchronization of the behavior of double-tug fishing vessels, captures the time domain and frequency domain changes of trajectory data in two-dimensional space, and improves the model's recognition ability for the behavior of double-tug fishing vessels. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] in:

[0037] Figure 1 The behavior of the double tugboat. (a) The speed change of the double tugboat; (b) The course change of the double tugboat; (c) The navigation track of the double tugboat;

[0038] Figure 2 AIS signal equipment for ships;

[0039] Figure 3 The DTW matrix formed by the two fishing boat trajectories;

[0040] Figure 4 A deep neural network model constructed for an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The embodiment of the present invention provides a method for identifying double-tow behavior of a fishing vessel, such as Figure 1-4 As shown, the method includes:

[0043] Step 101: Obtain the trajectory data of the fishing vessel, and perform preprocessing, similarity calculation, and data labeling; the trajectory data includes MMSI, longitude, latitude, speed, heading, time, vessel type, destination, and country;

[0044] Specifically, AIS equipment and radar systems are used to collect navigation information of ships at sea in real time, including key information such as MMSI, longitude, latitude, speed, heading, time, ship type, destination, country of origin, etc., and the information is updated at a high frequency to ensure the timeliness and accuracy of the data.

[0045] By resampling the trajectory points, the original data is unified to the same time interval, with a sampling interval of 120 seconds. At the same time, the higher density of original trajectory data increases the cost of data processing and storage. Resampling can reduce the amount of data and improve data processing efficiency.

[0046] The input data is preprocessed to unify the latitude and longitude format, speed unit, convert the timestamp into a time series, and use the ".csv" file storage format.

[0047] The Dynamic Time Warping (DTW) algorithm is used to measure the similarity of the trajectories of two fishing vessels. The trajectories of the double-tug fishing vessels are visualized and analyzed. At the same time, the double-tug fishing vessel behavior recognition dataset is labeled and produced in combination with the expert certification scheme.

[0048] Step 102: Construct a dynamic balanced loss learning framework;

[0049] Specifically, the dynamic equalized loss learning framework includes a synchronous timeseries module (Synchronous Timeseries Module, STM), a TimesBlocks feature extraction module, and a dynamic equalized loss (DynamicEqualizedLoss, DEL).

[0050] STM performs strict time alignment on binary time series trajectories to ensure the time synchronization of the two sequences; TimesBlocks first uses fast Fourier transform to mine the frequency information implicit in the periodic trajectories to construct two-dimensional time-frequency data, and uses Inception block to learn time-frequency features; the DEL loss function solves the problem of unbalanced distribution between data classes faced in learning.

[0051] Step 103: The dynamic balanced loss learning framework determines a common time interval between the first trajectory data and the second trajectory data of the fishing vessel;

[0052] Specifically, when the time intervals of the first trajectory and the second trajectory are T1 and T2 respectively, and satisfy The synchronized time series module extracts the common time interval of the two trajectories

[0053] For example, it is assumed that the time interval of trajectory A is T1={t0,...,t n}, the time interval of trajectory B is T2 = {t1,...,t m}, satisfying n≤m. STM first intercepts the common time interval of trajectory A and trajectory B To ensure that the overall time interval of the two trajectories is consistent.

[0054] Step 104: resampling the first trajectory data and the second trajectory data in the common time interval respectively with Δt as a sampling period to obtain trajectory data consisting of L sample points;

[0055] Specifically, the two trajectories are resampled according to the sampling period Δt = 120s, and then the resampled trajectory length is standardized to L; if the resampled sample length is less than L, it is filled with 0, and finally, the trajectory data consisting of L sample points is obtained, A L ={a′0,a′1,...,a′ l′ ,0,...,0},B L ={b′0,b′1,...,b′ l′ ,0,...,0}; where A L is the first trajectory data, B L is the first trajectory data, a′ i and b′ iis the sample point.

[0056] For example, the STM resamples the two navigation tracks with a sampling period of Δt, thereby constructing paired track data with one-to-one correspondence at each sampling point. For a pair of tracks consisting of track A and track B, the resampled track l = t n / Δt length. Considering that different trajectory pairs have different trajectory lengths, BTM uniformly sets the length of all trajectory pairs to L. If l<L, it is filled with 0.

[0057] Step 105: Determine each sample point a in the trajectory data i ′ and b i ′The corresponding timestamps are arranged in chronological order to form time series data;

[0058] Specifically, each sample point a′ i and b′ i The corresponding timestamp must satisfy t0<t1<...<t n ,Each trajectory point of the two fishing boats is arranged in time sequence, forming an ordered time series data.

[0059] Step 106: Perform FFT transformation, two-dimensional tensor conversion, feature extraction and splicing on the time series data through the constructed TimesBlock network to obtain the final feature representation;

[0060] Specifically, the input time series data is transformed by FFT, which is expressed as Where X[(k7]) represents the value of the frequency domain signal at frequency index k; x[n] represents the value of the time domain signal at time index n; -j(2π / N)kn is the rotation factor used to map the time domain signal to the frequency domain.

[0061] The time series data after FFT processing is converted into a two-dimensional tensor, and the Inception block is used to extract features through parallel convolution kernels of different sizes in two-dimensional space, and the extracted features are spliced ​​to form the final feature representation. Among them, C1, (C82), C3 represent the outputs of different convolution kernels, and P represents the output of the maximum pooling. Represents feature concatenation operation;

[0062] The final feature representation is converted from two-dimensional to one-dimensional form, and useful feature representations are extracted through an adaptive aggregation method.

[0063] For example, considering that time series data can usually be regarded as composed of periodic signals of different frequencies, TimesBlocks uses FFT to explicitly represent time series data in the time dimension and frequency dimension, and uses Inceptionblock to extract time-frequency features in two-dimensional space, which can better model time series data.

[0064] For each TimesBlock, the following calculation is performed. Let X represent the time domain sequence of the fishing boat trajectory, and FFT calculates its frequency {f1,...,f n},amplitude and the period {p1,...,p n}, as shown in formula (1):

[0065]

[0066] TimesBlock selects the top k frequencies with the most significant amplitudes and their corresponding periods, so that the model focuses on the most significant changes and more effectively captures and utilizes the periodic information of trajectory data.

[0067] Then, the Padding(·) operation fills the data length of X to a length that is divisible by k cycles. Further, according to the calculation result of formula (1), the Reshape(·) operation converts the padded fishing boat trajectory time domain series into a two-dimensional time-frequency tensor As shown in formula (2).

[0068]

[0069] Among them, p i is the number of rows of the transformed two-dimensional tensor, representing the i-th time period; f i is the number of columns of the transformed two-dimensional tensor, representing the i-th frequency signal.

[0070] After that, the Inception block uses multiple convolution kernels for feature extraction, which can easily capture the time domain and frequency domain features of the trajectory data. Specifically, the time-frequency changes between and within cycles of the trajectory data are extracted in two-dimensional space to better model the trajectory data.

[0071] Step 107: performing weighted summation on the extracted feature representations to obtain fused features; training a model using the fused features to determine whether the fishing vessel has double towing behavior;

[0072] Specifically, during the training process, the model is optimized through the DEL imbalance loss function to minimize the difference between its predicted results and the actual labels.

[0073] The features extracted by the Inception block are weighted and summed (the weights are based on the first k amplitude frequencies and calculated using Softmax) to obtain the fused features.

[0074] In order to alleviate the serious imbalance problem between large and small classes, the present invention designs DEL loss. DEL fully considers the imbalance degree of class distribution, the degree of sample classification error and the degree of model learning of classes, and adopts a dynamic update mechanism to balance the gradient deviation between large and small classes, and pays full attention to the degree of model learning of small classes. The DEL loss function is composed of the mitigation factor M ij , compensation factor C ij and feedback factor g i constitute;

[0075]

[0076] Among them, i represents the sample of the i-th category, and j represents the sample of the j-th category. ij is a mitigation factor, which is used to alleviate the suppression of the gradient of a large number of large-class samples on the gradient of small-class samples. ij is a compensation factor used to compensate for the penalty of false positive samples. i It is a feedback factor used to feedback the learning status of the current class and is calculated based on the confusion matrix.

[0077] (1) Mitigation Factor

[0078] During the training process, DEL dynamically counts the number of small class samples N i and the number of samples in the large category N j , and construct the mitigation factor, for a trajectory sequence of length N, it can be expressed as:

[0079] M ij =(N i / N j ) p ,if N j >N i ; where p is a hyperparameter for adjusting the degree of mitigation. j ≤N i When M ij =1, that is, the gradient ratio of small class samples and large class samples during training will not be changed. j >N i When , large-class samples appear more frequently than small-class samples, M ij It can alleviate the gradient suppression of a large number of large-category samples on small-category samples.

[0080] (2) Compensation factor

[0081] During the training process, p iis the probability that the sample is predicted to be a small class, p j is the probability that the sample is predicted to be a large category. DEL dynamically constructs the compensation factor, C ij =(p i / p j ) q ,if p i >p j ; where q is a hyperparameter of the degree of compensation. For a sample in category j, if its prediction appears p i >p j (i≠j). That is, the model misreports the large class sample as a small class, reducing the false positives of the small class. At this time, the model uses the compensation factor C ij Increase the penalty for false positive samples.

[0082] (3) Feedback factor

[0083] g i is a feedback factor used to feedback the model’s learning of small class samples. In each round of training, by dynamically calculating the proportion of small class samples in the confusion matrix that are mistakenly classified into large classes, the model can obtain g in real time. i Specifically, assuming that the total number of small class samples is s, denoted by T s The number of correctly classified small class samples is T i , the total number of misclassified small class samples is T s -T i . g i It is a dynamic indicator.

[0084] g i =(T s -T i ) / T s ;

[0085] During the training process, T s Represents the total number of fixed small class samples. When small class samples are misclassified as large classes, T s -T i The value of will increase accordingly. s -T i The increase in the value reflects the increase in the frequency of small-category samples being misclassified as large-category samples. i The value of g increases accordingly. At this time, the model pays more attention to the learning degree of small class samples. i This dynamic adjustment mechanism based on the degree of class learning can make the model more flexibly adapt to the distribution of imbalanced data sets.

[0086] In order to evaluate the effectiveness of the method proposed in the embodiment of the present invention, the present invention has carried out a large number of experiments on two unbalanced double-drag behavior recognition datasets. The experimental results show that the present invention is significantly superior to the existing SOTA time series model and unbalanced learning method. On the dataset with an imbalance of 70:1, the present invention achieved a 13.79% increase in recall and a precision of 93.14%; on the dataset with an imbalance of 50:1, the present invention achieved an 8.48% increase in recall and a precision of 93.90%. Furthermore, DEL can be flexibly combined with existing SOTA methods and has good generalization capabilities on different models.

[0087] The embodiment of the present invention further provides a device for identifying double-tow behavior of a fishing vessel, the device comprising: a data acquisition and processing module, a network construction module, and a training module.

[0088] The data acquisition and processing module is used to obtain the trajectory data of the fishing vessel, and perform preprocessing, similarity calculation, and data labeling; the trajectory data includes MMSI, longitude, latitude, speed, heading, time, vessel type, destination, and country;

[0089] The network construction module is used to construct a dynamic balanced loss learning framework; it is also used to determine the common time interval of the first trajectory data and the second trajectory data of the fishing boat; resample the first trajectory data and the second trajectory data in the common time interval with Δt as the sampling period to obtain trajectory data composed of L sample points; determine each sample point a in the trajectory data i ′ and b i ′The corresponding timestamp is arranged in chronological order to form time series data; it is also used to perform FFT transformation, two-dimensional tensor conversion, feature extraction and splicing on the time series data through the constructed TimesBlock network to obtain the final feature representation.

[0090] The training module is used to perform weighted summation on the extracted feature representations to obtain fused features; train the model through the fused features to determine whether the fishing vessel has double towing behavior; and optimize the model through the DEL imbalance loss function during the training process to minimize the difference between its predicted results and the actual labels.

[0091] The example of the present invention first conducts real-time monitoring and data collection of ship information in the offshore waters of Hainan Province, China, and the collection equipment includes a Class A AIS system, a Class B AIS system and a radar system. Secondly, the collected raw data is resampled, outlier processing, missing value processing, visualization analysis, DTW trajectory similarity calculation, expert experience, and data set production. Then, the data sets Pair_Trawler70 and Pair_Trawler50 are produced according to two imbalanced distribution ratios, and the positive and negative sample ratios are 1:70 and 1:50 respectively. Then, STM strictly aligns the trajectory data of the double trawler in time order. And the binary fishing boat trajectory data processed by STM is input into the TimesBlocks module to further capture the two-dimensional changes of the trajectory data in two-dimensional space. Finally, DEL is used to solve the imbalance problem in the data set in a targeted manner. DEL uses a dynamic update mechanism to balance the gradient deviation between large and small categories, and pays full attention to the learning degree of the model on small categories.

[0092] Step 1: Obtain original ship navigation data

[0093] The ship's AIS automatic identification system exchanges information such as the ship's position, speed, and heading through radio signals. When a ship is sailing, its AIS equipment should always be turned on; when moored or berthed, the AIS equipment can be turned off to protect privacy. The marine radar system detects the position and distance of surrounding objects by emitting microwave signals and receiving echoes. It can identify maritime obstacles, other ships, and coastlines in bad weather and low visibility conditions. Using these two systems, the ship's navigation information is collected in real time, as shown in Table 1:

[0094] Table 1 Ship behavior perception data collection specifications

[0095]

[0096]

[0097] Step 2: Preprocess the original data, calculate similarity, and label the data

[0098] Data resampling: resample the original trajectory samples according to the time feature (lastDT), and the resampling time interval T = 120s;

[0099] Abnormal value processing: mmsi with less than 9 digits or missing mmsi, beyond the offshore longitude and latitude range or missing latitude information, beyond the heading range or missing heading information, quality information less than 50, timestamp format that does not meet expectations, etc.;

[0100] Missing value processing: use interpolation or smoothing methods to fill in missing data;

[0101] Visual analysis: speed-time variation law, heading-time variation trajectory, dynamic navigation trajectory, etc. of double-towed fishing vessels;

[0102] Step 3: DTW measures the similarity of fishing boat trajectories

[0103] For the double-tow operation of fishing vessels, since two fishing vessels operate at the same time, the similarity between the two trajectories is usually measured by a certain set of distances between the trajectory points. The core is to dynamically time warp the navigation trajectories of the two ships so that the trajectories of the two ships can be matched in time to achieve reliable identification of the double-tow operation. DTW uses a dynamic programming method to find the best match between the two sequences, allowing the sequences to be stretched or compressed in the time dimension, thereby taking into account the phase difference between the sequences.

[0104] Assume that the trajectories of the two fishing boats are Q = [q1, q2, ...q m ] and C=[c1,c2,...c n ]. The DTW algorithm obtains the similarity between two trajectories by initializing the distance matrix, calculating the local distance, dynamically calculating the optimal path, and calculating the DTW distance. The distance matrix of the two fishing boats is initialized by constructing an m*n matrix D to align the two sequences. The position (i, j) of the matrix is ​​used to store the distance between points. The distance between each pair of sequence elements (q i ,c i ), and calculate the path with the minimum cost from D(1,1) to D(m,n). The final DTW distance is the sum of the distances between the points on the optimal path, which represents the similarity between the two trajectory sequences.

[0105] Step 4: Preparation of unbalanced double-towed fishing vessel identification dataset

[0106] The present invention collects AIS data and radar data collected from October 2022 to January 2023, and further constructs two data sets, Pair_Trawler70 and Pair_Trawler50. After data cleaning, we use time, longitude, latitude, speed, and heading as trajectory features of these two double-trawling fishing vessel behavior data sets, and complete label annotation in combination with expert experience. Among them, the number of available trajectories in the Pair_Trawler70 data set is 20.6752 million, including 182 pairs of double-trawling fishing vessels and 12,740 pairs of non-double-trawling fishing vessels, with an imbalance ratio of 1:70. The number of available trajectories in the Pair_Trawler50 data set is 14.8512 million, including 182 pairs of double-trawling fishing vessels and 9,100 pairs of non-double-trawling fishing vessels, with an imbalance ratio of 1:50.

[0107] Step 5: Build the Synchronous Time Series Module STM

[0108] When the time intervals of trajectory A and trajectory B are T1 and T2 respectively, and satisfy First, STM extracts the common time interval of the two trajectories The two trajectories are resampled according to the sampling period Δt = 120s, and then the resampled trajectory length is standardized to L. If the resampled sample length is less than L, it is padded with 0. Finally, trajectory data consisting of L sample points is obtained, as shown in formula (5).

[0109] A L ={a′0,a′1,...,a l ″,0,...,0},B L ={b′0,b′1,...,b′ l′ ,0,...,0} (5)

[0110] Among them, each sample point a′ i and b′ i The corresponding timestamp must satisfy formula (6):

[0111] t0<t1<...<t n (6)

[0112] Each trajectory point of the two ships is arranged in chronological order, forming an ordered time series data, ensuring the time continuity and synchronization of the data.

[0113] Step 6: Build TimesBlock structure

[0114] In the TimesBlock network, the input data is first transformed by FFT. FFT is a method of converting time domain signals to frequency domain, which can analyze the frequency, amplitude and period in trajectory data. For a trajectory sequence of length N, it can be expressed as:

[0115]

[0116] Where X[k] represents the value of the frequency domain signal at frequency index k; x[n] represents the value of the time domain signal at time index n; -j(2π / N)kn is the rotation factor used to map the time domain signal to the frequency domain.

[0117] Secondly, the trajectory data after FFT processing is converted into a two-dimensional tensor, and the Inception block is used to extract features in the two-dimensional space. The Inception block is a convolutional neural network module that extracts features through parallel convolution kernels of different sizes (1*1, 3*3, 5*5). The extracted features are concatenated to form the final feature representation, as shown in formula (8):

[0118]

[0119] Among them, C1, C2, C3 represent the outputs of different convolution kernels, and P represents the output of the maximum pooling. Represents a feature concatenation operation.

[0120] Next, the output features of the Inception block are converted from two-dimensional to one-dimensional form, and useful feature representations are extracted through an adaptive aggregation method to further complete the classification task.

[0121] Step 7: Design DEL loss function

[0122] This patent designs DEL loss to solve the problem of imbalanced training samples. Specifically, DEL is composed of the mitigation factor M ij , compensation factor C ij and feedback factor g i constitute.

[0123] (1) Mitigation Factor

[0124] During the training process, DEL dynamically counts the number of small class samples N i and the number of samples in the large category N j , and construct the mitigation factor, as shown in formula (9).

[0125] For a trajectory sequence of length N, it can be expressed as:

[0126] M ij =(N i / N j ) p ,if N j >N i (9)

[0127] Where p is a hyperparameter for adjusting the degree of mitigation. j ≤N i When M ij =1, that is, the gradient ratio of small class samples and large class samples during training will not be changed. j >N i When , large-class samples appear more frequently than small-class samples, M ijIt can alleviate the gradient suppression of a large number of large-category samples on small-category samples.

[0128] (2) Compensation factor

[0129] During the training process, p i is the probability that the sample is predicted to be a small class, p j is the probability that the sample is predicted to be a large category. DEL dynamically constructs the compensation factor, as shown in formula (10):

[0130] C ij =(p i / p j ) q ,if p i >p j (10)

[0131] Among them, q is a hyperparameter for adjusting the degree of compensation. For a sample in the large class j, if its prediction appears p i >p j (i≠j). That is, the model misreports the large class sample as a small class, reducing the false positives of the small class. At this time, the model uses the compensation factor C ij Increase the penalty for false positive samples.

[0132] (3) Feedback factor

[0133] g i is a feedback factor used to feedback the model’s learning of small class samples. In each round of training, by dynamically calculating the proportion of small class samples in the confusion matrix that are mistakenly classified into large classes, the model can obtain g in real time. i Specifically, assuming that the total number of small class samples is s, denoted by T s The number of correctly classified small class samples is T i , the total number of misclassified small class samples is T s -T i . g i It is a dynamic indicator, as shown in formula (11).

[0134] g i =(T s -T i ) / T s (11)

[0135] During the training process, T s Represents the total number of fixed small class samples. When small class samples are misclassified as large classes, T s -T i The value of will increase accordingly. s -T iThe increase in the value reflects the increase in the frequency of small-category samples being misclassified as large-category samples. i The value of g increases accordingly. At this time, the model pays more attention to the learning degree of small class samples. i This dynamic adjustment mechanism based on the degree of class learning can make the model more flexibly adapt to the distribution of imbalanced data sets.

[0136] Step 8: Verification of recognition performance of the invention embodiment

[0137] The identification of double-tug fishing boat behavior is essentially a classification problem. If the model misclassifies a double-tug fishing boat as a non-double-tug fishing boat, it is likely to directly lead to illegal behavior and economic losses. In this case, the model is more concerned with the ability to effectively identify double-tug behavior. Therefore, the experiment uses Recall as the primary evaluation indicator. In order to more accurately evaluate the efficiency and accuracy of the algorithm, the two evaluation indicators of Accuracy and F1 score are combined at the same time.

[0138] In order to verify the effectiveness of the invention example, we first conducted comparative experiments by combining our method with three advanced time series models, including Autoformer, FEDformer, and DLinear, as shown in Table 1. Our method significantly outperforms the SOTA method. On the Pair_Trawler70 dataset, our method achieves the highest recognition performance, with 80.49% recall, 83.02% F1 score, and 93.14% accuracy.

[0139] Furthermore, the DEL of the present invention can be supplemented to multiple existing SOTA models. Table 1 shows the recognition performance of DEL across different backbone networks. For the three models of Autoformer, FEDformer, and DLinear, when DEL replaces their original loss functions, the recognition performance of the models achieves different degrees of growth. The experimental results show that our DEL has good generalization ability on different models.

[0140] Table 1 Comparison results of the method of the present invention with the existing SOTA time series model on the Pair_Trawler70 dataset. The comparison methods include Autoformer, FEDformer, and DLinear. In the experiment, the method of the present invention used STM, TimesBlocks, and DEL. √ indicates the combination of DEL and the existing SOTA method.

[0141]

[0142] As shown in Table 2, on the Pair_Trawler70 and Pair_Trawler50 datasets, the recognition performance of DEL is significantly better than the existing SOTA loss functions, including Cross-Entropy loss, Focal loss, Seesaw loss, and PolyFocal loss.

[0143] On the Pair_Trawler70 dataset, the DEL of the present invention achieved 80.49% recall, 83.02% F1 score and 93.14% accuracy. Compared with the Cross-Entropyloss baseline, DEL improved Recall, F1 score and Accuracy by 37.49%, 16.01% and 1.85% respectively, especially Recall, which increased from 58.54% to 80.49%. Compared with the advanced Seesaw loss function, the DEL of the present invention achieved a 13.79% increase in recall. It is worth noting that the greater the imbalance ratio, the greater the increase in Recall.

[0144] Table 2 Comparison results of the proposed method with the existing SOTA loss functions on the Pair_Trawler70 and Pair_Trawler50 datasets. The comparison methods include Cross-Entropy loss, Focal loss, Seesaw loss, and PolyFocal loss. In the experiment, the model used the STM and TimesBlocks feature extraction structures.

[0145] Table 2 Pair_Trawler70 and Pair_Trawler50 datasets

[0146]

[0147]

[0148] Step 9: Ablation study of the invention embodiment

[0149] In order to verify the effect of each component in DEL, this embodiment carries out ablation experiments on each component of DEL. As shown in Table 3, g i Item and M ij The three factors significantly improve the recognition results. The synergy of the three components can achieve the most powerful performance and has a good effect on alleviating the serious class imbalance problem. The synergy of the three factors increases the Recall of the improved baseline from 58.54% to 80.49%, the F1 score from 71.56% to 83.02%, and the Accuracy from 91.71% to 93.41%.

[0150] Table 3 Ablation studies of DEL components. ij is the mitigation factor, C ij is the compensation factor, g i is the feedback factor.

[0151] <![CDATA[M ij ]]> <![CDATA[C ij ]]> <![CDATA[g i ]]> Recall F1-Score Accuracy √ 75.61 81.05 92.93 √ 43.90 65.00 88.54 √ 65.83 76.32 91.96 √ √ √ 80.49 83.02 93.41

[0152] Step 10: Hyperparameter study of the invention embodiment

[0153] DEL loss has two hyperparameters p and q. p is the ij The hyperparameter of C ij As shown in Tables 4 and 5, the performance is best when p = 0.8 and q = 0.5. Too small p value has a restrictive effect on the model. When p > = 0.8, the DEL method is not sensitive to the hyperparameters and the model is robust.

[0154] Table 4. The impact of hyperparameter p-value in DEL.

[0155] p Recall F1-score Accuracy 0.2 43.90 65.00 88.54 0.4 52.44 67.72 90.00 0.6 67.07 75.86 91.46 0.8 80.49 83.02 93.41 1.0 80.49 80.49 92.2

[0156] Table 5. The influence of the hyperparameter q value in DEL.

[0157]

[0158]

[0159] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying double-tow behavior of a fishing vessel, characterized in that: The method comprises: Obtain the trajectory data of the fishing boat, and perform preprocessing, similarity calculation, and data annotation; the trajectory data includes MMSI, longitude, latitude, speed, heading, time, ship type, destination, and country; build a dynamic balanced loss learning framework; The dynamic balanced loss learning framework determines a common time interval of the first trajectory data and the second trajectory data of the fishing vessel; Resample the first trajectory data and the second trajectory data in the common time interval respectively with Δt as the sampling period to obtain trajectory data consisting of L sample points; Determine each sample point a in the trajectory data i ′ and b i ′The corresponding timestamps are arranged in chronological order to form time series data; The time series data is subjected to Fast Fourier Transform (FFT), two-dimensional tensor conversion, feature extraction and splicing through the constructed TimesBlock network to obtain the final feature representation; Perform weighted summation on the extracted feature representations to obtain fusion features; The fusion feature is used to train a model to determine whether the fishing vessel has double towing behavior; During the training process, the model is optimized through the DEL imbalance loss function to minimize the difference between its predicted results and the actual labels.

2. The method for identifying double-tow behavior of a fishing vessel according to claim 1, characterized in that: The dynamic equalized loss learning framework includes a synchronous time series module (Synchronous Time series Module, STM), a TimesBlocks feature extraction module, and a dynamic equalized loss (Dynamic Equalized Loss, DEL).

3. The method for identifying double-tow behavior of a fishing vessel according to claim 2, characterized in that: The dynamic balanced loss learning framework determines the common time interval of the first trajectory data and the second trajectory data of the fishing boat, specifically including: when the time intervals of the first trajectory and the second trajectory are T1 and T2 respectively, and satisfy The synchronized time series module extracts the common time interval of the two trajectories 4. The method for identifying double-tow behavior of a fishing vessel according to claim 3, characterized in that: The first trajectory data and the second trajectory data in the common time interval are resampled with Δt as the sampling period to obtain trajectory data consisting of L sample points, specifically including: resampling the two trajectories according to the sampling period Δt=120s, and then standardizing the resampled trajectory length to L; if the resampled sample length is less than L, padding with 0, and finally obtaining trajectory data consisting of L sample points, A L ={a′0,a1′,...,a′ l ′,0,...,0},B L ={b′0,b1′,...,b′ l ′,0,...,0}; where A L is the first trajectory data, B L is the first trajectory data, a i ′ and b i ′ is the sample point.

5. The method for identifying double-tow behavior of a fishing vessel according to claim 4, characterized in that: The step of determining each sample point a in the trajectory data i ′ and b i ′ and the corresponding timestamps are arranged in chronological order to form time series data, including: each sample point a i ′ and b i The corresponding timestamp must satisfy t0<t1<…<t n ,Each trajectory point of the two fishing boats is arranged in time sequence, forming an ordered time series data.

6. The method for identifying double-tow behavior of a fishing vessel according to any one of claims 1 to 5, characterized in that: The TimesBlock network constructed by the time series data is subjected to FFT transformation, two-dimensional tensor conversion, feature extraction and splicing to obtain the final feature representation, which specifically includes: Perform FFT transformation on the input time series data, expressed as Where X[k] represents the value of the frequency domain signal at frequency index k; x[n] represents the value of the time domain signal at time index n; -j(2π / N)kn is the rotation factor used to map the time domain signal to the frequency domain; The time series data after FFT processing is converted into a two-dimensional tensor, and the Inception block is used to extract features in two-dimensional space through parallel convolution kernels of different sizes. The extracted features are spliced ​​to form the final feature representation. Among them, C1, C2, C3 represent the outputs of different convolution kernels, and P represents the output of the maximum pooling. Represents feature concatenation operation; The final feature representation is converted from two dimensions to one dimension, and useful feature representations are extracted through an adaptive aggregation method to complete the classification of the double-drag behavior.

7. The method for identifying double-tow behavior of a fishing vessel according to claim 6, characterized in that: During the training process, the model is optimized by using the DEL imbalance loss function to minimize the difference between the predicted result and the actual label. Specifically, the DEL loss function is composed of the mitigation factor M ij , compensation factor C ij and feedback factor g i constitute; During the training process, DEL dynamically counts the number of small class samples N i and the number of samples in the large category N j , and construct the mitigation factor, for a trajectory sequence of length N, expressed as: M ij =(N i / N j ) p ,if N j >N i ; p is a hyperparameter for adjusting the degree of mitigation. j ≤N i When M ij =1, that is, the gradient ratio of small-class samples and large-class samples during training is not changed. When N j >N i When , large-class samples appear more frequently than small-class samples, M ij It can alleviate the gradient suppression of a large number of large-class samples on small-class samples; During the training process, p i is the probability that the sample is predicted to be a small class, p j is the probability that the sample is predicted to be a large category. DEL dynamically constructs the compensation factor, C ij =(p i / p j ) q ,if p i >p j ; Among them, q is a hyperparameter for adjusting the degree of compensation; for samples in category j, if its prediction appears p i >p j (i≠j), using the compensation factor C ij Increase the penalty for false positive samples; In each round of training, assume that the total number of small class samples is s, denoted by T s , the number of correctly classified small class samples is T i , the total number of misclassified small class samples is T s -T i , g i is a dynamic indicator g i =(T s -T i ) / T s .

8. A device for identifying double-tow behavior of a fishing vessel, characterized in that: The device comprises: a data acquisition and processing module, a network construction module, and a training module; The data acquisition and processing module is used to obtain the trajectory data of the fishing vessel, and perform preprocessing, similarity calculation, and data labeling; the trajectory data includes MMSI, longitude, latitude, speed, heading, time, vessel type, destination, and country; The network construction module is used to construct a dynamic balanced loss learning framework; it is also used to determine the common time interval of the first trajectory data and the second trajectory data of the fishing boat; resample the first trajectory data and the second trajectory data in the common time interval with Δt as the sampling period to obtain trajectory data composed of L sample points; determine each sample point a in the trajectory data i ′ and b i ′ and arrange the corresponding timestamps in chronological order to form time series data; and also used to perform FFT transformation, two-dimensional tensor conversion, feature extraction and splicing on the time series data through the constructed TimesBlock network to obtain the final feature representation; The training module is used to perform weighted summation on the extracted feature representations to obtain fused features; train the model through the fused features to determine whether the fishing vessel has double towing behavior; and optimize the model through the DEL imbalance loss function during the training process to minimize the difference between its predicted results and the actual labels.

Images