Self-balancing difficulty diversity sampling active learning method, device, equipment and medium

Abstract

The embodiment of the invention discloses a self-balancing difficulty diversity sampling active learning method, device and equipment and a medium. The method comprises the following steps: collecting ship navigation data, and preprocessing to form an unlabeled sample data pool U; uniformly selecting samples from the U for expert marking to form an initial marking training set L, and setting an active learning round R and the number N of query samples in each round; training a deep learning model through the L; using the trained deep learning model to predict unlabeled samples in the U, and outputting a category probability p (x); based on the p (x), determining edge uncertainty M (x) and diversity contribution degree D (x) of the sample, and fusing to obtain difficulty diversity R (x); selecting high-value samples from the U according to the R (x), adding the high-value samples into a query set Q, sending the samples in the Q to an expert for labeling, and updating L; and repeating the operation until a termination condition is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and marine supervision technology, and particularly relates to a self-balancing difficult example diversity sampling active learning method, device, equipment and medium. BACKGROUND

[0002] In recent years, with the profound changes in the global marine strategic pattern, the strategic position of the ocean in national security, economic development and international cooperation has become increasingly prominent. The challenges faced by sea area ship management are also becoming increasingly complex. New governance needs have emerged in the field of maritime traffic safety and ship behavior supervision. The identification of abnormal behaviors of fishing vessels has become a key problem that needs to be solved.

[0003] In addition, illegal entry into a fishing restricted area, overfishing, abnormal trajectory, and illegal operation of fishing vessels also threaten marine ecological safety and pose potential risks to maritime traffic. Among the many abnormal behaviors of fishing vessels, the abnormal operation of double-towed fishing vessels has become a key and difficult point in the governance of marine ecological safety. Figure 1 As shown in the figure, double-towed fishing vessels usually use two fishing vessels to jointly tow a net with a length of hundreds of meters. Due to the large depth of the net into the water, the seabed ecosystem is easily damaged. Moreover, due to the need to maintain a certain distance and speed between the fishing vessels during double-towing operation, this operating mode increases the risk of collision at sea and poses a serious threat to maritime safety. Therefore, it is of great significance to build an efficient method for identifying abnormal behaviors of fishing vessels for the protection of marine environmental safety, the maintenance of maritime traffic order, and the promotion of the modernization of the marine governance system.

[0004] In the research of ship abnormal behavior identification, there are mainly two research directions: rule-based abnormal behavior identification method and deep learning-based abnormal behavior identification method. Although the rule-based abnormal behavior identification method has strong interpretability, with the increasing complexity of marine supervision needs, the rule-based method also has obvious limitations: it is difficult to adapt to the rapidly changing supervision needs, and it is difficult for predefined rules to handle new types of abnormal patterns. At the same time, the complexity of massive AIS and radar data has exceeded the processing scope of rule-based reasoning. In contrast, the deep learning-based abnormal behavior identification method has shown great potential due to its powerful feature learning and pattern recognition capabilities.

[0005] However, when applying deep learning models to actual open sea scenarios, the models will face a large amount of real-time data that is not covered by the training set. For such new data, the model cannot correctly identify it. Therefore, in the open sea scenario, the deep learning model needs to have the ability to adapt to dynamic scenarios, and through the iterative update of training data and identification models to adapt to environmental changes. However, this requirement forms a sharp contradiction with the high cost and low timeliness of expert labeling, further exacerbating the challenge of obtaining high-quality labeled samples

[0006] Although a large amount of unlabeled data is easily available in open sea scenarios, the dynamic adaptation capability of the model requires its parameters and training data to be continuously iterated with environmental evolution, which forms a sharp contradiction with the high cost and low timeliness dilemma of expert labeling. Not only does the labeling process rely on the experience of domain experts, but also the non-linear coupling between data size and scenario dynamics leads to an exponential increase in the cost of manpower and time. Active learning methods can reduce labeling work by iteratively selecting and labeling high-value samples that contribute most to improving model performance.

[0007] Active learning has shown important application potential in computer vision, natural language processing and other fields. Its core idea is to maximize model performance while minimizing labeling cost through intelligent sample selection mechanisms, thereby effectively solving the problem of deep learning model dependence on large-scale labeled data. An active learning sample is shown in Figure 2 , where Figure 2 (a) is a distribution diagram of two different samples, and the blue straight line is the true boundary line of the class. Randomly select 30 samples as the training set to train the classification model, Figure 2 (b) shows the classification results. As shown in the figure, the obtained boundary line deviates significantly from the true boundary line, and there are many classification errors. Select 30 samples using the active learning strategy, and the classification results of the trained model are shown in Figure 2 (c). The blue line is the boundary line of the trained model, which is basically consistent with the true boundary line, and the classification accuracy reaches 90%.

[0008] From the perspective of implementation paradigm, active learning strategies can be divided into three main types: data pool-based active learning, data stream-based active learning, and synthetic sample query-based active learning. A basic active learning process is shown in Figure 3 . Among them, data pool-based active learning has become the most representative technical route in practice due to its unique advantages. This paradigm establishes a unified pool of unlabeled sample candidates, dynamically selects the most informative samples for expert labeling during iterative training. This mechanism can flexibly integrate uncertainty sampling, committee query and other diversified selection strategies, thereby achieving optimal sample selection from a global perspective. Through the dynamic updating mechanism of the sample pool, efficient use of computing resources is achieved, which is particularly suitable for the task of fishing boat abnormal behavior recognition in massive data scenarios.

[0009] In the task of fishing vessel abnormal behavior recognition in open scenarios, how to select samples with the largest amount of information while ensuring the maximum diversity of samples to minimize information overlap is a major challenge. Selecting only samples with high uncertainty can lead to very similar redundant information, while simply maximizing diversity can lead to labeling samples with no information content. Samples based on uncertainty methods have high information content, allowing the model to effectively refine the decision boundary and improve generalization performance in training. Diversity-based methods focus on minimizing redundancy in the query batch. However, how to best balance uncertainty and diversity is a major challenge faced by the present invention. SUMMARY

[0010] Based on this, it is necessary to propose a self-balancing difficult example diversity sampling active learning method, device, equipment and medium for the above problems.

[0011] A self-balancing difficult example diversity sampling active learning method includes the following steps:

[0012] Collecting ship navigation data, including AIS data and radar data, and preprocessing to form an unlabeled sample data pool U;

[0013] Uniformly selecting samples from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and setting the number of active learning rounds R and the number of samples per round N;

[0014] Training a deep learning model using the initial training set L, wherein the deep learning model is a TimesNet network for processing ship trajectory time series data;

[0015] Using the trained deep learning model to predict unlabeled samples in the unlabeled sample data pool U, outputting class probability p(x), wherein the classes include fishing vessel abnormal behavior and other ship behavior;

[0016] Based on the class probability p(x), determine the marginal uncertainty M(x) and diversity contribution D(x) of the sample, and fuse to obtain the difficult example diversity R(x);

[0017] According to the difficult example diversity R(x), select high-value samples from the unlabeled sample data pool U to join the query set Q, and update the training set L after sending the samples in the query set Q to expert labeling; repeat the operation until the termination condition is met, the termination condition includes reaching the round R, the labeling budget is exhausted or the model performance threshold, to realize the dynamic optimization of the fishing vessel abnormal behavior recognition model.

[0018] Preferably, the ship AIS and radar data preprocessing includes denoising, interpolation, trajectory segmentation and downsampling operations, and takes longitude, latitude, speed, heading and timestamp as features; the fishing boat abnormal behavior includes at least one of double trawl operation, gillnet, purse seine or single trawl behavior, and the abnormal behavior identification is based on ship trajectory features.

[0019] Preferably, the using the trained deep learning model to predict the unlabeled samples in the unlabeled sample data pool U outputs the category probability p(x), specifically including: for a given input , the output of the classification model is processed by function to obtain the probability , and the norm is used to measure the probability distribution difference and between the two samples determined; wherein, , are any two different unlabeled samples in the unlabeled sample data pool U, C is the total number of categories, that is, the number of all categories in the classification task, respectively represent the probability of the sample , belonging to the category.

[0020] Preferably, the determining the edge uncertainty M(x) of the sample based on the category probability p(x) specifically includes: determining the edge uncertainty M(x) according to , wherein, , identify the most likely category and the second likely category of the sample.

[0021] Preferably, the determining the diversity contribution degree D(x) of the sample based on the category probability p(x) specifically includes: determining the diversity contribution degree D(x) of the sample according to , wherein, represents the initial diversity contribution degree, indicating the probability distribution distance between the unlabeled sample and the first difficult example sample selected in the current active learning round, represents the minimum value of the initial diversity contribution degree of all unlabeled samples, represents the maximum value of the initial diversity contribution degree of the unlabeled sample.

[0022] Preferably, the fusion obtains the difficult example diversity R(x), specifically including: according to fusion obtains the difficult example diversity R(x), wherein, represents a self-balancing factor.

[0023] An active learning device based on self-balancing difficult example diversity sampling, the device comprising:

[0024] A data collection module for collecting ship navigation data, including AIS data and radar data, and preprocessing to form an unlabeled sample data pool U;

[0025] A preprocessing module for preprocessing the ship navigation data to form an unlabeled sample data pool U;

[0026] An initialization module for uniformly selecting samples from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and setting the number of active learning rounds R and the number of query samples per round N;

[0027] A model training module for training a deep learning model using the initial training set L, wherein the deep learning model is a TimesNet network for processing ship trajectory time series data;

[0028] A prediction module for using the trained deep learning model to predict unlabeled samples in the unlabeled sample data pool U and outputting class probabilities p(x), wherein the classes include fishing boat abnormal behavior and other ship behavior;

[0029] A calculation module for determining the edge uncertainty M(x) and diversity contribution D(x) of the samples based on the class probabilities p(x), and fusing to obtain difficult example diversity R(x);

[0030] A sample selection module for selecting high-value samples from the unlabeled sample data pool U based on the difficult example diversity R(x) to join the query set Q;

[0031] An update control module for updating the training set L after sending samples in the query set Q to expert labeling; repeating the operation until the termination condition is met, the termination condition including reaching the number of rounds R, the labeled budget being exhausted, or the model performance threshold, to achieve dynamic optimization of the fishing boat abnormal behavior recognition model.

[0032] A computer device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to cause the processor to perform the following steps:

[0033] Collecting ship navigation data, including AIS data and radar data, and preprocessing to form an unlabeled sample data pool U;

[0034] Uniformly selecting samples from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and setting the number of active learning rounds R and the number of query samples per round N;

[0035] training a deep learning model through the initial training set L, wherein the deep learning model is a TimesNet network for processing ship trajectory time series data;

[0036] using the trained deep learning model to predict unlabelled samples in the unlabelled sample data pool U, and outputting a category probability p(x), wherein the categories include fishing boat abnormal behaviors and other ship behaviors;

[0037] based on the category probability p(x), determining an edge uncertainty M(x) and a diversity contribution D(x) of the sample, and fusing to obtain a difficult example diversity R(x);

[0038] selecting high-value samples from the unlabelled sample data pool U according to the difficult example diversity R(x) to join the query set Q, and updating the training set L after sending the samples in the query set Q to an expert for labelling; repeating the operation until a termination condition is met, the termination condition including reaching a round R, a labelling budget being exhausted, or a model performance threshold, so as to realize dynamic optimization of a fishing boat abnormal behavior recognition model.

[0039] A computer readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0040] collecting ship navigation data including AIS data and radar data, and pre-processing to form an unlabelled sample data pool U;

[0041] uniformly selecting samples from the unlabelled sample data pool U for expert labelling to form an initial labelled training set L, and setting an active learning round R and a number of query samples per round N;

[0042] training a deep learning model through the initial training set L, wherein the deep learning model is a TimesNet network for processing ship trajectory time series data;

[0043] using the trained deep learning model to predict unlabelled samples in the unlabelled sample data pool U, and outputting a category probability p(x), wherein the categories include fishing boat abnormal behaviors and other ship behaviors;

[0044] based on the category probability p(x), determining an edge uncertainty M(x) and a diversity contribution D(x) of the sample, and fusing to obtain a difficult example diversity R(x);

[0045] According to the difficult example diversity R(x), high-value samples are selected from the unlabeled sample data pool U to join the query set Q, and the samples in the query set Q are sent to an expert for labeling, and the training set L is updated; the operation is repeated until a termination condition is met, the termination condition includes reaching a round R, a label budget being exhausted, or a model performance threshold, so as to realize dynamic optimization of the fishing boat abnormal behavior recognition model.

[0046] By adopting the embodiment of the present application, the following beneficial effects are achieved:

[0047] The present application directly utilizes the model output probability space to calculate the edge uncertainty and diversity contribution of the sample, and fuses it into a difficult example diversity index, thereby achieving a substantial reduction in labeling cost (for example, on the HN_PT_1 data set, the recall rate is increased by 9.76% and the F1 score is increased by 12.99% compared with random sampling, and about 30% of the labeling workload can be reduced), and the calculation efficiency is high, avoiding complex operation in a high-dimensional feature space; the self-adaptive balancing mechanism can dynamically adjust the uncertainty and diversity weights according to the data distribution, adapt to different scene requirements, and has excellent generalization ability, and excellent performance is achieved on multiple data sets (such as HN_PT_1 and AIS1036); in addition, the method is practical and easy to deploy in an actual system, and provides an efficient and reliable solution for high real-time requirement applications such as marine supervision. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief descriptions will be given below for the drawings needed to be used in the embodiments or prior art descriptions. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Among them:

[0050] Figure 1 It is a schematic diagram of the abnormal operation behavior of the double-towed fishing boat;

[0051] Figure 2 It is a schematic diagram of the active learning example in the prior art;

[0052] Figure 3 It is a flowchart of the active learning process in the prior art;

[0053] Figure 4 It is a flowchart of a self-balancing difficult example diversity sampling active learning method according to an embodiment of the present application;

[0054] Figure 5 It is a process schematic diagram of constructing the HN_PT_1 data set in the self-balancing difficult example diversity sampling active learning method according to an embodiment of the present application;

[0055] Figure 6 This is a comparison chart of recognition results in the qualitative analysis of a self-balancing difficult example diversity sampling active learning method provided in an embodiment of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] This invention provides a self-balancing hard example diversity sampling active learning method, such as... Figures 1-6 As shown, it includes the following steps:

[0058] Step 101: Collect ship navigation data, including AIS data and radar data, and preprocess it to form an unlabeled sample data pool U;

[0059] Specifically, the ship AIS and radar data preprocessing includes noise reduction, interpolation, trajectory segmentation and downsampling operations, and uses longitude, latitude, speed, heading and timestamp as features;

[0060] The preprocessing includes operations such as noise reduction, interpolation, and trajectory segmentation.

[0061] Step 102: Select samples evenly from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and set the number of active learning rounds R and the number of query samples N in each round;

[0062] Step 103: Train a deep learning model using the initial training set L, wherein the deep learning model is a TimesNet network used to process ship trajectory time series data;

[0063] Step 104: Use the trained deep learning model to predict the unlabeled samples in the unlabeled sample data pool U, and output the class probability p(x), where the class includes abnormal behavior of fishing boats and other ship behaviors;

[0064] Specifically, for a given input The output of the classification model is processed The probability is obtained after function processing. ,use Norm to measure two samples and Differences in probability distributions between Determined; among them, , For any two different unlabeled samples in the unlabeled sample data pool U, the difference in probability distribution between the samples is used to calculate the diversity. C is the total number of categories, that is, the number of all categories in the classification task (e.g., C=2 in the identification of abnormal behavior of double tugs, including "abnormal behavior of double tugs" and "other ship behaviors"; C=3 on the AIS1036 dataset, including "gillnet", "purse seine", and "single tug"). Representing samples respectively , Belongs to the The probability of a class.

[0065] Unlike existing methods, SBHEDS selects samples directly based on the model's output probabilities. Traditional methods typically utilize the latent space for diversity exploration and assess uncertainty in the probability space, while SBHEDS achieves the selection of highly diverse and highly uncertain samples using only probabilistic representations.

[0066] Step 105: Based on the class probability p(x), determine the marginal uncertainty M(x) and diversity contribution D(x) of the sample, and fuse them to obtain the difficult example diversity R(x);

[0067] Specifically, this invention uses marginal uncertainty as an uncertainty measure, that is, the sample belongs to the most likely category. With secondary possible categories The probability difference between them, according to Determine the marginal uncertainty M(x), where, , The sample is identified as belonging to the most likely category and the next most likely category.

[0068] if , A higher margin uncertainty indicates greater marginal uncertainty and a more difficult sample. Marginal uncertainty is a commonly used uncertainty measure that effectively captures the decision boundaries between classes. Compared to other uncertainty measures such as entropy or minimum confidence, marginal uncertainty creates multiple local optima in the probability space across all classes, distributed along the class decision boundaries. Therefore, samples with maximum marginal uncertainty are diverse. Entropy sampling has a global optimum, meaning the entropy value is maximized when the probability is uniformly distributed. Samples selected based on entropy often result in a large number of redundant samples. Marginal uncertainty emphasizes the importance of diverse margins, thus capturing more diverse samples.

[0069] Traditional diversity induction methods typically tend to select samples that are far removed from the current batch. However, effectively incorporating diversity in the initial stage when the query batch is empty remains a challenge. Some methods measure diversity by calculating the distance to labeled samples, but this approach has two limitations: first, the computational cost increases with the labeled set; second, it fails to reflect the inherent difficulty of the samples themselves, and cannot guarantee both sample dispersion and high quality. A feasible strategy is to calculate the probability distribution distance between unlabeled samples and the first hard example selected in the current active round, thus initializing diversity. This approach reflects the inherent difficulty of the samples while incurring less computational cost.

[0070] Marginal uncertainty It perfectly meets this requirement by taking into account the probability difference between the best and second-best predictions. This directly indicates the difficulty level of the sample itself in the model, thus selecting the first difficult sample. At this point, through calculation The contribution of diversity to the remaining samples was completed. Initialization. Each time a sample is selected... Then, dynamically updated. The contribution of diversity to the remaining samples is .

[0071] If a sample is If the distance is small, the contribution is updated to a smaller value to avoid selecting similar samples. If a sample is close to... If the distance is large, the contribution remains unchanged because there are values ​​among the selected samples that are smaller in distance from this sample.

[0072] Finally, the diversity contribution is normalized to the range [0, 1] to align it with marginal uncertainty. Keep the range consistent, as in the formula As shown ,in, Indicates the initial diversity contribution, referring to unlabeled samples. The probability distribution distance between the sample and the first hard example selected in the current active learning round is used to initialize the diversity measure. . This represents the initial diversity contribution of all unlabeled samples. The minimum value in the range is used for normalization calculations. This indicates the initial diversity contribution from unlabeled samples. The maximum value in.

[0073] This invention proposes a hard-case diversity index that considers both the marginal uncertainty and diversity contribution of the samples. This metric allows for the selection of high-quality samples from a large number of candidate samples. Dynamically updated in each round of active learning, driven by marginal uncertainty. and normalized diversity contribution The two parts were calculated to obtain ,in, This represents the self-balancing factor, used to assess the diversity of difficult cases. Nonlinear control is performed. This is achieved by amplifying high uncertainty. or high diversity The contribution of the samples makes the sampling process adaptively favor these high-value samples, while simultaneously balancing the weights of both to adapt to different data distributions.

[0074] parameter This approach achieves non-linear control over the diversity of difficult examples, amplifying the contribution of high-scoring items and making it more inclined to select samples with high uncertainty or high diversity. In the initial sampling, sample uncertainty dominates.

[0075] Step 106: Based on the diversity of difficult examples R(x), select high-value samples from the unlabeled sample data pool U and add them to the query set Q. Then send the samples in the query set Q to experts for annotation and update the training set L.

[0076] Specifically, as samples are sampled into the query set In the unlabeled sample set, the number of samples with high uncertainty gradually decreases, and sample diversity gradually comes into play. If the sample distribution is uneven, the contribution of diversity to the probability space is generally large, and the contribution of diversity in the formula is significant. The proportion is relatively large; conversely, when the samples are more evenly distributed in the probability space, the algorithm relies more on uncertainty. This mechanism ensures that uncertainty selection and diversity selection can adaptively maintain a balance according to the specific circumstances of the current query batch, adapting to various data distributions. Based on the calculated diversity of each sample, it is converted into the probability of each sample being selected according to the formula, and a sample is selected based on this probability. Join middle.

[0077]

[0078] When query set The number of samples in the sample is equal to When that happens, it will stop and return. . The samples will be sent to experts for annotation. (Repeated) Each round completes the active learning process.

[0079] Step 107: Repeat steps 104 to 106 until the termination condition is met. The termination condition includes reaching round R, exhausting the tag budget, or a model performance threshold, in order to achieve dynamic optimization of the fishing vessel abnormal behavior recognition model.

[0080] Specifically, the abnormal behavior of the fishing vessel includes at least one of double trawling, gillnetting, purse seine, or single trawling, and the identification of the abnormal behavior is based on the characteristics of the vessel's trajectory.

[0081] Compared with the prior art, the present invention has the following advantages and technical effects:

[0082] 1. Significantly reduce annotation costs: On the HN_PT_1 dataset, compared with the random sampling method, the recall rate is improved by 9.76% and the F1 score is improved by 12.99%, which can reduce the annotation workload by about 30% while achieving the same performance.

[0083] High computational efficiency: Sample selection is performed directly based on the model output probability, avoiding complex calculations in the high-dimensional feature space and significantly reducing computational complexity.

[0084] Strong adaptive balancing capability: It can automatically adjust the weights of uncertainty and diversity according to the characteristics of data distribution, adapting to different application scenarios.

[0085] Excellent generalization ability: Experiments on multiple datasets (HN_PT_1, AIS1036) show that the method can achieve excellent performance on data of different sizes and types.

[0086] Highly practical: The method is efficient, easy to deploy and apply in actual systems, and suitable for scenarios with high real-time requirements, such as marine surveillance.

[0087] Example

[0088] 1. Dataset and Evaluation Metrics

[0089] This invention conducts experiments on the HN_PT_1 dataset to identify dual-drag anomaly behavior based on self-balancing hard example diversity sampling.

[0090] Combining the behavioral characteristics of trawler fishing vessels with expert experience, and after data cleaning, longitude, latitude, speed, heading, and timestamp were selected as features of abnormal trawler behavior. To simulate real-world scenarios, this invention collected navigation data of trawler fishing vessels during both closed and open fishing seasons, constructing two imbalanced trawler abnormal behavior datasets HN_PT_1. This process is as follows: Figure 5 As shown.

[0091] The experimental area for this invention is the coastal waters of Hainan Province, including the Qiongzhou Strait, the South China Sea, and the Beibu Gulf. Through the collaborative platform of this invention, AIS and radar navigation data of ships were collected from October 2022 to June 2024. The specific fields of the collected data are listed in detail.

[0092]

[0093] Raw AIS and radar data often contain issues such as noise, redundancy, and outliers. To address this, we first process the raw data for missing and outlier values, including missing position coordinates and abnormal speeds. Then, we remove redundant information and noise from the raw heading data to improve data quality.

[0094] In addition, considering that high-density raw trajectory data would increase storage and processing costs, in practice, downsampling at fixed time intervals is used to reduce redundant data and improve processing efficiency. Taking a ship with mmsi=412483782 as an example, the time intervals of the raw trajectory are inconsistent, and resampling is performed at 60-second intervals, as shown in Table 2.

[0095]

[0096] Combining the behavioral characteristics of tugboats with expert experience, and after data cleaning, longitude, latitude, speed, heading, and timestamp were selected as features of anomalous tugboat behavior. An imbalanced dataset of anomalous tugboat behavior, HN_PT_1, was successfully constructed. This process is as follows: Figure 3 As shown.

[0097] As shown in Table 3, the HN_PT_1 dataset contains 364 trawler fishing vessels and 25,480 non-trawler fishing vessels, with approximately 20,675,200 usable trajectory point samples, divided into a training set of 18,607,700 and a test set of 2,067,500. The ratio of samples showing abnormal behavior of trawler vessels to other vessel behaviors is 1:70.

[0098]

[0099] Furthermore, to comprehensively verify the effectiveness and generalization ability of SBHEDS, comparative experiments were conducted on the larger-scale AIS1036 public dataset to identify other potential abnormal fishing vessel behaviors (gillnetting, purse seine, and single trawling). The AIS1036 dataset originates from anonymized AIS data of real vessels in the East China Sea, containing 18,329 vessel behavior records and approximately 75,633,800 trajectory point samples. The detailed division of its training and test sets is shown in Table 4, with 62,024,700 and 13,619,100 samples in the training and test sets, respectively. It covers three typical fishing vessel behavior types: gillnetting, purse seine, and single trawling. This invention aims to more objectively evaluate the performance of SBHEDS in real-world application scenarios through experimental verification on large-scale, multi-type datasets.

[0100]

[0101] 2. Experimental Implementation Details To identify abnormal behaviors of fishing vessels such as double trawling, gillnetting, purse seine, and single trawling, this invention conducts experiments using SBHEDS based on the TimesNet deep learning model. The experiments were conducted in a PyTorch environment using the Adam optimizer with an initial learning rate of 8e-7. An NVIDIA RTX 3090 GPU was used for accelerated computation during training. Specific parameter settings are shown in Table 5. Experimental parameter settings on the HN_PT_1 dataset: Initial labeled training set... L The sample size is 200, and the active learning rounds are... R The number of queries is 7, and the number of samples per round is... N 200 records. Experimental parameter settings on the AIS1036 dataset: Initial labeled training set L The sample size is 200, and the active learning rounds are... R The number of queries is 16, and the number of samples per round is... N There are 528 entries.

[0102]

[0103] 3. Comparison with existing methods For the dual-drag anomaly behavior recognition task, Table 6 shows the recall rate comparison between SBHEDS and existing methods on the HN_PT_1 dataset. The comparison methods include Random sampling, Entropy sampling, Least confidence, Adversarial deepfool, Adversarial BIM, and Falcun.

[0104]

[0105] In the first round, SBHEDS achieved a recall of 19.51%, a 4.88% improvement over Random sampling, a 6.10% improvement over Entropy sampling, and even a 1.22% improvement over the better-performing Falcun. This demonstrates that SBHEDS can quickly establish effective recognition capabilities with minimal labeled data. As the training rounds increased, SBHEDS maintained its leading advantage. Particularly in the fourth round, the recall reached 59.76%, significantly higher than Random sampling's 42.07% and Adversarial deepfool's 46.34%. In the seventh round, SBHEDS achieved a recall of 79.27%, 1.22% higher than the second-best method Falcun, 4.88% higher than Entropy sampling, and 9.76% higher than Random sampling. This sustained leading advantage indicates that SBHEDS demonstrates robust performance improvement.

[0106] The accuracy results also demonstrate a significant advantage for SBHEDS, as shown in Table 7. Analysis of the accuracy data across the seven rounds of active learning shows that the accuracy steadily improved from 80.49% in round 1 to 92.68% in round 7, consistently maintaining a leading position. In round 1, SBHEDS achieved an accuracy of 80.49%, significantly higher than all other methods compared. In round 7, SBHEDS achieved an accuracy of 92.68%, a 0.24% improvement over the second-best performing method, Falcun. This further validates the effectiveness of SBHEDS in identifying abnormal behavior of fishing vessels.

[0107]

[0108] SBHEDS performed exceptionally well in terms of F1 score, steadily improving from 28.57% in round 1 to 81.25% in round 7, as shown in Table 8. In round 1, the F1 score was 28.57%; in round 4, it reached 67.59%, significantly higher than Randomsampling's 49.64% and Adversarial deepfool's 55.88%. In round 5, the F1 score reached 74.17%, surpassing the performance of most methods in round 7. In round 7, the F1 score reached 81.25%, an improvement of 0.75% compared to methods including the suboptimal Falcun. The experimental results in F1 score further confirm the effectiveness of the active learning method based on self-balancing difficult example diversity sampling. This method achieves high F1 scores in each active learning round, indicating that in practical applications it can accurately identify abnormal behavior while maximizing the detection of existing abnormal behavior. This balanced performance is particularly important for practical applications such as fishing vessel supervision, reducing false alarms while mitigating the risk of missed alarms and improving regulatory efficiency.

[0109]

[0110] 4. Generalization experiments on public datasets To fully validate the effectiveness and generalization ability of SBHEDS, this section further conducts comparative experiments on the AIS1036 public dataset to identify other potential abnormal behaviors of fishing vessels (gillnets, purse seines, and single trawlers). As shown in Table 9, by observing the recall rate of 16 rounds of active learning, it can be seen that the model trained by SBHEDS has a significant advantage over other methods. In the early training stage, the recall rate of SBHEDS is 61.99%, slightly lower than other methods, which may be due to the randomness of model initialization. From the 6th round onwards, SBHEDS begins to outperform all the comparative methods and maintains this lead until the 16th round. In the 14th round, SBHEDS reaches a recall rate of 82.01%, which is 1.29% higher than the second-ranked Entropysampling. SBHEDS continues to grow steadily from the 12th to the 16th round, increasing from 81.26% to 82.01%, indicating that it can effectively utilize newly labeled samples to improve the model even in the later stages.

[0111] Experimental results show that the self-balancing hard example diversity sampling active learning method achieves significant results by balancing information maximization and sample diversity. Especially when annotation resources are limited, this method can better select the most valuable samples for annotation, thereby obtaining better model performance with minimal annotation cost.

[0112]

[0113] As shown in Table 10, SBHEDS demonstrated excellent accuracy from the first round, reaching 82.15% accuracy by the 16th round, comprehensively outperforming all comparable methods. This indicates that SBHEDS is scalable and effective on large-scale fishing vessel anomalous behavior data. Starting from the 6th round, SBHEDS' accuracy steadily improved from 76.27% to 80.06%; the advantage further widened in rounds 11-16; by the 16th round, the proposed method outperformed the suboptimal entropy sampling method by 1.39%. Furthermore, SBHEDS exhibited a stable performance improvement trend from round 1 to round 16, without significant performance fluctuations or declines, demonstrating the method's reliability. Even when other methods slowed their performance improvement in rounds 10-16, SBHEDS maintained good improvement potential, increasing from 80.06% to 82.15%.

[0114]

[0115] As shown in Table 11, SBHEDS also demonstrated a significant advantage in F1 score evaluation. Its score rapidly improved from an initial 61.88% to 82.04% in round 16, indicating that SBHEDS excels in balancing accuracy and recall, providing more comprehensive performance for identifying abnormal fishing vessel behavior. From round 6 onwards, SBHEDS consistently led all comparable methods in F1 score. In rounds 10-16, this advantage further widened, ultimately reaching an F1 score of 82.04% in round 16, 1.37% higher than the second-best method, Entropy sampling. Simultaneously, SBHEDS exhibited a smooth performance improvement curve from round 4 to round 16, without significant fluctuations. In rounds 12-16, while the performance improvement of other methods slowed, SBHEDS maintained good upward potential, increasing from 81.27% to 82.04%.

[0116]

[0117] 5. Hyperparameter Experiment SBHEDS has an adjustable hyperparameter—the self-balancing factor. α This invention designed multiple sets of different hyperparameters for experiments to find the optimal hyperparameters that can better improve the performance of the method. Table 12 shows some of the experimental results. α The model's performance in active learning rounds 1, 4, and 7 is shown when the hyperparameter values ​​are 9, 11, 13, 15, and 17. When the hyperparameter values ​​are small, such as... α =9, the model's early performance was poor, with low scores across all metrics. When α=13, and the model's performance reached its optimum in the 7th round of active learning. This suggests that larger hyperparameter values ​​may help the model better identify rare categories in the early learning stages. When hyperparameter values ​​are large, such as... α When the value is 17, the model's recognition performance drops significantly. The hyperparameter performance exhibits an inverted U-shaped trend; values ​​that are too small or too large will lead to performance degradation, indicating the existence of an optimal value.

[0118]

[0119] 9. Qualitative Analysis To further verify the recognition performance of the model trained by the SBHEDS method, this section qualitatively presents six sets of recognition result images, such as... Figure 5 As shown. The navigation data of six pairs of ships were input into the recognition model. Figure 6 (b) through (h) illustrate the recognition performance of models trained using seven different query strategies. "GT" and "Pred" represent the true and predicted labels, respectively, while "0" and "1" represent other vessel behaviors and dual-tug abnormal behaviors, respectively. Figure 6 In (h), the method proposed in this invention accurately identified three pairs of dual-tow abnormal behaviors and three other ship behaviors, while six query strategies—Random sampling, Entropy sampling, Leastconfidence, Adversarial deepfool, Adversarial BIM, and Falcun—all exhibited varying degrees of misidentification. The qualitative experimental results fully demonstrate the effectiveness of the active learning method based on self-balancing difficult example diversity sampling.

[0120] This invention also provides an active learning device based on self-balancing hard example diversity sampling, the device comprising: The data acquisition module is used to collect ship navigation data, including AIS data and radar data, and preprocess it to form an unlabeled sample data pool U; The preprocessing module is used to preprocess the ship navigation data to form an unlabeled sample data pool U; An initialization module is used to uniformly select samples from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and to set the number of active learning rounds R and the number of query samples N in each round; The model training module is used to train a deep learning model using the initial training set L, wherein the deep learning model is a TimesNet network used to process ship trajectory time series data. The prediction module is used to predict the unlabeled samples in the unlabeled sample data pool U using the trained deep learning model, and output the class probability p(x), where the class includes abnormal behavior of fishing vessels and other vessel behaviors. The calculation module is used to determine the marginal uncertainty M(x) and diversity contribution D(x) of the sample based on the category probability p(x), and fuse them to obtain the hard case diversity R(x); The sample selection module is used to select high-value samples from the unlabeled sample data pool U and add them to the query set Q based on the diversity of difficult cases R(x); The update control module is used to send samples in the query set Q to experts for annotation and then update the training set L; the operation is repeated until the termination condition is met, the termination condition includes reaching round R, exhausting the label budget or the model performance threshold, so as to realize the dynamic optimization of the fishing vessel abnormal behavior recognition model.

[0121] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps: Collect ship navigation data, including AIS data and radar data, and preprocess it to form an unlabeled sample data pool U; Samples are uniformly selected from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and the number of active learning rounds R and the number of query samples N in each round are set. A deep learning model is trained using the initial training set L, wherein the deep learning model is a TimesNet network used to process ship trajectory time series data; The trained deep learning model is used to predict the unlabeled samples in the unlabeled sample data pool U, and the output class probability p(x) is given, where the classes include abnormal behavior of fishing vessels and other vessel behaviors. Based on the class probability p(x), the marginal uncertainty M(x) and diversity contribution D(x) of the sample are determined and fused to obtain the hard case diversity R(x); Based on the diversity of difficult examples R(x), high-value samples are selected from the unlabeled sample data pool U and added to the query set Q. The samples in the query set Q are sent to experts for labeling and the training set L is updated. The operation is repeated until the termination condition is met. The termination condition includes reaching the number of rounds R, exhausting the labeling budget, or a model performance threshold, so as to achieve dynamic optimization of the fishing vessel abnormal behavior recognition model.

[0122] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, causes the processor to perform the following steps: Collect ship navigation data, including AIS data and radar data, and preprocess it to form an unlabeled sample data pool U; Samples are uniformly selected from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and the number of active learning rounds R and the number of query samples N in each round are set. A deep learning model is trained using the initial training set L, wherein the deep learning model is a TimesNet network used to process ship trajectory time series data; The trained deep learning model is used to predict the unlabeled samples in the unlabeled sample data pool U, and the output class probability p(x) is given, where the classes include abnormal behavior of fishing vessels and other vessel behaviors. Based on the class probability p(x), the marginal uncertainty M(x) and diversity contribution D(x) of the sample are determined and fused to obtain the hard case diversity R(x); Based on the diversity of difficult examples R(x), high-value samples are selected from the unlabeled sample data pool U and added to the query set Q. The samples in the query set Q are sent to experts for labeling and the training set L is updated. The operation is repeated until the termination condition is met. The termination condition includes reaching the number of rounds R, exhausting the labeling budget, or a model performance threshold, so as to achieve dynamic optimization of the fishing vessel abnormal behavior recognition model.

[0123] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A self-balancing, hard example diversity sampling active learning method, characterized in that, Includes the following steps: Collect ship navigation data, including AIS data and radar data, and preprocess it to form an unlabeled sample data pool U; Samples are uniformly selected from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and the number of active learning rounds R and the number of query samples N in each round are set. A deep learning model is trained using the initial training set L, wherein the deep learning model is a TimesNet network used to process ship trajectory time series data; The trained deep learning model is used to predict the unlabeled samples in the unlabeled sample data pool U, and the output class probability p(x) is given, where the classes include abnormal behavior of fishing vessels and other vessel behaviors. Based on the class probability p(x), the marginal uncertainty M(x) and diversity contribution D(x) of the sample are determined and fused to obtain the hard case diversity R(x); Based on the diversity of difficult examples R(x), high-value samples are selected from the unlabeled sample data pool U and added to the query set Q. The samples in the query set Q are sent to experts for labeling and the training set L is updated. The operation is repeated until the termination condition is met. The termination condition includes reaching the number of rounds R, exhausting the labeling budget, or a model performance threshold, so as to achieve dynamic optimization of the fishing vessel abnormal behavior recognition model.

2. The self-balancing hard example diversity sampling active learning method according to claim 1, characterized in that, The ship's AIS and radar data preprocessing includes noise reduction, interpolation, trajectory segmentation, and downsampling operations, and uses longitude, latitude, speed, heading, and timestamp as features; The abnormal behavior of the fishing vessel includes at least one of the following: double trawling, gillnetting, purse seine, or single trawling, and the identification of the abnormal behavior is based on the characteristics of the vessel's trajectory.

3. The self-balancing hard example diversity sampling active learning method according to claim 1 or 2, characterized in that, The step of using a trained deep learning model to predict unlabeled samples in the unlabeled sample data pool U and outputting the class probability p(x) specifically includes: for a given input The output of the classification model is processed The probability is obtained after function processing. ,use Norm to measure two samples and Differences in probability distributions between Determined; among them, , Let C be any two distinct unlabeled samples in the unlabeled sample data pool U, and let C be the total number of categories, i.e., the number of all categories in the classification task. Representing samples respectively , Belongs to the The probability of a class.

4. The self-balancing hard example diversity sampling active learning method according to claim 3, characterized in that, The determination of the marginal uncertainty M(x) of a sample based on the class probability p(x) specifically includes: according to Determine the marginal uncertainty M(x), where, , The sample is identified as belonging to the most likely category and the next most likely category.

5. The self-balancing hard example diversity sampling active learning method according to claim 4, characterized in that, The determination of the diversity contribution D(x) of the sample based on the class probability p(x) specifically includes: according to Determine the diversity contribution of the sample, D(x), where, Indicates the initial diversity contribution, referring to unlabeled samples. The probability distribution distance between the sample and the first hard example selected in the current active learning round. This represents the initial diversity contribution of all unlabeled samples. The minimum value in, This indicates the initial diversity contribution from unlabeled samples. The maximum value in.

6. The self-balancing hard example diversity sampling active learning method according to claim 5, characterized in that, The fusion yields the diversity of difficult examples R(x), specifically including: according to The fusion yields the diversity of difficult examples R(x), where, This represents the self-balancing factor.

7. An active learning device based on self-balancing difficult example diversity sampling, characterized in that, The device includes: The data acquisition module is used to collect ship navigation data, including AIS data and radar data, and preprocess it to form an unlabeled sample data pool U; The preprocessing module is used to preprocess the ship navigation data to form an unlabeled sample data pool U; An initialization module is used to uniformly select samples from the unlabeled sample data pool U for expert labeling to form an initial labeled training set L, and to set the number of active learning rounds R and the number of query samples N in each round; The model training module is used to train a deep learning model using the initial training set L, wherein the deep learning model is a TimesNet network used to process ship trajectory time series data. The prediction module is used to predict the unlabeled samples in the unlabeled sample data pool U using the trained deep learning model, and output the class probability p(x), where the class includes abnormal behavior of fishing vessels and other vessel behaviors. The calculation module is used to determine the marginal uncertainty M(x) and diversity contribution D(x) of the sample based on the category probability p(x), and fuse them to obtain the hard case diversity R(x); The sample selection module is used to select high-value samples from the unlabeled sample data pool U and add them to the query set Q based on the diversity of difficult cases R(x); The update control module is used to send samples in the query set Q to experts for annotation and then update the training set L; the operation is repeated until the termination condition is met, the termination condition includes reaching round R, exhausting the label budget or the model performance threshold, so as to realize the dynamic optimization of the fishing vessel abnormal behavior recognition model.

8. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7 and is used for integration into a ship monitoring platform.

Images