An unmanned ship target detection method based on incremental learning

Abstract

The application relates to an unmanned ship target detection method based on incremental learning. First, an initial category data set is constructed; a teacher network model is constructed, and the teacher network model is trained by using the initial category data set to detect the initial category sea surface target; a new category data set is constructed; a student network model is constructed, the overall structure of the student network is the same as that of the teacher network, and a corresponding classification subnetwork is added according to the number of new category targets; a distillation loss function is constructed between the teacher network and the student network; the incremental target detection network is trained on the new category sea surface target data set, the loss function is converged, and the trained model parameters are saved; the trained model is deployed on an unmanned ship platform to detect targets; when a new category detection target is added, steps three to seven are repeatedly executed. The application constructs a target detector that can continuously learn, and can enable the unmanned ship to quickly have the ability to identify new category sea surface targets.

Description

Technical Field

[0001] This invention belongs to the field of unmanned intelligent ships, specifically relating to an unmanned surface vessel target detection method based on incremental learning. Background Technology

[0002] Environmental perception is a prerequisite for intelligent mobile platforms such as unmanned surface vessels (USVs) to effectively complete various predetermined tasks. Target detection is a fundamental task of environmental perception, and improving the practicality of target detection algorithms is of great significance for enhancing the working capabilities of various unmanned devices. Similarly, conducting research on sea surface target detection is key to improving the mission execution capabilities of USVs. In recent years, thanks to the continuous development of deep learning theory and the emergence of large-scale target picking datasets, research on target detection has made groundbreaking progress.

[0003] When researching object detection algorithms, researchers typically assume that the training and test sets are independent and identically distributed, meaning all image data are collected from the same distribution and are independent of each other. This assumption usually holds true when the training and test sets come from the same dataset. However, the marine environment faced by unmanned surface vessels (USVs) in actual navigation is highly uncertain. When the target type changes, targets not present in the training set cannot be accurately identified.

[0004] When the target type changes, the model needs to be retrained. Training with only new data samples without incremental learning can lead to catastrophic forgetting of existing knowledge. Retraining a model from scratch every time new data is collected involves enormous computational overhead, lacks scalability, and is inefficient with data. Incremental learning, on the other hand, allows training with new data on top of the existing model, preventing the model from forgetting previous knowledge. This continuous learning capability is crucial for unmanned surface vessels (USVs) to cope with changing sea surface targets during actual navigation. Summary of the Invention

[0005] The purpose of this invention is to provide an incremental learning-based unmanned surface vessel (USV) target detection method that enables USVs to adapt to changes in the types of targets on the sea surface.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] An unmanned surface vessel (USV) target detection method based on incremental learning includes the following steps:

[0008] Step 1: Construct an initial category dataset. Take images of sea surface targets in a real marine environment using unmanned surface vessels, and label the initial sea surface targets with category and location information to construct a sea surface target detection dataset.

[0009] Step 2: Construct a teacher network model and train it using the initial category dataset to detect sea surface targets of the initial category;

[0010] Step 3: Construct a new category dataset: Take images of sea surface targets in a real marine environment using unmanned surface vessels. Label the new category of sea surface targets with category and location information to construct a sea surface target detection dataset.

[0011] Step 4: Construct the student network model. The overall structure of the student network is the same as that of the teacher network. Add corresponding classification subnetworks according to the number of new category targets.

[0012] Step 5: Construct a distillation loss function between the teacher network and the student network;

[0013] Step 6: Train the incremental target detection network on the new category of sea surface target dataset until the loss function converges, and save the trained model parameters;

[0014] Step 7: Deploy the trained model on the unmanned surface vessel platform to detect targets;

[0015] Step 8: When adding a new category of detection target, repeat steps 3 through 7.

[0016] Furthermore, in step two, the teacher network model uses an arbitrary object detection network as the teacher network; the initial category of sea surface target dataset is read to train and test the model; the loss function is:

[0017]

[0018] in, For bounding box loss, For classifying losses, For confidence loss;

[0019] Once the loss value converges, the model is tested. If the loss value does not converge, the model hyperparameters are adjusted until the model converges.

[0020] Furthermore, the new category in step three refers to the sea surface target category that did not appear in the initial category dataset, that is, the category that was not input into the target detection network model.

[0021] Furthermore, the student network constructed in step four has the same structure as the teacher network model and still uses the same target detection network as the student network; if the newly added sea surface target category is n, then n neurons need to be added to the output layer of the student network to expand the classification sub-network; the parameters in the student network are initialized using the parameters of the trained teacher network, and the newly added neurons in the classification sub-network are randomly initialized.

[0022] Furthermore, the distillation loss function in step five mainly consists of two parts; the first part constructs the loss function across the last layer of the entire network, as shown in the following formula:

[0023]

[0024] Where P represents the feature map of the last layer of the teacher network model. This represents the feature map of the last layer of the student network model.

[0025] The second part establishes a construction loss function between the last layer of the backbone feature extraction networks of the teacher network and the student network, as shown in the following formula:

[0026]

[0027] Where F represents the feature map of the last layer of the teacher network backbone feature extraction network. The feature map of the last layer of the network is extracted from the backbone features of the student network.

[0028] Furthermore, in step six, the training process requires adding the distillation loss to the loss during normal training of the object detection network to obtain a new loss function, as shown in the following formula:

[0029]

[0030] Among them, λ1 and λ2 are hyperparameters used to balance different losses;

[0031] During training, images are simultaneously input into both the teacher network and the student network. The parameters of the teacher network are frozen, and only the parameters in the student network are updated until the loss value converges to the minimum.

[0032] Furthermore, in step seven, deploying the trained model on the unmanned surface vessel platform only requires deploying the student network on the embedded platform carried by the unmanned surface vessel to complete the detection of new types of sea surface targets.

[0033] The beneficial effects of this invention are as follows:

[0034] This invention employs a knowledge distillation method to incrementally learn about sea surface targets, adapting to changes in these targets. This enables the target detection model to both recognize new targets and retain the memory of initial targets. It overcomes the catastrophic forgetting problem that traditional deep learning algorithms suffer after training on new samples, constructing a target detector capable of continuous learning. Attached Figure Description

[0035] Figure 1 This is a flowchart of an unmanned surface vessel target detection method based on incremental learning according to the present invention;

[0036] Figure 2 This is a schematic diagram of the incremental target detection network structure of the present invention;

[0037] Figure 3 This is a schematic diagram illustrating the deployment of the model of the present invention on an unmanned surface vessel platform. Detailed Implementation

[0038] The present invention will now be further described with reference to the accompanying drawings.

[0039] As shown in the attached figure Figure 1 The diagram shows a flowchart of an unmanned surface vessel (USV) target detection method based on incremental learning, which includes the following steps:

[0040] (1) Construct an initial category dataset by taking images of sea surface targets in a real marine environment using unmanned surface vessels, and labeling the initial sea surface targets with category and location information to construct a sea surface target detection dataset.

[0041] (2) Construct the teacher network model: An arbitrary object detection network is used as the teacher network. The initial dataset of sea surface objects is read, and the model is trained and tested. The loss function is as follows:

[0042]

[0043] in, For bounding box loss, For classifying losses, This represents the confidence loss.

[0044] Once the loss value converges, the model is tested. If the loss value does not converge, the model hyperparameters are adjusted until the model converges.

[0045] (3) Construct a new category dataset: take images of sea surface targets in a real marine environment by using unmanned surface vessels. Only the new category of sea surface targets are labeled with category and location information to construct a sea surface target detection dataset.

[0046] Among them, the new category of sea surface targets refers to the sea surface target categories that did not appear in the initial category dataset, that is, the categories that were not input into the target detection network model.

[0047] (4) Construct the student network model. The student network has the same structure as the teacher network model and uses the same target detection network as the student network. Assuming that the newly added sea surface target category is n, n neurons need to be added to the output layer of the student network to expand the classification subnetwork. The parameters in the student network are initialized using the parameters of the trained teacher network, and the newly added neurons in the classification subnetwork are randomly initialized.

[0048] (5) Construct a distillation loss function between the teacher network and the student network to ensure that the student network can fully learn the knowledge of the teacher network. The distillation loss function mainly consists of two parts: first, a loss function is established between the last layer of the entire network, as shown in the following formula:

[0049]

[0050] Where P represents the feature map of the last layer of the teacher network model. This represents the feature map of the last layer of the student network model.

[0051] Secondly, a loss function is established between the last layer of the backbone feature extraction networks of the teacher network and the student network, as shown in the following formula:

[0052]

[0053] Where F represents the feature map of the last layer of the teacher network backbone feature extraction network. The feature map of the last layer of the network is extracted from the backbone features of the student network.

[0054] (6) Train the incremental target detection network on the new category of sea surface target dataset. During the training process, the distillation loss and the loss during normal training of the target detection network need to be added together to obtain a new loss function, as shown in the following formula:

[0055]

[0056] Here, λ1 and λ2 are hyperparameters used to balance different losses.

[0057] During training, images are simultaneously input into both the teacher network and the student network. The parameters of the teacher network are frozen, and only the parameters in the student network are updated until the loss value converges to the minimum.

[0058] (7) Deploy the trained model on the unmanned surface vessel (USV) platform to detect targets. When deploying the trained model, it is only necessary to deploy the trained student network model on the embedded platform carried by the USV to complete the detection of new types of sea surface targets.

[0059] (8) When adding a new category of detection target, repeat steps three through seven.

[0060] like Figure 2The diagram shows the structure of an incremental object detection network. In the diagram, A represents the teacher network model trained on the initial category dataset, and B represents the student network model that needs to incrementally learn new object categories. A distillation loss function is established between the student and teacher networks to learn the knowledge from the teacher network. The distillation loss function mainly consists of two parts: first, a loss function L is established between the last layer of the entire network. dist_output Secondly, a loss function L is established between the last layer of the backbone feature extraction networks of the teacher network and the student network. dist_feature L cls +L box +L obj This represents the loss function of the student network when learning a new target category. During training, images are simultaneously input into both the teacher and student networks. The parameters of the teacher network are frozen, and only the parameters of the student network are updated until the loss value converges to its minimum.

[0061] like Figure 3 The diagram illustrates the deployment of the model on an unmanned surface vessel (USV) platform. When deploying the target detection model, simply porting the well-learned student network model to the embedded platform mounted on the USV is sufficient to detect targets on the sea surface. During the USV's navigation, images of the sea surface are obtained using its onboard camera. These images are then input into the target detection network to obtain detection results. Finally, the results are input into the USV's control system, providing environmental information for obstacle avoidance and navigation.

[0062] In summary, the incremental learning-based unmanned surface vessel (USV) target detection method of this invention employs knowledge distillation to solve the catastrophic forgetting problem when learning new target categories. When the types of sea surface targets change, incremental learning of sea surface targets can be performed, gaining the ability to recognize new targets while retaining the memory of the original targets. A continuously learning target detector is constructed, enabling USVs to quickly acquire the ability to recognize new categories of sea surface targets.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection method for unmanned surface vessels based on incremental learning, characterized in that: Includes the following steps: Step 1: Construct an initial category dataset. Take images of sea surface targets in a real marine environment using unmanned surface vessels, and label the initial sea surface targets with category and location information to construct a sea surface target detection dataset. Step 2: Construct a teacher network model and train it using the initial category dataset to detect sea surface targets of the initial category; Step 3: Construct a new category dataset: Take images of sea surface targets in a real marine environment using unmanned surface vessels. Label the new category of sea surface targets with category and location information to construct a sea surface target detection dataset. Step 4: Construct the student network model. The overall structure of the student network is the same as that of the teacher network. Add corresponding classification subnetworks according to the number of new category targets. Step 5: Construct a distillation loss function between the teacher network and the student network; The distillation loss function mainly consists of two parts; the first part constructs the loss function between the last layer of the entire network, as shown in the following formula: Where P represents the feature map of the last layer of the teacher network model. This represents the feature map of the last layer of the student network model. The second part establishes a construction loss function between the last layer of the backbone feature extraction networks of the teacher network and the student network, as shown in the following formula: Where F represents the feature map of the last layer of the teacher network backbone feature extraction network. The feature map of the last layer of the network is extracted from the backbone features of the student network. Step 6: Train the incremental target detection network on the new category of sea surface target dataset until the loss function converges, and save the trained model parameters; Step 7: Deploy the trained model on the unmanned surface vessel platform to detect targets; Step 8: When adding a new category of detection target, repeat steps 3 through 7.

2. The unmanned surface vessel target detection method based on incremental learning as described in claim 1, characterized in that: In step two, the teacher network model is as follows: an arbitrary object detection network is used as the teacher network; the initial category of sea surface target dataset is read, and the model is trained and tested; the loss function is: in, For bounding box loss, For classifying losses, For confidence loss; Once the loss value converges, the model is tested. If the loss value does not converge, the model hyperparameters are adjusted until the model converges.

3. The unmanned surface vessel target detection method based on incremental learning as described in claim 1, characterized in that: In step three, the new category refers to the sea surface target category that did not appear in the initial category dataset, that is, the category that was not input into the target detection network model.

4. The unmanned surface vessel target detection method based on incremental learning as described in claim 1, characterized in that: The student network constructed in step four has the same structure as the teacher network model and still uses the same target detection network as the student network. If the newly added sea surface target category is n, then n neurons need to be added to the output layer of the student network to expand the classification sub-network. The parameters in the student network are initialized using the parameters of the trained teacher network, and the newly added neurons in the classification sub-network are randomly initialized.

5. The unmanned surface vessel target detection method based on incremental learning as described in claim 2, characterized in that: In step six, the training process requires adding the distillation loss to the loss during normal training of the object detection network to obtain a new loss function, as shown in the following formula: Among them, λ1 and λ2 are hyperparameters used to balance different losses; During training, images are simultaneously input into both the teacher network and the student network. The parameters of the teacher network are frozen, and only the parameters in the student network are updated until the loss value converges to the minimum.

6. The unmanned surface vessel target detection method based on incremental learning as described in claim 1, characterized in that: The deployment of the trained model on the unmanned surface vessel platform in step seven only requires deploying the student network on the embedded platform carried by the unmanned surface vessel to complete the detection of new types of sea surface targets.

Images