A dual-branch semi-supervised infrared ship target segmentation method

Abstract

The application discloses a kind of double-branch semi-supervised infrared ship target segmentation methods, first, training set and test set are divided according to certain proportion, and the data in training set is divided into labeled image and unlabeled image according to different proportion;Second, a double-branch semi-supervised infrared ship target segmentation model is constructed;Then, model training: first, using improved segmentation cross-entropy loss as loss function, using back propagation and gradient descent algorithm Adam trains learning network A, B in double learning network branch, then, using the trained learning network A, B generates pseudo label for unlabeled image, then, using labeled data and unlabeled data trains segmentation network F in segmentation network branch, with segmentation cross-entropy loss as loss function, using back propagation and gradient descent algorithm Adam trains segmentation network F.The double-branch semi-supervised infrared ship target segmentation method of the application effectively improves the segmentation performance of infrared ship target.

Description

A dual-branch semi-supervised infrared ship target segmentation method Technical Field

[0001] This invention belongs to the field of infrared image target segmentation, and particularly relates to a dual-branch semi-supervised infrared ship target segmentation method. Background Technology

[0002] Infrared ship target segmentation refers to classifying and marking ship targets as pixels in an infrared ship image. Infrared ship target segmentation not only helps combat illegal vessels but also enables accurate maritime rescue, playing a vital role in maintaining maritime security, safeguarding maritime rights, and protecting people's lives and property.

[0003] Infrared ship target segmentation is a crucial and challenging task within the target segmentation problem. Currently, most infrared ship segmentation methods are based on fully supervised learning. This involves designing a deep network model and training it using all precisely labeled images to enable image segmentation capabilities, ultimately yielding segmentation results for test images. While fully supervised learning methods have achieved good segmentation results, they require precisely labeled pixel-level tags for all data during training. Obtaining these tags for infrared ship images is time-consuming and labor-intensive.

[0004] Semi-supervised learning methods do not require pixel-level labels for all images during training, aiming to improve segmentation accuracy by training the network using a small number of labeled images and a large number of unlabeled images. This invention first designs a method for generating pseudo-labels from unlabeled data; then, it trains a segmentation network using labeled data and unlabeled data with pseudo-labels to obtain infrared ship target segmentation results. This invention is highly operable and versatile, capable of completing various infrared image target segmentation tasks. Summary of the Invention

[0005] The main technical problem addressed by this invention is to provide a semi-supervised infrared ship target segmentation method based on a dual-branch architecture, which solves the problem of requiring too many labeled images in the training set. This invention first generates pseudo-labels for unlabeled images using a dual-learning network, expanding the unlabeled images into labeled images; then, it trains the segmentation network using all training set images, improving the segmentation performance of the network. To achieve the above objective, this invention adopts the following technical solution:

[0006] A dual-branch semi-supervised infrared ship target segmentation method includes the following steps:

[0007] Step 1: Divide the dataset D into a training set D in a 4:1 ratio. train (including N) train (samples) and test set Dtest (including N) test (samples), training set D train (including N) train The samples were divided into labeled dataset T by a ratio of M:1 (M∈{2,4,8}). label (including N) label (samples) and unlabeled dataset T unlabel (including N) unlabel (samples);

[0008] Step 2: Construct a dual-branch semi-supervised infrared ship target segmentation model. This model consists of two branches: a dual-learning network branch and a segmentation network branch. The dual-learning network consists of two identical networks A and B, and the segmentation network contains a segmentation network F. Learning networks A and B, and segmentation network F, all use UNet as their backbone network. For the i-th (1≤i≤N)... train ) input samples x i (1≤i≤N train The dual-branch semi-supervised infrared ship target segmentation model first uses a dual-learning network to obtain pseudo-labels for unlabeled data, and then uses labeled data and unlabeled data with pseudo-labels to train the segmentation network to obtain the segmentation results.

[0009] Step 3: Train a dual-branch semi-supervised infrared ship target segmentation network. Specific steps include:

[0010] Step 3.1: Select the number of iterations and the learning rate parameters. In each iteration, select the training set D. train The labeled dataset T in label Training networks A and B in a dual-learning network uses an improved segmentation cross-entropy loss as the loss function. This improved loss consists of two parts: segmentation cross-entropy loss and dual-learning loss. Networks A and B are trained using backpropagation and the Adam gradient descent algorithm. The trained networks are then saved. The improved segmentation cross-entropy loss function is:

[0011]

[0012] in, To split the cross-entropy loss, L mse This is a dual learning loss, where α represents the weight of the differences between the two learning networks;

[0013] Segmentation cross-entropy loss for:

[0014]

[0015] Where, xm (1≤m≤N label ) is a labeled dataset T label The image contains labels, y m (1≤m≤N label ) represents the corresponding real label, P sm (T k (x m ))(1≤m≤N label Let k∈{A, B}) be the predicted outputs of learning networks A and B, and f k ()(k∈{A,B}) represents the learning network, P sm () represents the softmax function;

[0016] Double learning loss L mse for:

[0017]

[0018] Where, x m (1≤m≤N label ) is a labeled dataset T label The labeled image in P sm (f A (x m )) P sm (f B (x m ))(1≤m≤N label f represents the predicted outputs of learning networks A and B. A (), f B () represents learning networks A, B, and P. sm () represents the softmax function;

[0019] Step 3.2: Train learning networks A and B according to the method in step 3.1 to enable the learning networks to have a certain segmentation ability. Use learning networks A and B to generate pseudo-labels for unlabeled images respectively.

[0020] The segmentation map normalized by softmax is called the segmentation confidence map. The segmentation confidence maps obtained from the two learning networks are added together to obtain the total segmentation confidence map. The calculation method is as follows:

[0021] I score,n =P sm (f A (x n ))+P sm (T B (x n (4)

[0022] Where, x n(1≤j≤N unlabel ) is an unlabeled dataset T unlabel Unlabeled images in I score,n It is a total segmentation confidence map generated from an unlabeled image;

[0023] The overall segmentation confidence map generated by the two learning networks combines the segmentation results of the two networks. For each pixel, the category with the highest score is selected as the category of that pixel. The pseudo-label is calculated as follows:

[0024] t pseudo，n =argmax(I score,n (5)

[0025] Among them, argmax() is used to get the category with the highest score for each pixel;

[0026] Step 3.3: Select the number of iterations and learning rate parameters, and use the training set D. train The segmentation network F of the segmentation network branch is trained using cross-entropy loss as the loss function. The network is trained using backpropagation and gradient descent algorithm Adam, and the trained network is saved.

[0027] The segmentation cross-entropy loss consists of two parts: the cross-entropy loss of the labeled image and the cross-entropy loss of the unlabeled image, calculated as follows:

[0028] L S =L adaw +L ces (6)

[0029] Among them, L adaw For cross-entropy loss of unlabeled images, L ces Cross-entropy loss for labeled data;

[0030] L adaw The calculation method is as follows:

[0031]

[0032] Where ω represents the weights for updating network parameters using unlabeled images, and x... n (1≤j≤N unlabel ) is an unlabeled dataset T unlabel Unlabeled images in t pseudo，n For unlabeled image x n (1≤j≤N unlabel The corresponding pseudo-tag, f F () represents the segmented network F;

[0033] The method for calculating ω is as follows:

[0034]

[0035] Among them, t pseudo,A ′ and t pseudo,B ' are the pseudo-labels generated by learning networks A and B, respectively;

[0036] L ces The calculation method is as follows:

[0037]

[0038] Where, x m (1≤m≤N label ) is a labeled dataset T label The image contains labels, y m (1≤m≤N label ) represents the corresponding real label, f F () denotes the segmented network F, P sm () represents the softmax function;

[0039] Step 4: In test set D test Test the trained segmentation network and calculate the segmentation accuracy. Attached Figure Description

[0040] Figure 1 is a flowchart of the basic method of the present invention.

[0041] Figure 2 shows a sample of infrared ship image data. Figure a is a labeled image; Figure b is an unlabeled image.

[0042] Figure 3 shows the model structure of the present invention. Detailed Implementation

[0043] This invention provides a semi-supervised infrared ship target segmentation method based on dual branches. The invention will be explained and described below with reference to the accompanying drawings:

[0044] The dataset used in this invention is an infrared ship dataset, which contains 11 categories, including 10 ship categories and 1 background category, with a total of 4671 samples. The SeISTS-DB network is used as the segmentation model.

[0045] The implementation process of this invention is as follows:

[0046] Step 1: Divide the dataset containing 4671 infrared ships into a training set D at a ratio of 4:1. train (Contains 3743 samples) and test set D test (Contains 928 samples), training set D train The dataset T is divided into labeled datasets in an 8:1 ratio. label(Contains 416 samples) and the unlabeled dataset T unlabel (Contains 3327 samples), and a portion of the training set samples are shown in Figure 2;

[0047] Step 2: Construct a dual-branch semi-supervised infrared ship target segmentation model, which consists of two parts: a dual learning network and a segmentation network. The dual learning network consists of two identical networks A and B, and the segmentation network contains a segmentation network F. Learning networks A and B, and segmentation network F, all use UNet as their backbone network. For the i-th (1≤i≤3743) input sample x... i (1≤i≤3743), the dual-branch semi-supervised infrared ship target segmentation model first uses a dual-learning network to obtain pseudo-labels for unlabeled data, and then uses labeled data and unlabeled data with pseudo-labels to train the segmentation network to obtain the segmentation results.

[0048] Step 3: Train a dual-branch semi-supervised infrared ship target segmentation network. Specific steps include:

[0049] Step 3.1: Select 100 iterations and a learning rate of 0.0002. In each iteration, select training set D. train The labeled dataset T in label Training networks A and B in a dual-learning network uses an improved segmentation cross-entropy loss as the loss function. This improved loss consists of two parts: segmentation cross-entropy loss and dual-learning loss. Networks A and B are trained using backpropagation and the Adam gradient descent algorithm. The trained networks are then saved. The improved segmentation cross-entropy loss function is:

[0050]

[0051] in, To split the cross-entropy loss, L mse For dual learning loss, α represents the weight of the difference between the two learning networks, and we choose α = 1;

[0052] Segmentation cross-entropy loss for:

[0053]

[0054] Where, x m (1≤m≤416) is the labeled dataset T. label The image contains labels, y m (1≤m≤416) represents the corresponding true label, P sm (f k (x m())(1≤m≤416, k∈{A,B}) represents the predicted outputs of learning networks A and B, f k ()(k∈{A,B}) represents the learning network, P sm () represents the softmax function;

[0055] Double learning loss L mse for:

[0056]

[0057] Where, x m (1≤m≤416) is the labeled dataset T. label The labeled image in P sm (f A (x m )) P sm (f B (x m ())(1≤m≤416) represents the predicted outputs of learning networks A and B, f A (), f B () represents learning networks A, B, and P. sm () represents the softmax function;

[0058] Step 3.2: Train learning networks A and B according to the method in step 3.1 to enable the learning networks to have a certain segmentation ability. Use learning networks A and B to generate pseudo-labels for unlabeled images respectively.

[0059] The segmentation map normalized by softmax is called the segmentation confidence map. The segmentation confidence maps obtained from the two learning networks are added together to obtain the total segmentation confidence map. The calculation method is as follows:

[0060] I score,n =P sm (f A (x n ))+P sm (f B (x n (4)

[0061] Where, x n (1≤j≤3327) is the unlabeled dataset T. unlabel Unlabeled images in I score,n It is a total segmentation confidence map generated from an unlabeled image;

[0062] The overall segmentation confidence map generated by the two learning networks combines the segmentation results of the two networks. For each pixel, the category with the highest score is selected as the category of that pixel. The pseudo-label is calculated as follows:

[0063] t pseudo，n =argmax(I score,n (5)

[0064] Among them, argmax() is used to get the category with the highest score for each pixel;

[0065] Step 3.3: Select 30 iterations and a learning rate of 0.0002, and use the training set D. train The segmentation network F of the segmentation network branch is trained using cross-entropy loss as the loss function. The network is trained using backpropagation and gradient descent algorithm Adam, and the trained network is saved.

[0066] The segmentation cross-entropy loss consists of two parts: the cross-entropy loss of the labeled image and the cross-entropy loss of the unlabeled image, calculated as follows:

[0067] L S =L adaw +L ces (6)

[0068] Among them, L adaw For cross-entropy loss of unlabeled images, L ces Cross-entropy loss for labeled data;

[0069] L adaw The calculation method is as follows:

[0070]

[0071] Where ω represents the weights for updating network parameters using unlabeled images, and x... n (1≤j≤3327) is the unlabeled dataset T. unlabel Unlabeled images in t pseudo，n For unlabeled image x n The pseudo-label corresponding to (1≤j≤3327), f F () represents the segmented network F;

[0072] The method for calculating ω is as follows:

[0073]

[0074] Among them, t pseudo，A ′ and t pseudo，B ' are the pseudo-labels generated by learning networks A and B, respectively;

[0075] L ces The calculation method is as follows:

[0076]

[0077] Where, x m (1≤m≤416) is the labeled dataset T. label The image contains labels, y m (1≤m≤416) represents the corresponding true label, f F () denotes the segmented network F, P sm () represents the softmax function;

[0078] Step 4: In test set D test The test results are shown in Table 1.

[0079] Table 1. Segmentation accuracy of infrared ship images

[0080]

[0081] The examples above are for illustrative purposes only and are not intended to limit the technical solutions described herein. Therefore, all technical solutions and improvements thereof that do not depart from the spirit and scope of this invention should be covered within the scope of the claims of this invention.

Claims

1. A semi-supervised infrared ship target segmentation method based on dual-branch architecture, characterized in that, Includes the following steps: Step 1: Transfer the dataset The training set was divided into a 4:1 ratio. and test set training set The dataset is divided into labeled datasets with a ratio of M:

1. and unlabeled datasets The training set Include One sample, test set Include A dataset with samples and labels. Include One sample; Step 2: Construct a dual-branch semi-supervised infrared ship target segmentation model. This model consists of two branches: a dual-learning network branch and a segmentation network branch. The dual-learning network consists of two identical networks A and B, and the segmentation network contains a segmentation network F. Learning networks A and B, as well as the segmentation network F, all use UNet as their backbone network. For the first... Input Samples ， The dual-branch semi-supervised infrared ship target segmentation model first uses a dual-learning network to obtain pseudo-labels for unlabeled data, and then uses labeled data and unlabeled data with pseudo-labels to train the segmentation network to obtain the segmentation results. Step 3: Train the dual-branch semi-supervised infrared ship target segmentation network. Specific steps include: Step 3.1: Select the number of iterations and learning rate parameters; select the training set for each iteration. Labeled datasets in Training networks A and B in the dual-learning network branch uses an improved segmentation cross-entropy loss as the loss function. This improved loss consists of two parts: segmentation cross-entropy loss and dual-learning loss. Networks A and B are trained using backpropagation and the Adam gradient descent algorithm. The trained networks are then saved. The improved segmentation cross-entropy loss function is: (1); where, To split the cross-entropy loss, For double learning loss, Weights representing the differences between two learning networks; segmentation cross-entropy loss. for: (2); where, For labeled datasets The image contains labels. For the corresponding real tags, To learn the predicted outputs of networks A and B, Indicates learning network, This represents the softmax function; ； Double learning loss for: (3); among them, For labeled datasets The image contains labels. To learn the predicted outputs of networks A and B, This indicates learning networks A and B. The softmax function is used; Step 3.2: Train learning networks A and B according to the method in Step 3.1 to enable them to have certain segmentation capabilities. Use learning networks A and B to generate pseudo-labels for unlabeled images respectively; The segmentation map normalized by softmax is called the segmentation confidence map. The segmentation confidence maps obtained from the two learning networks are added together to obtain the total segmentation confidence map, which is calculated as follows: (4); among them, For unlabeled datasets Unlabeled images in It is a total segmentation confidence map generated from an unlabeled image; The overall segmentation confidence map generated by the two learning networks combines the segmentation results of the two networks. For each pixel, the category with the highest score is selected as the category of that pixel. The pseudo-label is calculated as follows: (5); where argmax() is used to obtain the category with the highest score for each pixel; Step 3.3: Select the number of iterations and learning rate parameters, and use the training set. The segmentation network F, a branch of the segmentation network, is trained using cross-entropy loss as the loss function. The network is trained using backpropagation and the Adam gradient descent algorithm, and the trained network is saved. The segmentation cross-entropy loss consists of two parts: the cross-entropy loss for labeled images and the cross-entropy loss for unlabeled images, calculated as follows: (6); among them, For cross-entropy loss of unlabeled images, Cross-entropy loss for labeled data; The calculation method is as follows: (7); among which, Update the weights of the network parameters for unlabeled images. For unlabeled datasets Unlabeled images in Unlabeled image The corresponding pseudo-tags, Describe the segmentation network F; The calculation method is as follows: (8); among them, and These are the pseudo-labels generated by learning networks A and B, respectively. The calculation method is as follows: (9); among them, For labeled datasets The labeled images in the image are the corresponding real labels. Let F represent the segmentation network. Represents the softmax function; Step 4: In the test set Test the trained segmentation network and calculate the segmentation accuracy.

Images