An improved infrared ship image semantic segmentation method and system of a domain adaptive Deeplab model
Patent Information
- Application Number
- CN202410905872.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2026-08-18
- Estimated Expiration
- 2044-07-08
AI Technical Summary
[0003]本发明的目的在于提供一种基于域自适应Deeplab模型的改进方法，克服红外舰船语义分割中平均交并比低的问题
[0049] This invention improves the average intersection-over-union ratio of infrared ship image datasets by modifying the model architecture and training strategy of Deeplab, thereby increasing the accuracy of the trained deep convolutional neural network model and enhancing the network model's ability to classify and locate ships in infrared images.
Smart Images
Figure CN118781344B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual semantic segmentation, specifically to an improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images and a related system. Background Technology
[0002] Semantic segmentation of ships in infrared scenes is a crucial foundation for efficient maritime monitoring and national defense security. Due to the low signal-to-noise ratio and blurred semantic boundaries of infrared ship images, accurate pixel-level annotation is difficult to obtain. Conversely, visible light ship images are easier to annotate. However, visible light and infrared images exhibit domain shift, therefore segmentation models trained on visible light images cannot be directly applied to semantic segmentation tasks in infrared images. Domain-adaptive semantic segmentation methods for infrared ship images are a wise choice. Infrared ship image semantic segmentation refers to pixel-level annotation of ship targets in infrared images, typically achieved using advanced artificial intelligence visual semantic segmentation techniques. Summary of the Invention
[0003] The purpose of this invention is to provide an improved method based on the domain adaptive Deeplab model to overcome the problem of low average intersection-union ratio in infrared ship semantic segmentation.
[0004] To address the aforementioned technical problems, this invention provides an improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images, comprising the following steps:
[0005] Obtain the visible light ship image dataset X s and the corresponding label Y S ;
[0006] Obtain the infrared ship image dataset X t And the infrared ship image dataset X t Divided into training set X train and test set X test ;
[0007] Construct an improved domain-adaptive Deeplab model;
[0008] Using the visible light ship image dataset X s Tag Y S and training set X train Semantic segmentation was performed on the improved domain-adaptive Deeplab model to obtain the image semantic segmentation results.
[0009] Preferably, the improved domain-adaptive Deeplab model includes a backbone network, a master classifier, an auxiliary classifier, a master discriminant network, and an auxiliary discriminant network.
[0010] Preferably, the visible light ship image dataset X is used. s Tag Y S and training set X train The improved domain-adaptive Deeplab model is used for semantic segmentation to obtain image semantic segmentation results. The specific steps include:
[0011] X s and training set X train The samples are input into the backbone network, and the five convolutional layers of the backbone network output the first feature maps O for visible light and infrared light, respectively. s1 Second feature map O s2 Third feature map O s3 Fourth feature map O s4 and the fifth feature map O s5 ;
[0012] The fourth feature map O of visible light and infrared light s4 and the fifth feature map O s5 The first semantic prediction maps O for visible light and infrared light are obtained by inputting the main classifier and the auxiliary classifier respectively. cp Second semantic prediction graph O ca ;
[0013] The first semantic prediction map of visible light and infrared light O cp Second semantic prediction graph O ca The main discriminant network and the auxiliary discriminant network are input respectively to obtain the first domain discrimination result O. dp Second domain discrimination result O da ;
[0014] Based on the first semantic prediction map of visible light O cp Second semantic prediction graph O ca And label Y S The final segmentation loss is calculated.
[0015] Based on the first semantic prediction map O of infrared cp Second semantic prediction graph O ca The consistency regularization term is calculated.
[0016] Based on the first domain discrimination result O dp The second domain discrimination result O da The final adversarial loss is calculated by combining the final segmentation loss with the final adversarial loss.
[0017] Preferably, based on the first semantic prediction map O of visible light cp Second semantic prediction graph O ca And label Y S The final segmentation loss is calculated, which includes the following steps:
[0018] The loss function for the main classifier, auxiliary classifier, main discriminant network, and auxiliary discriminant network is the cross-entropy loss function;
[0019] The cross-entropy loss function is:
[0020]
[0021] in, and These represent the segmentation losses of the main classifier and the auxiliary classifier, respectively. Let $\mathbf{i}$ represent the true label corresponding to the $i$-th sample in the source domain; $N$ represents the number of training samples; $C$ represents the total number of categories; $h$, $w$, and $c$ represent the length, width, and category of the sample, respectively; $O$ represents the true label of the i-th sample in the source domain; $N$ represents the number of training samples; $C$ represents the total number of categories; $h$, $w$, and $c$ represent the cp and O ca These represent the first semantic prediction graph and the second semantic prediction graph, respectively.
[0022] Segmentation loss generated by the main classifier and the auxiliary classifier and The final partition loss is obtained after weighting:
[0023]
[0024] Where, α gp With (1-α) gp ) represent the coefficients of the main classifier and the auxiliary classifier, respectively; L seg This represents the final partition loss.
[0025] Preferably, the coefficients of the main classifier and the auxiliary classifier are 0.7 and 0.3, respectively.
[0026] Preferably, based on the first semantic prediction map O of infrared light... cp Second semantic prediction graph O ca The consistency regularization term is calculated, which includes the following steps:
[0027] The consistency regularization term used is the KL divergence, and the formula for the consistency regularization term is:
[0028]
[0029] Where C represents the total number of categories in the training samples; h, w, and c represent the length, width, and category of the sample, respectively; O cp and O ca These represent the first semantic prediction graph and the second semantic prediction graph, respectively.
[0030] Preferably, based on the first domain discrimination result O dp The second domain discrimination result O da The final adversarial loss is calculated by combining the final segmentation loss with the final adversarial loss, specifically including the following steps:
[0031] The loss functions of the main discriminant network and the auxiliary discriminant network are adversarial loss functions, respectively.
[0032] The adversarial loss function is:
[0033]
[0034] Among them, O dp Indicates the result of the first domain discrimination; O da This represents the discrimination result in the second domain; N represents the total number of training samples; h, w, and c represent the length, width, and class of the sample, respectively; and These represent the adversarial losses generated by the primary and secondary discriminant networks, respectively.
[0035] and The final adversarial loss, obtained after weighted averaging, is as follows:
[0036]
[0037] in, and Let α represent the adversarial losses of the primary discriminant network and the secondary discriminant network, respectively. dp With (1-α) dp ) represent the coefficients of the primary discriminant network and the secondary discriminant network, respectively; L adv This indicates the final losses in the confrontation.
[0038] Preferably, the method further includes the following steps:
[0039] The backbone network and master and slave classifiers were trained using stochastic gradient descent.
[0040] The Adam optimizer was used as the optimization method to train the master and slave discriminant networks.
[0041] Preferably, the method further includes the following steps:
[0042] Using test set X test The trained, improved domain-adaptive Deeplab model was tested to obtain the average crossover ratio (CRO).
[0043] This invention also provides an improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images, comprising:
[0044] The acquisition module is used to acquire the visible light ship image dataset X. s and the corresponding label Y S ;
[0045] The partitioning module is used to acquire the infrared ship image dataset X.t And the infrared ship image dataset X t Divided into training set X train and test set X test ;
[0046] Build modules for constructing improved domain-adaptive Deeplab models;
[0047] The semantic segmentation module is used to utilize the visible light ship image dataset X. s Tag Y S and training set X train Semantic segmentation was performed on the improved domain-adaptive Deeplab model to obtain the image semantic segmentation results.
[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0049] This invention improves the average intersection-over-union ratio of infrared ship image datasets by modifying the model architecture and training strategy of Deeplab, thereby increasing the accuracy of the trained deep convolutional neural network model and enhancing the network model's ability to classify and locate ships in infrared images. Attached Figure Description
[0050] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0051] Figure 1 This is a schematic diagram of the basic method flow of the present invention;
[0052] Figure 2 This is the overall network architecture design for the present invention;
[0053] Figure 3 This is a structural diagram of the backbone network and the main and auxiliary classifiers;
[0054] Figure 4 To determine the structure of the network. Detailed Implementation
[0055] Numerous specific details are set forth in the following description to provide a full understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.
[0056] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.
[0057] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."
[0058] The present invention will now be described in further detail with reference to the accompanying drawings:
[0059] This invention provides an improved semantic segmentation method for infrared ship images using a domain-adaptive Deeplab model, comprising the following steps:
[0060] Obtain the visible light ship image dataset X s and the corresponding label Y S ;
[0061] Obtain the infrared ship image dataset X t And the infrared ship image dataset X t Divided into training set X train and test set X test ;
[0062] Construct an improved domain-adaptive Deeplab model;
[0063] Using the visible light ship image dataset X s Tag Y S and training set X train Semantic segmentation was performed on the improved domain-adaptive Deeplab model to obtain the image semantic segmentation results.
[0064] Preferably, the improved domain-adaptive Deeplab model includes a backbone network, a master classifier, an auxiliary classifier, a master discriminant network, and an auxiliary discriminant network.
[0065] Preferably, the visible light ship image dataset X is used. s Tag YS and training set X train The improved domain-adaptive Deeplab model is used for semantic segmentation to obtain image semantic segmentation results. The specific steps include:
[0066] X s and training set X train The samples are input into the backbone network, and the five convolutional layers of the backbone network output the first feature maps O for visible light and infrared light, respectively. s1 Second feature map O s2 Third feature map O s3 Fourth feature map O s4 and the fifth feature map O s5 ;
[0067] The fourth feature map O of visible light and infrared light s4 and the fifth feature map O s5 The first semantic prediction maps O for visible light and infrared light are obtained by inputting the main classifier and the auxiliary classifier respectively. cp Second semantic prediction graph O ca ;
[0068] The first semantic prediction map of visible light and infrared light O cp Second semantic prediction graph O ca The main discriminant network and the auxiliary discriminant network are input respectively to obtain the first domain discrimination result O. dp Second domain discrimination result O da ;
[0069] Based on the first semantic prediction map of visible light O cp Second semantic prediction graph O ca And label Y S The final segmentation loss is calculated.
[0070] Based on the first semantic prediction map O of infrared cp Second semantic prediction graph O ca The consistency regularization term is calculated.
[0071] Based on the first domain discrimination result O dp The second domain discrimination result O da The final adversarial loss is calculated by combining the final segmentation loss with the final adversarial loss.
[0072] Preferably, based on the first semantic prediction map O of visible light cp Second semantic prediction graph O ca And label Y S The final segmentation loss is calculated, which includes the following steps:
[0073] The loss function for the main classifier, auxiliary classifier, main discriminant network, and auxiliary discriminant network is the cross-entropy loss function;
[0074] The cross-entropy loss function is:
[0075]
[0076] in, and These represent the segmentation losses of the main classifier and the auxiliary classifier, respectively. Let $\mathbf{i}$ represent the true label corresponding to the $i$-th sample in the source domain; $N$ represents the number of training samples; $C$ represents the total number of categories; $h$, $w$, and $c$ represent the length, width, and category of the sample, respectively; $O$ represents the true label of the i-th sample in the source domain; $N$ represents the number of training samples; $C$ represents the total number of categories; $h$, $w$, and $c$ represent the cp and O ca These represent the first semantic prediction graph and the second semantic prediction graph, respectively.
[0077] Segmentation loss generated by the main classifier and the auxiliary classifier and The final partition loss is obtained after weighting:
[0078]
[0079] Where, α gp With (1-α) gp ) represent the coefficients of the main classifier and the auxiliary classifier, respectively; L seg This represents the final partition loss.
[0080] Preferably, the coefficients of the main classifier and the auxiliary classifier are 0.7 and 0.3, respectively.
[0081] Preferably, based on the first semantic prediction map O of infrared light... cp Second semantic prediction graph O ca The consistency regularization term is calculated, which includes the following steps:
[0082] The consistency regularization term used is the KL divergence, and the formula for the consistency regularization term is:
[0083]
[0084] Where C represents the total number of categories in the training samples; h, w, and c represent the length, width, and category of the sample, respectively; O cp and O ca These represent the first semantic prediction graph and the second semantic prediction graph, respectively.
[0085] Preferably, based on the first domain discrimination result O dp The second domain discrimination result O da The final adversarial loss is calculated by combining the final segmentation loss with the final adversarial loss, specifically including the following steps:
[0086] The loss functions of the main discriminant network and the auxiliary discriminant network are adversarial loss functions, respectively.
[0087] The adversarial loss function is:
[0088]
[0089] Among them, O dp Indicates the result of the first domain discrimination; O da This represents the discrimination result in the second domain; N represents the total number of training samples; h, w, and c represent the length, width, and class of the sample, respectively; and These represent the adversarial losses generated by the primary and secondary discriminant networks, respectively.
[0090] and The final adversarial loss, obtained after weighted averaging, is as follows:
[0091]
[0092] in, and Let α represent the adversarial losses of the primary discriminant network and the secondary discriminant network, respectively. dp With (1-α) dp ) represent the coefficients of the primary discriminant network and the secondary discriminant network, respectively; L adv This indicates the final losses in the confrontation.
[0093] Preferably, the method further includes the following steps:
[0094] The backbone network and master and slave classifiers were trained using stochastic gradient descent.
[0095] The Adam optimizer was used as the optimization method to train the master and slave discriminant networks.
[0096] Preferably, the method further includes the following steps:
[0097] Using test set X test The trained, improved domain-adaptive Deeplab model was tested to obtain the average crossover ratio (CRO).
[0098] This invention also provides an improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images, comprising:
[0099] The acquisition module is used to acquire the visible light ship image dataset X. s and the corresponding label Y S ;
[0100] The partitioning module is used to acquire the infrared ship image dataset X. tAnd the infrared ship image dataset X t Divided into training set X train and test set X test ;
[0101] Build modules for constructing improved domain-adaptive Deeplab models;
[0102] The semantic segmentation module is used to utilize the visible light ship image dataset X. s Tag Y S and training set X train Semantic segmentation was performed on the improved domain-adaptive Deeplab model to obtain the image semantic segmentation results.
[0103] This invention utilizes a semantic segmentation model based on an improved domain-adaptive Deeplab model to perform semantic segmentation on infrared ship images. The Deeplab model can be viewed as a combination of a deep convolutional neural network and a dilated spatial pyramid pooling module. Its main idea is to extract basic feature information from the image using a deep convolutional neural network in the backbone network, and then capture features at different scales in the image using a dilated spatial pyramid pooling module. The dilated spatial pyramid pooling module is a special type of dilated convolutional module that uses dilated convolutions with different dilation rates, enabling it to better handle multi-scale information in the image.
[0104] To better illustrate the technical effects of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:
[0105] Example 1: An improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images:
[0106] The infrared ship image dataset used in this invention contains four categories: three ship categories and one background category. Visible light images, serving as the source domain, comprise 2961 images and their true pixel-level labels, all used for training. Infrared images, serving as the target domain, comprise 614 images, with 429 images in the training set and 185 images in the test set. All images are 256×256 pixels in size and have 3 channels.
[0107] like Figure 1 As shown, the image semantic segmentation method specifically includes the following steps:
[0108] Step 1: Obtain the visible light ship image dataset X s (2961 samples) and their corresponding labels Y S (2961 samples); Obtain the infrared ship image dataset X t (614 samples), X t The training set X is divided into two parts in a 7:3 ratio.train (429 samples) and test set X test (185 samples);
[0109] Step 2: As Figure 2 As shown, an improved domain-adaptive Deeplab model is constructed, which consists of 5 branches: the backbone network, the main classifier, the auxiliary classifier, the main discriminant network, and the auxiliary discriminant network.
[0110] Step 2.1: Construct the backbone network; the backbone network consists of the first 5 convolutional layers of ResNet-101, and is constructed in a hierarchical manner into 5 feature extraction stages, denoted as S1, S2, S3, S4 and S5 respectively. The output feature map of each stage is denoted as the first feature map O. s1 Second feature map O s2 Third feature map O s3 Fourth feature map O s4 and the fifth feature map O s5 Each stage is a residual block, which includes a 1x1 convolutional layer, a batch normalization layer, a ReLU activation function layer, a 3x3 convolutional layer, a batch normalization layer, a ReLU activation function layer, and a skip connection layer. The 1x1 convolutional layer has a 1x1 kernel size and a stride of 1, used for dimensionality reduction of the input image channels. The 3x3 convolutional layer has a 3x3 kernel size, a stride of 2, and padding of 1, used for feature extraction and downsampling. The batch normalization layer is used to accelerate training and improve model stability. The ReLU activation function layer is used to introduce non-linearity, enabling the network to learn more complex functions. The skip connection layer adds the input of the residual block directly to the output, which allows gradients to flow directly to shallower layers, thus alleviating the gradient vanishing problem.
[0111] Step 2.2: As Figure 3 As shown, a main classifier and an auxiliary classifier are constructed. The classifiers analyze the features extracted by the backbone network and generate the final semantic prediction map. The main and auxiliary classifiers are connected to the fifth and fourth convolutional layers of the backbone network, respectively, and their input data are O. s5 and O s4Each classifier has the same structure, consisting of a dilated spatial pyramid pooling layer and an upsampling layer. The dilated spatial pyramid pooling layer is used to capture multi-scale image context information, with pooling sampling rates of 6, 12, 18, and 24. The upsampling layer is used to restore the pooled feature maps to the original image size, generating the final semantic prediction map. Specifically, for an input image of size 256×256×3, features are first extracted using five convolutional layers of the backbone network. Then, the extracted features are input into two classifiers. The feature vectors in each classifier first pass through a dilated pyramid pooling module to obtain four feature maps of size 4×4×1024. Finally, the four feature maps are added along the channel dimension and upsampled to obtain a prediction map of size 256×256×4. The semantic prediction maps output by the main classifier and the auxiliary classifier are defined as the first semantic prediction map O. cp Second semantic prediction graph O ca ;
[0112] Step 2.3: As Figure 4 As shown, a main discriminant network and an auxiliary discriminant network are constructed. The main and auxiliary discriminant networks provide domain discrimination information to the model by distinguishing the semantic prediction graphs of the source and target domains. Their input data are O cp and O ca Both the main and auxiliary discriminant networks are five-layer convolutional networks; the kernel size of each of the five convolutional layers is 4×4, the stride is 2, and the number of channels is 64, 128, 256, 512, and 1, respectively; the final upsampling operation is used to restore the output feature map to the input size and return the discrimination result; the discrimination result output by the discriminant network represents the domain discrimination information; when the result is close to 1, it indicates that the input prediction map is judged as the source domain; when the result is close to 0, it indicates that the input prediction map is judged as the target domain; the domain discrimination results output by the main discriminant network and the auxiliary discriminant network are defined as the first domain discrimination result O. dp Second domain discrimination result O da ;
[0113] Step 3: The loss function includes the loss functions of the main and auxiliary classifiers and the loss function of the discriminator network, specifically:
[0114] Step 3.1: The primary and secondary classifiers classify the data based on the primary and secondary segmentation maps O. cp and O ca With label Y S The segmentation loss and consistency regularization term are calculated; the loss function used is the cross-entropy loss function, and the consistency regularization term used is the KL divergence.
[0115] The cross-entropy loss function is:
[0116]
[0117] in, and These represent the segmentation losses of the main classifier and the auxiliary classifier, respectively. Let $\mathbf{i}$ represent the true label corresponding to the $i$-th sample in the source domain; $N$ represents the number of training samples; $C$ represents the total number of categories; $h$, $w$, and $c$ represent the length, width, and category of the sample, respectively; $O$ represents the true label of the i-th sample in the source domain; $N$ represents the number of training samples; $C$ represents the total number of categories; $h$, $w$, and $c$ represent the cp and O ca These represent the semantic prediction graphs output by the main classifier and the auxiliary classifier, respectively. and These represent the segmentation losses of the main classifier and the auxiliary classifier, respectively:
[0118]
[0119] Where, α gp With (1-α) gp ) represent the coefficients of the main classifier and the auxiliary classifier, respectively, taking values of 0.7 and 0.3; L seg This represents the final partition loss;
[0120] The formula for the consistency regularization term is:
[0121]
[0122] Where N represents the total number of training samples; C represents the total number of categories in the training samples; h, w, and c represent the length, width, and category of the sample, respectively; O cp and O ca These represent the semantic prediction graphs output by the main classifier and the auxiliary classifier, respectively.
[0123] Step 3.2: Determine the loss function of the discriminant network as adversarial loss;
[0124] The adversarial loss function is:
[0125]
[0126] Among them, O dp Indicates the result of the first domain discrimination; O da This represents the discrimination result in the second domain; N represents the total number of training samples; h, w, and c represent the length, width, and class of the sample, respectively; and Let represent the adversarial losses generated by the primary discriminant network and the secondary discriminant network, respectively. Their weighted average yields the final adversarial loss as follows:
[0127]
[0128] in, and Let α represent the adversarial loss of the main classifier and the adversarial loss of the auxiliary classifier, respectively.dp With (1-α) dp ) represent the coefficients of the primary discriminant network and the secondary discriminant network, respectively, and are set to 0.8 and 0.2; L adv This indicates the final losses in the confrontation.
[0129] Step 4: Train the backbone network and master and slave classifiers using stochastic gradient descent (SGD); train the master and slave discriminator networks using the Adam optimizer as the optimization method.
[0130] Step 5: Test the trained improved domain adaptive Deeplab model on the test set, and calculate the average crossover ratio as 68.77%.
[0131] The average intersection-over-union ratios of the Deeplab models before and after the improvement on the test set are shown in Table 1:
[0132] Table 1
[0133]
[0134] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules, units, or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units, modules, or components may be combined or integrated into another device, or some features may be ignored or not executed.
[0135] The units may or may not be physically separate. The components shown as units can be one or more physical units, meaning they can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0136] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0137] In particular, according to embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this invention. It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0139] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. An improved semantic segmentation method for infrared ship images using a domain-adaptive Deeplab model, characterized in that, Includes the following steps: Obtain a dataset of visible light ship images and corresponding tags ; Acquiring Infrared Ship Image Dataset And the infrared ship image dataset Divided into training set and test set ; Construct an improved domain-adaptive Deeplab model; Using visible light ship image datasets ,Label and training set The improved domain-adaptive Deeplab model is used for semantic segmentation to obtain image semantic segmentation results. The specific steps include: Visible light ship image dataset and training set The samples are input into the backbone network, and the five convolutional layers of the backbone network output the first feature maps of visible light and infrared light, respectively. Second feature map Third feature map Fourth feature map and the fifth feature map ; The fourth feature map of visible light and infrared light and the fifth feature map The first semantic prediction maps for visible light and infrared light are obtained by inputting them into the main classifier and the auxiliary classifier, respectively. Second semantic prediction graph ; First semantic prediction maps of visible light and infrared light Second semantic prediction graph The main discriminant network and the auxiliary discriminant network are input respectively to obtain the first domain discrimination result. Second domain discrimination result ; Based on the first semantic prediction map of visible light Second semantic prediction graph and tags The final segmentation loss is calculated. Based on the first semantic prediction map of infrared Second semantic prediction graph The consistency regularization term is calculated. Based on the first domain discrimination result Second domain discrimination result The final adversarial loss is calculated from the final segmentation loss. The improved domain-adaptive Deeplab model includes a backbone network, a master classifier, an auxiliary classifier, a master discriminant network, and an auxiliary discriminant network.
2. The improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images according to claim 1, characterized in that, Based on the first semantic prediction map of visible light Second semantic prediction graph and tags The final segmentation loss is calculated, which includes the following steps: The loss functions for the main classifier and the auxiliary classifier are the cross-entropy loss functions; The cross-entropy loss function is: （1）； （2）； in, and These represent the segmentation losses of the main classifier and the auxiliary classifier, respectively. Represents the source domain. The true label corresponding to each sample; This represents the total number of training samples; This represents the total number of categories in the training samples; , and These represent the length, width, and category of the sample, respectively. and These represent the first semantic prediction graph and the second semantic prediction graph, respectively. Segmentation loss generated by the main classifier and the auxiliary classifier and After weighting, the final partition loss is obtained: （3）； in, and These represent the coefficients of the main classifier and the auxiliary classifier, respectively. This represents the final partition loss.
3. The improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images according to claim 2, characterized in that: The coefficients of the main classifier and the auxiliary classifier are set to 0.7 and 0.3, respectively.
4. The improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images according to claim 3, characterized in that, Based on the first semantic prediction map of infrared Second semantic prediction graph The consistency regularization term is calculated, which includes the following steps: The consistency regularization term used is the KL divergence, and the formula for the consistency regularization term is: （4）； in, This represents the total number of categories in the training samples; , and These represent the length, width, and category of the sample, respectively. and These represent the first semantic prediction graph and the second semantic prediction graph, respectively.
5. The improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images according to claim 4, characterized in that, Based on the first domain discrimination result Second domain discrimination result The final adversarial loss is calculated by combining the final segmentation loss with the final adversarial loss, specifically including the following steps: The loss functions of the main discriminant network and the auxiliary discriminant network are adversarial loss functions, respectively. The adversarial loss function is: （5）； （6）； in, This indicates the result of the first domain discrimination; This indicates the result of the second domain discrimination; This represents the total number of training samples; , and These represent the length, width, and category of the sample, respectively. and These represent the adversarial losses generated by the primary and secondary discriminant networks, respectively. and The final adversarial loss, obtained after weighted averaging, is as follows: （7）； in, and These represent the adversarial losses of the primary discriminant network and the secondary discriminant network, respectively. and These represent the coefficients of the primary discriminant network and the secondary discriminant network, respectively. This indicates the final losses in the confrontation.
6. The improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images according to claim 5, characterized in that, It also includes the following steps: The backbone network and master and slave classifiers were trained using stochastic gradient descent. The Adam optimizer was used as the optimization method to train the master and slave discriminant networks.
7. The improved domain-adaptive Deeplab model for semantic segmentation of infrared ship images according to claim 6, characterized in that, It also includes the following steps: Using the test set The trained, improved domain-adaptive Deeplab model was tested to obtain the average crossover ratio (CRO).
8. An improved domain-adaptive Deeplab model-based semantic segmentation system for infrared ship images, used to implement the improved domain-adaptive Deeplab model-based semantic segmentation method for infrared ship images as described in any one of claims 1-7, characterized in that, include: The acquisition module is used to acquire visible light ship image datasets. and corresponding tags ; The segmentation module is used to acquire infrared ship image datasets. And the infrared ship image dataset Divided into training set and test set ; Build modules for constructing improved domain-adaptive Deeplab models; The semantic segmentation module is used to utilize visible light ship image datasets. ,Label and training set Semantic segmentation was performed on the improved domain-adaptive Deeplab model to obtain the image semantic segmentation results.