Remote sensing image semantic segmentation method and system based on prototype perception and edge enhancement
The semantic segmentation of remote sensing images is performed by using the Resnet50 network and PMMs to generate prototypes. The Sobel operator is combined to enhance edge information, which solves the problems of semantic ambiguity and neglect of edge details in remote sensing images in the case of few samples, and achieves higher segmentation accuracy and classification accuracy.
Patent Information
- Application Number
- CN202510357351.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-25
- Publication Date
- 2025-10-03
- Estimated Expiration
- 2045-03-25
AI Technical Summary
Traditional remote sensing image semantic segmentation methods suffer from semantic ambiguity and neglect of edge details in the case of few samples, resulting in poor segmentation results, especially in complex or fuzzy boundary areas.
The Resnet50 network is used to extract features, and PMMs are combined to generate prototypes and perform pixel-level matching. The Sobel operator is used to enhance edge information, and the query features, global matching features and edge enhancement features are fused for prediction.
Improve segmentation accuracy when labeled data is limited, reduce dependence on large-scale datasets, enhance segmentation accuracy in boundary areas, and improve classification accuracy.
Smart Images

Figure CN120164220B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing image semantic segmentation, and in particular to a remote sensing image semantic segmentation method and system based on prototype perception and edge enhancement. Background Art
[0002] Semantic segmentation of remote sensing images involves classifying each pixel in a remote sensing image and labeling it as a specific semantic category, such as forest, road, or building. This method has important applications in land use, environmental monitoring, urban planning, and other fields. With the rapid development of deep learning technology, the accuracy of semantic segmentation of remote sensing images has significantly improved. However, traditional deep learning methods typically rely on large amounts of labeled data to train the model, which is particularly difficult and costly for remote sensing imagery.
[0003] Few-shot segmentation involves accurately segmenting objects in an image using only a small amount of labeled data. Specifically, few-shot segmentation tasks typically utilize a small number of support samples to guide the model in segmenting the query image. This approach not only reduces the reliance on large datasets but also achieves good segmentation results in data-scarce environments. In the field of remote sensing, in particular, the high cost and inefficiency of data annotation make few-shot segmentation a very promising solution.
[0004] However, existing traditional few-shot segmentation methods often have certain limitations. Most methods rely on directly obtaining a single prototype from the support image to segment the query image, which leads to semantic ambiguity and difficulty in distinguishing its category, especially when the number of samples is small. In addition, most methods also ignore the spatial details in the query image, especially the information of the object edge. The edge is an important dividing line between the object and the background, and it plays a vital role in object recognition. However, traditional methods often rely only on overall category features and lack detailed attention to the object edge, which leads to poor segmentation effect of the model in complex or fuzzy boundary areas. Summary of the Invention
[0005] In order to address the shortcomings of the existing technology, the present invention provides a remote sensing image semantic segmentation method and system based on prototype perception and edge enhancement. First, Resnet50 is used as a feature extractor to generate corresponding query and support features. Secondly, PMMs are used to obtain multiple prototypes from the support features, and global matching features are obtained through the PF Matching module to achieve pixel-level matching between the query features and the support prototypes. Thirdly, the Sobel operator is used to extract the edge information of the query image to enhance the query features, thereby obtaining edge-enhanced features. Finally, the query features, global matching features, and edge-enhanced features are fused together, and the fused features are subjected to spatial pyramid pooling and sent to the predictor to obtain the final segmentation result.
[0006] On the one hand, a remote sensing image semantic segmentation method based on prototype perception and edge enhancement is provided, including:
[0007] Obtain support images and query images; randomly sample K+1 images from a remote sensing dataset, K of which serve as support images and one as a query image; the support images are set with target object category labels; the query image has no label;
[0008] K supporting images and one query image are input into a trained remote sensing image semantic segmentation model to obtain a semantic segmentation result of the query image; wherein the trained remote sensing image semantic segmentation model includes: extracting features from the K supporting images to obtain supporting features; extracting a number of prototypes from the supporting features; extracting features from the query image to obtain query features; performing pixel-level feature matching on the query features based on the prototypes to obtain global matching features; extracting edge information from the query image, and enhancing the query features based on the extracted edge information to obtain edge enhanced features; fusing the query features, global matching features and edge enhanced features to obtain fused features, and predicting the fused features to obtain an image segmentation result.
[0009] On the other hand, a remote sensing image semantic segmentation system based on prototype perception and edge enhancement is provided, including:
[0010] An acquisition module is configured to: acquire support images and query images; randomly sample K+1 images from a remote sensing dataset, wherein K images are used as support images and one image is used as a query image; the support images are provided with target object category labels; and the query image is unlabeled;
[0011] The segmentation module is configured to: input K supporting images and one query image into a trained remote sensing image semantic segmentation model to obtain a semantic segmentation result of the query image; wherein the trained remote sensing image semantic segmentation model includes: extracting features from the K supporting images to obtain supporting features; extracting several prototypes from the supporting features; extracting features from the query image to obtain query features; performing pixel-level feature matching on the query features based on the prototypes to obtain global matching features; extracting edge information from the query image, and enhancing the query features based on the extracted edge information to obtain edge enhanced features; fusing the query features, global matching features and edge enhanced features to obtain fused features, and predicting the fused features to obtain an image segmentation result.
[0012] The above technical solution has the following advantages or beneficial effects:
[0013] Compared to traditional deep learning methods, this approach can improve segmentation accuracy when labeled data is limited, reduce reliance on large datasets, and lower dataset annotation costs. Addressing the shortcomings of traditional few-shot segmentation methods in processing object boundaries, this method leverages edge prior information to enhance features, resulting in more refined segmentation results in boundary regions, reducing boundary ambiguity and improving classification accuracy, making it potentially applicable to a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0015] Figure 1 This is a flow chart of the method of embodiment 1. DETAILED DESCRIPTION
[0016] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.
[0017] Example 1
[0018] This embodiment provides a remote sensing image semantic segmentation method based on prototype perception and edge enhancement;
[0019] like Figure 1 As shown in FIG, the remote sensing image semantic segmentation method based on prototype perception and edge enhancement includes:
[0020] S101: Acquire support images and query images; randomly sample K+1 images from a remote sensing dataset, wherein K images are used as support images and one image is used as a query image; the support images are provided with target object category labels; the query image is unlabeled;
[0021] S102: K supporting images and one query image are input into the trained remote sensing image semantic segmentation model to obtain the semantic segmentation result of the query image; wherein the trained remote sensing image semantic segmentation model includes: extracting features from the K supporting images to obtain supporting features; extracting several prototypes from the supporting features; extracting features from the query image to obtain query features; performing pixel-level feature matching on the query features based on the prototypes to obtain global matching features; extracting edge information from the query image, and enhancing the query features based on the extracted edge information to obtain edge enhanced features; fusing the query features, global matching features and edge enhanced features to obtain fused features, and predicting the fused features to obtain image segmentation results.
[0022] Furthermore, the remote sensing data set includes:
[0023] First, the original remote sensing image is cropped into a size of The images are divided into training set and test set according to the ratio of 7:3. Depend on The test set consists of image-mask pairs. Depend on image-label pairs, where for RGB image, is the corresponding segmentation label GroundTruth. The label not only marks the category of the target object, but also marks the area where the target object is located.
[0024] Then, all the land feature types in the dataset are divided into two non-overlapping category subsets and , taking the iSAID dataset as an example, the dataset has 15 categories in total, 10 of which are classified as : Ships, storage tanks, baseball fields, tennis courts, basketball courts, track and field fields, bridges, large vehicles, small vehicles, helicopters, and the remaining 5 categories are classified as : Swimming pool, island, football field, airplane, port.
[0025] In the training phase of the remote sensing image semantic segmentation model, The categories in the training image are set to be invisible. Pixels of these categories are considered as background and do not participate in the calculation of subsequent loss functions, nor will they trigger any parameter updates or gradient backpropagation for these categories.
[0026] In the testing phase of the remote sensing image semantic segmentation model, The categories in are set to be visible and participate in the calculation of the final loss function.
[0027] In the training phase of the remote sensing image semantic segmentation model, the training set Random sampling image-label pairs form a set of training data , each set of training data contains K support image-label pairs and 1 query image-label pair .
[0028] During the training phase of the remote sensing image semantic segmentation model, the model learns the mapping relationship between image-label pairs and predicts the query image. The segmentation label , and calculate the cross entropy loss function Update the model weights. After the model training is completed, evaluate the model on the test set.
[0029] In the testing phase of the remote sensing image semantic segmentation model, the test set Random sampling The image-label pairs form a set of test data, each set of test data contains K support image-label pairs and 1 query image-label pair. The model finally predicts the segmentation label of the query image , and with Comparison is used to evaluate the model segmentation performance.
[0030] Furthermore, the feature extraction of the K supporting images to obtain supporting features uses a Resnet50 network as a feature extractor.
[0031] Furthermore, the feature extraction of the query image to obtain the query features uses a Resnet50 network as a feature extractor.
[0032] It should be understood that a pre-trained convolutional neural network (CNN) is used as a feature extractor, specifically using ResNet50 as the backbone network to extract features of the query image and the support image. The ResNet50 network includes: a first residual block conv2_x, a second residual block conv3_x, a third residual block conv4_x, and a fourth residual block conv5_x connected in sequence, each residual block containing multiple residual units. Assume that the input image size is , H and W represent the height and width of the image respectively, and 3 is the number of channels. After the input image passes through the ResNet50 network, the output features of the fourth residual block conv5_x of the ResNet50 network are .
[0033] In order to extract global context information more comprehensively, the pyramid pooling module (PPM) is used to perform multi-scale feature integration on the output of the fourth residual block of the ResNet50 network. Specifically, the pyramid pooling module is used to perform multi-scale feature integration on the output of the fourth residual block conv5_x. 、 、 The average pooling operation of scale is performed and The convolution reduces the channel dimension to one third, and then uses bilinear interpolation to upsample the pooled feature map to restore it to its original size. Finally, the feature maps from different scales are spliced to obtain the support features of the support image. .
[0034] ;
[0035] in, For splicing operation, is the upsampling operation, is the output of the fourth residual block, Represents the pooling output of the corresponding scale.
[0036] Among them, the query features of the query image The acquisition process of is the same as that of the support features of the support image.
[0037] Furthermore, several prototypes are extracted from the supporting features by using Prototype Mixture Models (PMMs) to generate several prototypes from the supporting features.
[0038] A prototype is a representative vector calculated from the features of the support image. It provides the segmentation model with a representative feature of a category. Associating a category with multiple prototypes can more effectively capture details and utilize supporting features in the support image. This allows the model to compare these prototypes with the query image to identify and segment the target object.
[0039] Furthermore, several prototypes are extracted from the supporting features. The specific process includes:
[0040] Assume that the feature map of the support image is , split it according to the channel dimension to obtain The length is Feature samples , .
[0041] PMMs are probabilistic mixture models that combine the probabilities of the underlying distributions:
[0042] ;
[0043] ;
[0044] ;
[0045] ;
[0046] in, is the mixing weight, satisfying and , are parameters learned as the model progresses, represents the i-th feature sample, represents the k-th probability model based on the kernel distance function, are parameters that are updated as the model is trained, is the normalization factor, is the Bessel function, is a fixed parameter, set to 20, express The mean vector of , that is, the prototype, is the kernel distance function, is the L2 vector norm. K prototypes are obtained by calculating the mean vector of K probability models based on the kernel distance function.
[0047] During the training process of the probabilistic mixture model PMMs, optimization is performed based on the expectation maximization algorithm. Each iteration takes the following steps:
[0048] (1-1): Calculate the expected value of each sample belonging to each prototype , according to the sample Calculate its belonging to each prototype Probability of:
[0049] ;
[0050] in, It is a fixed parameter and is set to 20.
[0051] (1-2): Update the prototype based on the calculated expected value :
[0052] ;
[0053] Repeat steps (1-1) and (1-2) to continuously optimize the prototype estimate until convergence.
[0054] Furthermore, performing pixel-level feature matching on the query feature based on the prototype to obtain a global matching feature includes:
[0055] Assume that the query image feature map ,for For each pixel position (x, y), calculate its feature vector and prototype The cosine similarity between :
[0056] ;
[0057] For each prototype, a total of cosine similarity values and use them to construct a similarity graph , ; Calculate a similarity value for each pixel. Pixels constitute the similarity graph;
[0058] By calculating the similarity between K prototypes and query image features, K pixel-level similarity maps are obtained.
[0059] For each pixel position in the query feature, select the prototype with the greatest similarity Match it and replace each pixel position in the query feature with the prototype with the greatest similarity Specifically, for each pixel position (x, y), select the prototype with the greatest similarity :
[0060] ;
[0061] Represents the index of the most similar prototype at pixel position (x,y);
[0062] Each pixel of the query feature is then replaced with its most similar prototype , thereby constructing global matching features , global matching features The size of is the same as the query feature, and the global matching feature The feature of each position comes from the prototype with the greatest similarity .
[0063] The advantage of the above technical solution is that the generated prototype can be matched with the query feature at the pixel level to improve the segmentation accuracy.
[0064] Furthermore, the edge information extraction of the query image includes: using the Sobel operator to extract the edge information: first, using the Sobel operator to extract the edge information of the query image For edge detection, the Sobel operator calculates The gradient of each pixel highlights the edge area in the image;
[0065] After the Sobel operator is processed, the processing result is normalized by the Sigmoid activation function to obtain the edge map ;
[0066] ;
[0067] Where S is the sigmoid activation function.
[0068] Furthermore, the step of enhancing the query feature based on the extracted edge information to obtain the edge enhanced feature includes:
[0069] The edge graph Feature map of the query image Perform element-wise multiplication to highlight edge regions while suppressing features in background regions:
[0070] ;
[0071] in, represents an element-wise multiplication operation, is a learnable weight coefficient used to control the edge feature pair The impact of It is an edge enhancement feature.
[0072] Furthermore, the query feature, the global matching feature and the edge enhancement feature are fused to obtain the fused feature, including:
[0073] Will 、 、 Fusion is performed in the channel dimension and The convolution reduces the number of channels, and then the fused features are passed to the Atrous Spatial Pyramid Pooling (ASPP) module to obtain the final feature map The ASPP module obtains multi-scale information through convolutional layers with different expansion rates, which helps to capture the features of objects of different scales in remote sensing images. :
[0074] ;
[0075] Furthermore, the prediction of the fusion features to obtain the image segmentation result includes:
[0076] The feature map Send it to the predictor to get the final prediction output;
[0077] The predictor includes: upsampling layers and Convolutional layer.
[0078] Furthermore, in step S102, both the support image and the query image are input into the trained remote sensing image semantic segmentation model to obtain an image segmentation result; wherein the training process of the trained remote sensing image semantic segmentation model includes:
[0079] Constructing a remote sensing dataset, wherein the remote sensing dataset is K support images with known segmentation results and one query image; dividing the remote sensing dataset into a training set and a test set in proportion;
[0080] Input the training set into the model and train the model. During the training process, the support image and query image are used as the input values of the model, and the image segmentation result is used as the output value of the model.
[0081] When the total loss function value of the model no longer decreases, or the number of iterations reaches the set number, the training is stopped to obtain the trained remote sensing image semantic segmentation model.
[0082] Furthermore, the total loss function of the model includes:
[0083] ;
[0084] ;
[0085] ;
[0086] ;
[0087] in, is an adjustable constant used to control the effect of auxiliary loss, represents the total loss function, represents the cross entropy loss function, represents the auxiliary loss function; Represents the total number of N pixels; represents a probability distribution; is the predicted value of the pixel at position (i, j), and are the height and width of the image, is a hyperparameter, is a constant.
[0088] Using the cross entropy loss function Used to measure the difference between the predicted label and the true label. Assuming that the input image has N pixels, the output of the model is a probability distribution of each pixel belonging to one of K categories. .
[0089] In order to alleviate the scale imbalance problem in remote sensing images, an auxiliary loss function is used to make the model focus on small objects that are difficult to segment, and the size of the target object in the image is specified to be less than Pixels are small objects. First, we get the scale score by calculating the probability of each pixel in the prediction mask. , the rating It is obtained by weighted summing of the predicted probabilities of each pixel. When it is small, it indicates that the size of the target object is small, or there is a large deviation between the model's prediction in this area and the true label. The value of dynamically adjusts the loss weight of each pixel.
[0090] in, is a hyperparameter that controls the degree of attention paid to small objects. This is a very small constant used to avoid numerical instability when the score is very small. When the score is close to 0 (indicating a small object), the loss value will increase significantly, prompting the model to focus on small objects.
[0091] Example 2
[0092] This embodiment provides a remote sensing image semantic segmentation system based on prototype perception and edge enhancement, including:
[0093] An acquisition module is configured to: acquire support images and query images; randomly sample K+1 images from a remote sensing dataset, wherein K images are used as support images and one image is used as a query image; the support images are provided with target object category labels; and the query image is unlabeled;
[0094] The segmentation module is configured to: input K supporting images and one query image into a trained remote sensing image semantic segmentation model to obtain a semantic segmentation result of the query image; wherein the trained remote sensing image semantic segmentation model includes: extracting features from the K supporting images to obtain supporting features; extracting several prototypes from the supporting features; extracting features from the query image to obtain query features; performing pixel-level feature matching on the query features based on the prototypes to obtain global matching features; extracting edge information from the query image, and enhancing the query features based on the extracted edge information to obtain edge enhanced features; fusing the query features, global matching features and edge enhanced features to obtain fused features, and predicting the fused features to obtain an image segmentation result.
[0095] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. A remote sensing image semantic segmentation method based on prototype perception and edge enhancement, characterized by: include: Get support images and query images; Randomly sample K+1 images from a remote sensing dataset, where K images are used as support images and one image is used as a query image; the support images are set with target object category labels; the query image has no label; K supporting images and a query image are input into a trained remote sensing image semantic segmentation model to obtain a semantic segmentation result of the query image; wherein the trained remote sensing image semantic segmentation model includes: extracting features from the K supporting images to obtain supporting features; extracting a number of prototypes from the supporting features; extracting features from the query image to obtain query features; performing pixel-level feature matching on the query features based on the prototypes to obtain global matching features; extracting edge information from the query image and enhancing the query features based on the extracted edge information to obtain edge enhanced features; fusing the query features, the global matching features, and the edge enhanced features to obtain fused features, and predicting the fused features to obtain an image segmentation result; The total loss function of the model includes: ; ; ; ; in, is an adjustable constant used to control the effect of auxiliary loss, represents the total loss function, represents the cross entropy loss function, represents the auxiliary loss function; Represents the total number of N pixels; represents a probability distribution; is the predicted value of the pixel at position (i, j), and are the height and width of the image, is a hyperparameter, is a constant; using the cross entropy loss function Used to measure the difference between the predicted label and the true label. Assuming that the input image has N pixels, the output of the model is a probability distribution of each pixel belonging to one of K categories. ; During the training phase, the model learns the mapping relationship between supported image-label pairs and predicts the query image. The segmentation label , and calculate the cross entropy loss function Update the model's weights.
2. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement as claimed in claim 1, characterized in that: Feature extraction is performed on K support images to obtain support features, using the Resnet50 network as the feature extractor; Feature extraction is performed on the query image to obtain query features. The Resnet50 network is used as the feature extractor; Use the pyramid pooling module to perform the output of the fourth residual block conv5_x separately 、 、 The average pooling operation of scale is performed, and The convolution reduces the channel dimension to one third, and then uses bilinear interpolation to upsample the pooled feature map to restore it to its original size. Finally, the feature maps from different scales are spliced to obtain the support features of the support image. ; ; in, For splicing operation, is the upsampling operation, is the output of the fourth residual block, Represents the pooling output of the corresponding scale; query features of the query image The acquisition process of is the same as that of the support features of the support image.
3. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement as claimed in claim 1, characterized in that: Extract several prototypes from the supporting features. The specific process includes: Assume that the feature map of the support image is , split it according to the channel dimension to obtain The length is Feature samples , ; PMMs are probabilistic mixture models that combine the probabilities of the underlying distributions: ; ; ; ; in, is the mixing weight, satisfying and , are parameters learned as the model progresses, represents the i-th feature sample, represents the k-th probability model based on the kernel distance function, are parameters that are updated as the model is trained, is the normalization factor, is the Bessel function, is a fixed parameter, express The mean vector of , that is, the prototype, is the kernel distance function, is the L2 vector norm; K prototypes are obtained by calculating the mean vector of K probability models based on the kernel distance function.
4. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement as claimed in claim 3, characterized in that: During the training process of the probabilistic mixture model PMMs, optimization is performed based on the expectation maximization algorithm. Each iteration takes the following steps: (1-1): Calculate the expected value of each sample belonging to each prototype , according to the sample Calculate its belonging to each prototype Probability of: ; in, is a fixed parameter; (1-2): Update the prototype based on the calculated expected value : ; Repeat steps (1-1) and (1-2) to continuously optimize the prototype estimate until convergence.
5. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement as claimed in claim 1, characterized in that: The pixel-level feature matching of the query feature based on the prototype to obtain the global matching feature includes: Assume that the query image feature map ,for For each pixel position (x, y), calculate its feature vector and prototype The cosine similarity between : ; For each prototype, a total of cosine similarity values and use them to construct a similarity graph , ; Calculate a similarity value for each pixel. Pixels constitute the similarity graph; By calculating the similarity between K prototypes and query image features, K pixel-level similarity maps are obtained; For each pixel position in the query feature, select the prototype with the greatest similarity Match it and replace each pixel position in the query feature with the prototype with the greatest similarity Specifically, for each pixel position (x, y), select the prototype with the greatest similarity : ; Represents the index of the most similar prototype at pixel position (x,y); Each pixel of the query feature is then replaced with its most similar prototype , thereby constructing global matching features , global matching features The size of is the same as the query feature, and the global matching feature The feature of each position comes from the prototype with the greatest similarity .
6. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement according to claim 1, wherein: Extract edge information from the query image, including: using the Sobel operator to extract edge information: First, use the Sobel operator to extract edge information from the query image. For edge detection, the Sobel operator calculates The gradient of each pixel highlights the edge area in the image; After the Sobel operator is processed, the processing result is normalized by the Sigmoid activation function to obtain the edge map ; ; Where S is the sigmoid activation function.
7. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement according to claim 1, wherein: Based on the extracted edge information, the query feature is enhanced to obtain edge enhancement features, including: The edge graph Feature map of the query image Perform element-wise multiplication to highlight edge regions while suppressing features in background regions: ; in, represents an element-wise multiplication operation, is a learnable weight coefficient used to control the edge feature pair The impact of It is an edge enhancement feature.
8. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement as claimed in claim 1, characterized in that: The query feature, the global matching feature and the edge enhancement feature are fused to obtain the fused feature, including: Will 、 、 Fusion is performed in the channel dimension and The convolution reduces the number of channels, and then the fused features are passed to the void space pyramid pooling ASPP module to obtain the final feature map The ASPP module obtains multi-scale information through convolutional layers with different expansion rates, which helps to capture the features of objects of different scales in remote sensing images. : 。 9. The remote sensing image semantic segmentation method based on prototype perception and edge enhancement according to claim 1, wherein: The trained remote sensing image semantic segmentation model. The training process includes: Constructing a remote sensing dataset, wherein the remote sensing dataset is K support images with known segmentation results and one query image; dividing the remote sensing dataset into a training set and a test set in proportion; Input the training set into the model and train the model. During the training process, the support image and query image are used as the input values of the model, and the image segmentation result is used as the output value of the model. When the total loss function value of the model no longer decreases, or the number of iterations reaches the set number, the training is stopped to obtain the trained remote sensing image semantic segmentation model.
10. A remote sensing image semantic segmentation system based on prototype perception and edge enhancement, characterized by: include: An acquisition module is configured to: acquire a support image and a query image; Randomly sample K+1 images from a remote sensing dataset, where K images are used as support images and one image is used as a query image; the support images are set with target object category labels; the query image has no label; The segmentation module is configured to: input K support images and a query image into a trained remote sensing image semantic segmentation model to obtain a semantic segmentation result of the query image; wherein the trained remote sensing image semantic segmentation model includes: extracting features from the K support images to obtain support features; extracting a plurality of prototypes from the support features; extracting features from the query image to obtain query features; performing pixel-level feature matching on the query features based on the prototypes to obtain global matching features; extracting edge information from the query image and enhancing the query features based on the extracted edge information to obtain edge enhanced features; performing feature fusion on the query features, the global matching features, and the edge enhanced features to obtain fused features, and predicting the fused features to obtain an image segmentation result; The total loss function of the model includes: ; ; ; ; in, is an adjustable constant used to control the effect of auxiliary loss, represents the total loss function, represents the cross entropy loss function, represents the auxiliary loss function; Represents the total number of N pixels; represents a probability distribution; is the predicted value of the pixel at position (i, j), and are the height and width of the image, is a hyperparameter, is a constant; using the cross entropy loss function Used to measure the difference between the predicted label and the true label. Assuming that the input image has N pixels, the output of the model is a probability distribution of each pixel belonging to one of K categories. ; During the training phase, the model learns the mapping relationship between supported image-label pairs and predicts the query image. The segmentation label , and calculate the cross entropy loss function Update the model's weights.
Citation Information
Patent Citations
Intestinal polyp segmentation method and system based on small sample learning
CN115049603A