A few-shot semantic segmentation method based on foreground enhancement and background diffusion contrast

Abstract

The application discloses a few-shot semantic segmentation method based on foreground enhancement and background diffusion contrast, comprising: obtaining the image background and image foreground of each support image, diffusing the image background to obtain a background diffusion input set; inputting the updated support set and query set into two backbone networks sharing parameters to extract common image features; inputting the extracted support feature set and query feature into a foreground enhancement module and a background diffusion module in a parallel manner respectively to construct a foreground similarity map, foreground enhancement features with the same size as the support features and background diffusion weights respectively; performing feature reparameterization on the query feature according to the background diffusion weight, and inputting the reparameterized query feature, the foreground similarity map and the foreground enhancement features into a feature fusion module to generate a final binary segmentation result. The application effectively improves the generalization ability of the few-shot semantic segmentation model for new semantic concepts.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a few-sample semantic segmentation method based on foreground enhancement and background diffusion contrast. Background Technology

[0002] In recent years, with the continuous development of deep convolutional neural networks (CNNs), significant progress has been made in various computer vision tasks. However, training visual models using CNNs heavily relies on large-scale manually labeled datasets, which often require substantial human effort to construct. Furthermore, such trained deep learning networks can only work on predefined classes within the dataset, making it difficult to generalize to classes not included in the dataset, resulting in extremely limited generalization. Therefore, few-shot learning has been proposed. This type of task addresses the problem of unseen classes by training a general model, enabling the identification of novel concepts not covered in the dataset given very limited information.

[0003] As a classic task in computer vision, semantic segmentation requires pixel-level labeled datasets, which are more complex to obtain. Therefore, few-shot segmentation (FSS) has attracted widespread attention. FSS aims to define a deep learning model trained on minimal labeled data to segment objects of specific semantic categories not included in the training set. Existing FSS models typically mine object features from support set images and query set images separately, then utilize different feature matching methods to extract target category information to predict the segmentation result of the query image. The most commonly used method is prototype learning, which uses Masked Average Pooling (MAP) to generate one or more prototypes for meta-inference based on the extracted feature maps, providing general foreground feature matching cues for the query object. These prototypes obtained by compressing support features through MAP are often more robust than pixel-level features, but their spatial information is inevitably lost, making the semantic information covered by these prototypes too general and unable to effectively distinguish the inherent appearance differences between objects of the same category. The significant appearance diversity among target objects makes it difficult for support features to cover the potential appearance variations of target objects in the query image. Furthermore, the MAP operation can lead to object loss when processing input images with small objects, resulting in the generated foreground prototypes lacking any effective information. Moreover, in existing prototype-based FSS models, background information of the input image is often naturally ignored. The information used to segment unseen classes comes entirely from foreground objects in the support set images, resulting in limited information categories and a lack of generalization ability, thus restricting the overall generalization capability of the model.

[0004] Patent publication number CN116935041A discloses a few-shot semantic segmentation method, apparatus, device, and storage medium, comprising: acquiring an image to be segmented and acquiring a target segmentation model constructed based on an encoder-decoder structure; extracting target features from the image to be segmented using the encoder of the target segmentation model; performing foreground and background segmentation on the image to be segmented using the target segmentation model to obtain a background segmentation map, and fusing the target features and the background segmentation map to obtain segmentation features; decoding the segmentation features using the decoder of the target segmentation model to obtain a feature image, and inputting the feature image into a classifier in the target segmentation model to obtain a segmentation result labeled with categories. This invention improves the accuracy of few-shot semantic segmentation.

[0005] Patent publication number CN112150471B discloses a semantic segmentation method, apparatus, electronic device, and storage medium based on a limited number of samples. The method includes: using a combination of target images as input to a trained feature extraction network to obtain image features of each image in the combination; wherein the combination of target images includes a target category to be identified, several support images, and a query image, and the support images carry a foreground mask and a background mask corresponding to the target category to be identified; calculating the foreground class prototype and background class prototype corresponding to the target category to be identified based on the image features, foreground class prototype, and background class prototype of each support image; and determining the predicted foreground mask corresponding to the target category to be identified on the query image based on the image features, foreground class prototype, and background class prototype of the query image. This application can achieve semantic segmentation for target categories with a limited number of samples by using a feature extraction network trained with sample images labeled with other categories.

[0006] However, neither of the aforementioned inventions addresses the technical problems inherent in the aforementioned MAP operation. Summary of the Invention

[0007] The technical problem to be solved by this invention is that, during the prototype generation process, the support set image mask needs to be scaled to the same size as the support set feature map, and then multiplied element-wise with the feature map to obtain a foreground feature map with the background filtered out. Because some categories of segmented objects occupy a small proportion of the input image, the segmentation targets may completely disappear during the scaling process using bilinear interpolation, causing the subsequent MAP operation to generate a prototype that does not contain foreground information.

[0008] To address this issue, this invention proposes a few-shot semantic segmentation method based on foreground enhancement and background diffusion contrast. It directly calculates channel attention on the feature map using a gated channel transformation method to generate attention prototypes, which are then concatenated with prototypes generated by conventional MAP to calculate a dual similarity map for subsequent feature matching. Simultaneously, this method utilizes diffusion to diffuse the background of the input image, generating multiple background prototypes which are then interactively reparameterized to assist in the subsequent feature matching process. These rich and generalizable prototypes provide targeted guidance for segmentation.

[0009] Technical solution:

[0010] A few-shot semantic segmentation method based on foreground enhancement and background diffusion contrast, the few-shot semantic segmentation method includes the following steps:

[0011] Step A: Divide the given semantic segmentation dataset into a training set and a test set. During training, each segment is sampled from the training set to obtain a support set and a query set.

[0012] Step B involves processing all supporting images in the support set to obtain the image background and image foreground of each supporting image. The image background is diffused, and the diffused image background is combined with the image foreground to obtain the background diffusion input set, which updates the support set. The updated support set and query set are then input into two backbone networks with shared parameters to extract common image features, resulting in the support feature set and query features.

[0013] Step C: The extracted support feature set and query features are input into the foreground enhancement module and background diffusion module in parallel to construct the foreground similarity map, the foreground enhancement feature with the same size as the support features, and the background diffusion weights, respectively.

[0014] Step D: Based on the background diffusion weight, the query features are reparameterized. The reparameterized query features, foreground similarity map, and foreground enhancement features are then fed into the feature fusion module to generate the final binary segmentation result.

[0015] Further, in step A, the given semantic segmentation dataset is divided into a training set and a test set. During training, the process of sampling each segment from the training set to obtain the support set and query set includes the following steps:

[0016] Given a semantic segmentation dataset D, where the semantic categories contained in the dataset D are represented by a set c, divide the dataset into training sets D. train and test set D test Training set D train and test set D test The included categories are C train={C1, C2, ..., C n}and

[0017] During training, each segment is taken from the training set D. train Mid-sampling yields support set And query set Q = {I q M q}, where K represents the number of samples in each segment; and I q This represents the i-th supporting image and the query image, with corresponding binary masks of respectively. and M q .

[0018] Further, in step B, all supporting images in the support set are processed to obtain the image background and image foreground of each supporting image. The image background is diffused, and the diffused image background is combined with the image foreground to obtain the background diffusion input set. The process of updating the support set includes the following steps:

[0019] Step B1, for each support image in the support set S Foreground binary mask The binary mask of its background is obtained through inversion.

[0020] Step B2, the binary mask With supporting images Element-wise multiplication yields the supporting image background. and image foreground Based on the diffusion process of the Diffusion model, the image background is divided into... Diffusion extends to rounds 25 and 50, affecting the image background. Add noise with varying degrees of diffusion:

[0021]

[0022] in Given a scalar, Z is a random number that follows a standard Gaussian distribution.

[0023] Step B3: Combine the diffused image background with the original image foreground to obtain the background diffusion input set. Where k = 0 represents the original image. After diffusion processing, the input support set is transformed into...

[0024] Step B4, will With Q={I q Mq The data is fed into a backbone network with shared parameters, and common image features are extracted to obtain a support feature set. and query feature F q It supports the generation of images from feature maps contained in the feature set, which are composed of the original background, the background diffused for 25 steps, and the background diffused for 50 steps.

[0025] Furthermore, in step C, the extracted support feature set The query features and the foreground enhancement feature are input into the foreground enhancement module in parallel to construct a foreground similarity map and a foreground enhancement feature of the same size as the supporting features. The specific steps include:

[0026] Step C11, The channel attention prototype is obtained by performing gated channel transformation and MAP operation respectively. and supporting future prototypes The masked average pooling (MAP) operation is performed according to the following formula:

[0027]

[0028] Where (x, y) represents the position of an element on the feature map, and h and w represent the maximum values ​​of the length and width of the feature map.

[0029] Step C12, utilizing the supporting foreground prototype With query feature F q Calculate the cosine similarity to obtain the foreground similarity map. fg ;

[0030] Step C13 will support the foreground prototype. With channel attention prototype Cascade and then with query feature F q Calculate the cosine similarity to obtain a double similarity map (Map). Dual The index map (Map) is obtained by using the Argmax function to find the index of the element with the maximum cosine similarity to the query feature for the concatenated prototype. index ;

[0031] Step C14, based on the index map (Map) index Embed the corresponding supporting foreground prototype at each element position Or channel attention prototype Construct foreground enhancement features of the same size as the supporting features.

[0032] Furthermore, in step C, the extracted support feature set The process of inputting query features in parallel into the background diffusion module to construct the background diffusion weights includes the following steps:

[0033] Step C21, Support Feature Set After passing through the MAP operation layer and the fully connected layer respectively, the corresponding background prototype set is obtained. In the MAP operation, a support set background mask is used to filter out foreground information, resulting in a background prototype that is only associated with the background.

[0034] Step C22: The supporting features and background prototypes are fed into the weighted interaction layer. For each supporting feature, weights are assigned to two background prototypes that do not correspond to it. The feature map with the new weights is then compared with the corresponding background mask, and a pair of background diffusion prototype sets are obtained through a MAP operation.

[0035] Step C23, set the background diffusion prototype. Each background diffusion prototype in the model is associated with the query feature F. q Calculate the cosine similarity and examine its relationship with the query feature F. q To determine the relevance of information, a monotonically decreasing reparameterization module based on an exponential function is used to re-parameterize the cosine similarity calculation results. The re-parameterized similarity maps are then summed to obtain the background diffusion weight. bgd .

[0036] The present invention also discloses a few-shot semantic segmentation device based on foreground enhancement and background diffusion comparison based on the aforementioned method, characterized in that the few-shot semantic segmentation device includes a dataset input module, a diffusion module, two backbone networks with shared parameters, a foreground enhancement module, a background diffusion module, a feature fusion module, and a prediction result output module;

[0037] The dataset input module is used to divide a given semantic segmentation dataset into a training set and a test set. During training, each segment is sampled from the training set to obtain a support set and a query set.

[0038] The diffusion module is used to process all supporting images in the support set to obtain the image background and image foreground of each supporting image, diffuse the image background, combine the diffused image background with the image foreground to obtain the background diffusion input set, and update the support set.

[0039] The two backbone networks sharing the parameters process the updated support set and query set respectively, extract common image features, and obtain the support feature set and query features;

[0040] The foreground enhancement module processes the extracted support feature set and query features to construct a foreground similarity map and a foreground enhancement feature of the same size as the support features;

[0041] The background diffusion module processes the extracted support feature set and query features to construct the background diffusion weights;

[0042] The feature fusion module reparameterizes the query features based on the background diffusion weight, processes the reparameterized query features, foreground similarity map, and foreground enhancement features to generate the final binary segmentation result, which is output by the prediction result output module.

[0043] Beneficial effects:

[0044] This invention presents a few-shot semantic segmentation method based on a combination of foreground enhancement and background diffusion. Unlike other methods that directly use foreground masks to filter out all background information, this invention focuses on discovering semantic information in the foreground features that is beneficial to segmentation. This invention aims to support traditional prototype learning methods by maximizing the use of information contained in the background. It enriches background information through background diffusion and compares the common background features that have the highest and lowest correlation with the query set features through weight interaction. These features are then labeled to provide useful guidance for the final feature fusion process, thereby improving the generalization ability of the few-shot semantic segmentation model for novel semantic concepts. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating the principle of a few-shot semantic segmentation method based on a combination of foreground enhancement and background diffusion, as described in an embodiment of the present invention.

[0046] Figure 2 This is a structural diagram of the foreground enhancement module according to an embodiment of the present invention;

[0047] Figure 3 This is a structural diagram of the background diffusion module according to an embodiment of the present invention. Detailed Implementation

[0048] The following embodiments are provided to enable those skilled in the art to more fully understand the present invention, but do not limit the invention in any way.

[0049] Example 1

[0050] This embodiment discloses a few-shot semantic segmentation method based on foreground enhancement and background diffusion contrast. The few-shot semantic segmentation method includes the following steps:

[0051] Step A: Divide the given semantic segmentation dataset into a training set and a test set. During training, each segment is sampled from the training set to obtain a support set and a query set.

[0052] Step B involves processing all supporting images in the support set to obtain the image background and image foreground of each supporting image. The image background is diffused, and the diffused image background is combined with the image foreground to obtain the background diffusion input set, which updates the support set. The updated support set and query set are then input into two backbone networks with shared parameters to extract common image features, resulting in the support feature set and query features.

[0053] Step C: The extracted support feature set and query features are input into the foreground enhancement module and background diffusion module in parallel to construct the foreground similarity map, the foreground enhancement feature with the same size as the support features, and the background diffusion weights, respectively.

[0054] Step D: Based on the background diffusion weight, the query features are reparameterized. The reparameterized query features, foreground similarity map, and foreground enhancement features are then fed into the feature fusion module to generate the final binary segmentation result.

[0055] To enhance the generalization ability of the model, the training of FSS models typically follows a meta-learning paradigm, using the fragment learning method within this paradigm. Given a semantic segmentation dataset D, whose semantic categories are represented by a set C, the dataset is divided into a training set D. train and test set D test The categories it contains are C train ={C1, C2, ..., C n} and C test ={C n+1 C n+2 C N}, During training, each segment will be generated from D. train A support set input is obtained by mid-sampling. and a query set input (Q={I q M q}), where K represents the number of samples in each segment; Let I and Iq represent the i-th supporting image and the query image, respectively, and their corresponding binary masks are respectively and M q The model can only obtain M during the training phase. q .

[0056] This embodiment employs a dual-branch structure that primarily includes two parallel modules: foreground enhancement and background diffusion. Before the support set S and query set Q enter this dual-branch module, they undergo the following information processing steps:

[0057] Step 1, for each input image in S Foreground binary mask The binary mask of its background is obtained through inversion.

[0058] Step 2, and Element-wise multiplication yields the image background. and image foreground Based on the diffusion process in the Diffusion model, respectively... The diffusion extends to rounds 25 and 50. According to the formula...

[0059]

[0060] right Including noise with varying degrees of diffusion, among which Given a scalar, Z is a random number that follows a standard Gaussian distribution.

[0061] Step 3: Combine the diffused background with the original foreground to obtain the background diffusion input set. Where k = 0 represents the original image. After this diffusion process, the input support set becomes

[0062] Step 4: Feed S and Q into a backbone network with shared parameters, extract common image features, and obtain a support feature set. and query feature F q .

[0063] The extracted features will be fed into the foreground enhancement module and the background diffusion module in parallel for further feature extraction and matching. The foreground enhancement module and the background diffusion module will be introduced in turn below.

[0064] The implementation process of the foreground enhancement module includes the following steps, and its output is a foreground similarity map. fg and foreground enhancement features

[0065] Step 1, The channel attention prototype is obtained by performing gated channel transformation and MAP operation respectively. and supporting future prototypes The Masked Average Pooling (MAP) operation is based on the following formula:

[0066]

[0067] Step 2, using With query feature F q Calculate the cosine similarity to obtain the foreground similarity map. fg .

[0068] Step 3, and Cascade and then with query feature F q Calculate the cosine similarity to obtain a double similarity map (Map). Dual By using Argmax to obtain the index of the element with the maximum cosine similarity to the query feature after concatenation of the prototype, an index map is obtained. index .

[0069] Step 4, based on the index map index Embed the corresponding supporting foreground prototype at each element position Or channel attention prototype Construct foreground enhancement features of the same size as the supporting features.

[0070] The implementation process of the background diffusion module includes the following steps, and its output is the background diffusion weight. bgd :

[0071] Step 1, Support Feature Sets After passing through a MAP operation and a fully connected layer, the corresponding background prototype set is obtained. Its MAP operation uses a support set background mask to filter out foreground information, resulting in a prototype that is only associated with the background.

[0072] Step 2: Feed the supporting features and background prototypes into the weighted interaction layer. For each supporting feature, assign weights using two background prototypes that do not correspond to it. The feature map with the new weights is then compared with the corresponding background mask, and a pairwise diffusion prototype set is obtained through a MAP operation.

[0073] Step 3, Each background diffusion prototype in the model is associated with the query feature F. q Calculate the cosine similarity and examine the information contained therein with F. q The relevance of information in the text. Due to... It contains relevant features supporting the background of the set, while F q The information in F is the foreground information of the query set. q China and Vietnam The more correlated the elements are, the weaker their relationship with the foreground. Therefore, this method designs a monotonically decreasing reparameterization module based on an exponential function to reassign values ​​to the cosine similarity calculation results. The output of this module, the background diffusion weight, is obtained by summing the reassigned similarity maps. bgd .

[0074] After the two parallel modules mentioned above, the background diffusion weight is first applied. bgd For query feature F q Perform feature reparameterization, and map the reparameterized query features with the foreground similarity map. fg Foreground enhancement features They are then fed into the feature fusion module to generate the final binary segmentation result.

[0075] Example 2

[0076] The following is combined Figures 1 to 3 The present invention provides a detailed description of the few-shot semantic segmentation method based on foreground enhancement and background diffusion contrast proposed in this invention:

[0077] This embodiment uses ResNet-50 as the backbone network. The backbone network is initialized using weights pre-trained on ImageNet, and the trainable parameters are frozen throughout to improve its generalization ability. The model uses the SGD optimizer with an initial learning rate of 0.001, momentum of 0.9, and weight decay rate of 0.0001. The datasets used are PASCAL-5i and COCO-20i. PASCAL-5i is derived from the PASCALVOC 2012 dataset and expanded with annotations using the SDS dataset. All 20 classes are evenly divided into 4 groups, each containing 5 classes. The COCO-20i dataset comes from MS COCO and has 80 classes, also divided into 4 groups, each containing 20 classes. Input images from PASCAL-5i are cropped to 473×473×3, and images from COCO-20i are cropped to 553×553×3. During training, the input image is processed by mirroring and random rotation from -10 degrees to 10 degrees.

[0078] Next, we will explain in detail using the example of sample size K=1. The support set input is S={I} s M s The query set input is Q = {I} q M q}

[0079] The first step involves performing background diffusion processing on the input image and then feeding it into the backbone network to extract features. Binary mask M s It contains three elements: 0, 1, and 255. 0 represents the background, 1 represents the foreground, and 255 represents irrelevant information. The inversion process of the foreground binary mask involves changing all 0 elements to 1 and all 1 elements to 0 to obtain the background binary mask. The diffusion process is as described above, in which... For a given scalar, α in this method iThis is an arithmetic sequence with the first term being 0.0002, the last term being 0.04, and 500 terms. This step yields the support feature set F. s ={F s0 F s25 F s50} and query feature F q .

[0080] The second step involves feeding the extracted features into the foreground enhancement module and the background diffusion module in parallel for further feature extraction and matching. In the foreground enhancement module, F... s0 In the gated channel transformation operations, L2 norm was used for normalization, which reduced the computational cost of the model to some extent. This resulted in the channel attention prototype P. gct and supporting prospective prototype P s0 Then, it needs to be compared with the query feature F. q Perform cosine similarity calculation. Let P... s0 With F q Taking the calculation as an example, the calculation is based on the following formula:

[0081]

[0082] Based on formula (3), the foreground similarity map (Map) is calculated. fg and double similarity map Dual Map Dual After obtaining the index through Argmax, it serves as the guide for prototype embedding. Following the ResNet-50 backbone network, taking a 473×473×3 PASCAL-5i input image as an example, F... q The size will be 60×60×256, and the size of the prototype generated by the MAP operation is 1×1×256. At this time, the index map (Map)... index The size is 60×60, and the index points to P. gct and P s0 One of them. Therefore, according to the index map Map index By embedding the prototype corresponding to the index value at each position, a prototype feature map with the same size of 60×60×256 can be obtained, which is one of the outputs of this module: foreground enhancement features.

[0083] In the background diffusion module, using F s0 This example illustrates the workflow of the weighted interaction layer. (Not related to F) s0 The two corresponding background prototypes are P sB25 and P sB50 F s0 First with P sB25Element-wise multiplication yields a reparameterized feature map; this process does not change F. s0 The size of the feature map is then used to obtain the background diffusion prototype P by performing a MAP operation. sB0D25 The same process applies to P. sB50 After one processing step, the background diffusion prototype P is obtained. sB0D50 F s25 With F s50 The process is the same as above, generating {P} through the weight interaction layer respectively. sB25D0 P sB25D50} and {P sB50D0 P sB50D25 The monotonically decreasing exponential function used in the monotonically decreasing reparameterized module based on the exponential function designed in this module is y = e^(-π / 2). -3x The values ​​are obtained from repeated experiments. This module generates the background diffusion weight. bgd .

[0084] The third step is to reparameterize the query feature Fq. 和 Foreground Similarity Map fg Foreground enhancement features The data is then fed into the feature fusion module, which uses a Pyramid Scene Parsing (PSP) network. This network can extract global and local contextual information at different scales, helping to enhance the model's understanding of the semantic content in the image. Four context scales are selected: [60, 30, 15, 8]. Based on the PSP module, binary segmentation results are generated.

[0085] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A few-sample semantic segmentation method based on foreground enhancement and background diffusion contrast, characterized in that, The few-shot semantic segmentation method includes the following steps: Step A: Divide the given semantic segmentation dataset into a training set and a test set. During training, each segment is sampled from the training set to obtain a support set and a query set. Step B involves processing all supporting images in the support set to obtain the image background and image foreground of each supporting image. The image background is diffused, and the diffused image background is combined with the image foreground to obtain the background diffusion input set, which updates the support set. The updated support set and query set are then input into two backbone networks with shared parameters to extract common image features, resulting in the support feature set and query features. Specifically, the following steps are performed on all supporting images in the support set to obtain the image background and image foreground of each supporting image. The image background is diffused, and the diffused image background is combined with the image foreground to obtain the background diffusion input set. The support set is then updated: Step B1, for each support image in the support set S Foreground binary mask The binary mask of its background is obtained through inversion. ; Step B2, the binary mask With supporting images Element-wise multiplication yields the supporting image background. and image foreground , = - Based on the diffusion process of the Diffusion model, the image background is divided into... Diffusion extends to rounds 25 and 50, targeting the image background. Add noise with varying degrees of diffusion: ； in Given a diffusion scalar, it is used to limit the complexity of the noise addition. The numbers are random and follow a standard Gaussian distribution. ; Step B3: Combine the diffused image background with the original image foreground to obtain the background diffusion input set. ,in Represents the original image. , After diffusion processing, the input support set is transformed into... ; Step B4, will and The data is fed into a backbone network with shared parameters, and common image features are extracted to obtain a support feature set. and query features It supports the generation of images from feature sets containing feature maps composed of the original background, a background diffused for 25 steps, and a background diffused for 50 steps. Step C: The extracted support feature set and query features are input into the foreground enhancement module and background diffusion module in parallel to construct the foreground similarity map, the foreground enhancement feature with the same size as the support features, and the background diffusion weights, respectively. Specifically, the foreground similarity map and foreground enhancement features of the same size as the supporting features are constructed according to the following steps: Step C11, The channel attention prototype is obtained by performing gating channel transformation and MAP operation respectively. and supporting future prototypes The masked average pooling (MAP) operation is performed according to the following formula: ； Where (x,y) represents the position of an element on the feature map, and h and w represent the maximum values ​​of the length and width of the feature map; Step C12, utilizing the supporting foreground prototype With query features Calculate the cosine similarity to obtain the foreground similarity map. ; Step C13 will support the foreground prototype. With channel attention prototype Cascade and then combine with query features Calculate the cosine similarity to obtain a double similarity map. The index map is obtained by using the Argmax function to find the index of the element with the maximum cosine similarity to the query feature for the concatenated prototype. ; Step C14, according to the index diagram Embed the corresponding supporting foreground prototype at each element position Or channel attention prototype Construct foreground enhancement features of the same size as the supporting features. ; Further, the background diffusion weights are constructed using the following steps: Step C21, Support Feature Set After passing through the MAP operation layer and the fully connected layer respectively, the corresponding background prototype set is obtained. In the MAP operation, a support set background mask is used to filter out foreground information, resulting in a background prototype that is only associated with the background. Step C22: The supporting features and background prototypes are fed into the weighted interaction layer. For each supporting feature, weights are assigned to two background prototypes that do not correspond to it. The feature map with the new weights is then compared with the corresponding background mask, and a pair of background diffusion prototype sets are obtained through a MAP operation. For each prototype element in this set, with For example, this prototype represents the... right A new background diffusion model was generated by reparameterizing and using MAP. Step C23, set the background diffusion prototype. Each background diffusion prototype in the query features is associated with a different query feature. Calculate cosine similarity to examine the information and query features contained therein. To determine the relevance of information, a monotonically decreasing reparameterization module based on an exponential function is used to re-parameterize the cosine similarity calculation results. The re-parameterized similarity maps are then summed to obtain the background diffusion weight. ; Step D: Based on the background diffusion weight, the query features are reparameterized. The reparameterized query features, foreground similarity map, and foreground enhancement features are then fed into the feature fusion module to generate the final binary segmentation result.

2. The few-shot semantic segmentation method based on foreground enhancement and background diffusion contrast according to claim 1, characterized in that, In step A, the given semantic segmentation dataset is divided into a training set and a test set. During training, the process of sampling each segment from the training set to obtain the support set and query set includes the following steps: Given a semantic segmentation dataset D, the semantic categories contained in the semantic segmentation dataset D are composed of a set express, This represents the i-th semantic category, and the dataset is divided into training and training sets. and test set training set and test set The included categories are as follows and , ; During training, each segment is taken from the training set. Mid-sampling yields support set and query set , where K represents the number of samples in each segment; and This represents the i-th supporting image and the query image, with corresponding binary masks of respectively. and .

3. A few-sample semantic segmentation apparatus based on foreground enhancement and background diffusion contrast, according to the method described in any one of claims 1-2, characterized in that, The few-shot semantic segmentation device includes a dataset input module, a diffusion module, two backbone networks with shared parameters, a foreground enhancement module, a background diffusion module, a feature fusion module, and a prediction result output module. The dataset input module is used to divide a given semantic segmentation dataset into a training set and a test set. During training, each segment is sampled from the training set to obtain a support set and a query set. The diffusion module is used to process all supporting images in the support set to obtain the image background and image foreground of each supporting image, diffuse the image background, combine the diffused image background with the image foreground to obtain the background diffusion input set, and update the support set. The two backbone networks sharing the parameters process the updated support set and query set respectively, extract common image features, and obtain the support feature set and query features; The foreground enhancement module processes the extracted support feature set and query features to construct a foreground similarity map and a foreground enhancement feature of the same size as the support features; The background diffusion module processes the extracted support feature set and query features to construct the background diffusion weights; The feature fusion module reparameterizes the query features based on the background diffusion weight, processes the reparameterized query features, foreground similarity map, and foreground enhancement features to generate the final binary segmentation result, which is output by the prediction result output module.

Images