Small sample image semantic segmentation method and device based on prior information optimization and storage medium

Abstract

The invention discloses a small sample image semantic segmentation method and device based on prior information optimization and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: determining a multi-scale support middle-layer feature map, a multi-scale support high-layer feature map, a multi-scale query middle-layer feature map and a multi-scale query high-layer feature map through a double-branch encoder; determining foreground positioning prior information according to the multi-scale support high-level feature map and the multi-scale query high-level feature map; multi-modal prior information is determined based on multi-scale query of the high-level feature map and the scene text features; performing fusion processing on the query feature map, the foreground positioning prior information, the multi-modal prior information and the support category prototype to obtain an enhanced query feature map; and performing feature interaction processing on the enhanced query feature map and the support feature map to obtain an image semantic segmentation result of the query image. By using the technical scheme provided by the invention, the interference of a complex background in the image can be suppressed, and the semantic segmentation precision of the image is improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a method, apparatus and storage medium for semantic segmentation of few-sample images based on prior information optimization. Background Technology

[0002] Few-shot semantic segmentation is a fundamental task in computer vision, aiming to assign a specific semantic category label to each pixel in an image. However, models trained under full supervision using large-scale pixel-level labeled data often struggle to be effectively deployed in many real-world applications due to the scarcity of labeled data. Few-shot semantic segmentation technology aims to solve this problem, its core being to achieve pixel-level recognition and segmentation of specific target categories in a query image with only a small number of labeled samples.

[0003] Currently, mainstream methods in the field of few-shot semantic segmentation typically employ a metric learning-based framework. These methods first utilize a pre-trained visual encoder (such as a deep convolutional neural network) to extract high-level feature maps from the support image and the query image, respectively. Then, through a pixel-level matching mechanism, they calculate the similarity between the features of the query image and the supporting foreground prototypes, thereby activating target regions belonging to the same category in the query image and providing crucial segmentation prior information for the subsequent decoder.

[0004] Despite the initial progress made by such methods, their performance is still constrained by two inherent bottlenecks: (1) Existing methods heavily rely on high-level features extracted from deep convolutional networks. Since such networks often have a large receptive field in deep layers, the feature maps they generate suffer from information confusion in the spatial dimension, resulting in a certain coupling between the target foreground features and complex background features. This ambiguity in features will seriously interfere with the accuracy of the pixel-level matching process, especially under complex conditions such as variable target shapes or low contrast with the background. Effective foreground information in the supporting image is difficult to be accurately transmitted to the query image, resulting in a significant decrease in the segmentation performance of the model. (2) The prior information relied upon by existing methods usually comes from only a single visual modality, and the features extracted by deep networks themselves have obvious class bias. This coarse-grained prior information lacks clear, high-level semantic guidance, resulting in poor generalization ability to new categories and difficulty in accurately and stably identifying the true target foreground in complex scenes.

[0005] In summary, background interference during feature extraction, as well as the limitations and coarseness of prior information representation, jointly restrict the accuracy and robustness of existing few-sample semantic segmentation models in real-world complex scenarios. Therefore, a more reliable solution is needed. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, apparatus and storage medium for small sample image semantic segmentation based on prior information optimization, which can suppress the interference of complex backgrounds in images and improve the accuracy and robustness of image semantic segmentation.

[0007] To achieve the above objectives, the present invention employs the following technical solution:

[0008] On one hand, the present invention provides a few-sample image semantic segmentation method optimized based on prior information, the method comprising:

[0009] The support image and the query image are input into a dual-branch encoder for feature extraction processing to obtain the multi-scale support mid-level feature map and the multi-scale support high-level feature map of the support image, as well as the multi-scale query mid-level feature map and the multi-scale query high-level feature map of the query image.

[0010] Feature extraction is performed on the preset foreground text prompts and preset background text prompts to obtain scene text features;

[0011] Based on the multi-scale support high-level feature map and the multi-scale query high-level feature map, the foreground localization prior information of the query image is determined;

[0012] Based on the multi-scale query high-level feature map and the scene text features, determine multimodal prior information;

[0013] The query feature map, the foreground localization prior information, the multimodal prior information, and the support category prototype are fused to obtain an enhanced query feature map. The query feature map is determined based on the multi-scale query mid-layer feature map, and the support category prototype is determined based on the multi-scale support mid-layer feature map.

[0014] The enhanced query feature map and the support feature map are subjected to feature interaction processing to obtain the image semantic segmentation result of the query image. The support feature map is determined based on the multi-scale support mid-layer feature map.

[0015] In some possible implementations, the dual-branch encoder includes a mid-level feature extraction encoder and a high-level feature extraction encoder; the step of inputting the support image and the query image into the dual-branch encoder for feature extraction processing to obtain multi-scale support mid-level feature maps and multi-scale support high-level feature maps of the support image, and multi-scale query mid-level feature maps and multi-scale query high-level feature maps of the query image, includes:

[0016] The mid-level visual feature extraction encoder is used to extract mid-level visual features from the support image and the query image to obtain the multi-scale support mid-level feature map and the multi-scale query mid-level feature map.

[0017] The high-level feature extraction encoder extracts high-level visual features from the support image and the query image to obtain the multi-scale support high-level feature map and the multi-scale query high-level feature map.

[0018] In some possible implementations, the mid-level feature extraction encoder is a ResNet50 encoder, and the high-level feature extraction encoder is a CLIP visual encoder; the multi-scale supporting mid-level feature map includes supporting mid-level feature maps at multiple scales, the multi-scale query mid-level feature map includes query mid-level feature maps at multiple scales, the multi-scale supporting high-level feature map includes supporting high-level feature maps at multiple scales, and the multi-scale query high-level feature map includes query high-level feature maps at multiple scales.

[0019] The step of inputting the support image and the query image into a dual-branch encoder for feature extraction processing to obtain multi-scale support mid-level feature maps and multi-scale support high-level feature maps of the support image, as well as multi-scale query mid-level feature maps and multi-scale query high-level feature maps of the query image, includes:

[0020] The ResNet50 encoder is used to extract mid-level visual features from the support image, and the outputs of the second and third residual blocks in the ResNet50 encoder are determined to be mid-level support feature maps of multiple scales.

[0021] The query image is subjected to mid-level visual feature extraction using the ResNet50 encoder, and the outputs of the second and third residual blocks in the ResNet50 encoder are determined to be mid-level feature maps of the query at multiple scales.

[0022] The CLIP visual encoder is used to extract high-level visual features from the supporting image, and the outputs of the 10th and 11th Transformer layers in the CLIP visual encoder are determined to be multi-scale support high-level feature maps.

[0023] The CLIP visual encoder extracts high-level visual features from the query image, and determines that the outputs of the 10th and 11th Transformer layers in the CLIP visual encoder are query high-level feature maps of multiple scales.

[0024] In some possible implementations, determining the foreground localization prior information of the query image based on the multi-scale support high-level feature map and the multi-scale query high-level feature map includes:

[0025] Weighted global average pooling is performed on the support high-level feature map at each scale of the multi-scale support high-level feature map to determine the foreground prototype and background prototype at each scale.

[0026] Based on the query high-level feature map at each scale in the multi-scale query high-level feature map and the foreground prototype, determine the foreground similarity map at each scale;

[0027] Based on the query high-level feature map at each scale and the background prototype, determine the background similarity map at each scale;

[0028] The foreground localization prior information is determined based on the foreground similarity map and background similarity map at each scale.

[0029] In some possible implementations, determining the foreground localization prior information based on the foreground similarity map and background similarity map at each scale includes:

[0030] The maximum foreground similarity in the foreground similarity map at each scale is determined as the foreground confidence at each scale;

[0031] Compare the foreground confidence at each scale, determine the foreground similarity map corresponding to the scale with the highest foreground confidence as the target foreground similarity map, and determine the background similarity map corresponding to the scale with the highest foreground confidence as the target background similarity map;

[0032] The target foreground similarity map and the target background similarity map are normalized and reshaped to obtain the foreground prior mask and the background prior mask, respectively.

[0033] The foreground positioning prior information is determined based on the foreground prior mask and the background prior mask.

[0034] In some possible implementations, the scene text features include foreground text features; determining multimodal prior information based on the multi-scale query high-level feature map and the scene text features includes:

[0035] Extract the feature activation map of the query image at the target feature layer;

[0036] Determine the similarity map between the multi-scale query high-level feature map and the foreground text features;

[0037] Determine the gradient tensor of the similarity map with respect to the feature activation map;

[0038] Global average pooling is performed on the gradient tensor to determine the channel weight of each channel in the feature activation graph;

[0039] The channel weights are used to weight each channel of the query high-level feature map of the target feature layer to obtain a class activation mapping map.

[0040] The multimodal prior information is determined based on the class activation map.

[0041] In some possible implementations, the fusion of the query feature map, the foreground localization prior information, the multimodal prior information, and the supporting category prototype to obtain the enhanced query feature map includes:

[0042] The support mid-level feature maps of multiple scales in the multi-scale support mid-level feature map and the query mid-level feature map in the multi-scale query mid-level feature map are concatenated along the channel dimension to obtain the support mid-level concatenated feature map and the query mid-level concatenated feature map.

[0043] The support mid-level spliced ​​feature map and the query mid-level spliced ​​feature map are respectively subjected to dimensionality reduction processing to obtain the support feature map and the query feature map;

[0044] The support feature map is weighted and global average pooled using the support mask corresponding to the support image to generate the support category prototype;

[0045] The query feature map, the supporting category prototype, the foreground localization prior information, and the multimodal prior information are fused along the channel dimension to obtain the enhanced query feature map.

[0046] In some possible implementations, the image semantic segmentation result includes a query segmentation mask; the step of performing feature interaction processing on the enhanced query feature map and the support feature map to obtain the image semantic segmentation result of the query image includes:

[0047] The support feature map and the enhanced query feature map are input into the target decoder. The support feature map is processed through two independent linear projection layers to obtain the attention key vector and the attention value vector. The enhanced query feature map is processed through one linear projection layer to obtain the attention query vector.

[0048] The cross-attention weight matrix is ​​determined based on the first dot product of the attention query vector and the attention key vector;

[0049] Determine the second dot product between the cross-attention weight matrix and the attention value vector;

[0050] The query segmentation mask is obtained by processing the second dot product using the classification header.

[0051] On the other hand, a few-sample image semantic segmentation device based on prior information optimization is provided, the device comprising:

[0052] The feature extraction module is used to input the support image and the query image into the dual-branch encoder for feature extraction processing, to obtain the multi-scale support mid-level feature map and the multi-scale support high-level feature map of the support image, as well as the multi-scale query mid-level feature map and the multi-scale query high-level feature map of the query image; and to extract features from the preset foreground text prompt and the preset background text prompt to obtain scene text features.

[0053] The foreground localization prior information optimization module is used to determine the foreground localization prior information of the query image based on the multi-scale support high-level feature map and the multi-scale query high-level feature map.

[0054] The multimodal prior information determination module is used to determine multimodal prior information based on the multi-scale query high-level feature map and the scene text features;

[0055] The feature enhancement module is used to fuse the query feature map, the foreground localization prior information, the multimodal prior information, and the support category prototype to obtain an enhanced query feature map. The query feature map is determined based on the multi-scale query mid-layer feature map, and the support category prototype is determined based on the multi-scale support mid-layer feature map.

[0056] The semantic segmentation determination module is used to perform feature interaction processing on the enhanced query feature map and the support feature map to obtain the image semantic segmentation result of the query image, wherein the support feature map is determined based on the multi-scale support mid-layer feature map.

[0057] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction and at least one program are stored therein, the at least one instruction and the at least one program being loaded and executed by a processor to implement the few-sample image semantic segmentation method based on prior information optimization as described above.

[0058] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0059] In this invention, by inputting the support image and the query image into a dual-branch encoder for feature extraction, multi-scale support mid-level feature maps and multi-scale support high-level feature maps of the support image, as well as multi-scale query mid-level feature maps and multi-scale query high-level feature maps of the query image, can be obtained. This allows for the extraction of mid-level features rich in texture details and high-level features rich in semantics from the support and query images. Feature extraction is performed on preset foreground text prompts and preset background text prompts to obtain scene text features, which can extract text semantic features. Based on the multi-scale support high-level feature maps and multi-scale query high-level feature maps, prior information for foreground localization of the query image is determined to optimize foreground region localization and improve the accuracy of visual prior information. Based on the multi-scale query high-level feature maps and scene text features, multi-modal prior information is determined, which can generate multi-modal prior information for visual-text semantic alignment. This effectively suppresses background interference on semantic segmentation and reduces the missegmentation rate of visually similar interfering objects, thereby improving... To improve segmentation accuracy, the query feature map, foreground localization prior information, multimodal prior information, and supporting category prototypes are fused to obtain an enhanced query feature map. The query feature map is determined based on a multi-scale query mid-level feature map, and the supporting category prototype is determined based on a multi-scale support mid-level feature map. This constructs a fused and optimized query feature map of "support prototype, foreground localization prior, and multimodal prior," which can provide target feature patterns, spatial location guidance, and semantic alignment information, thereby achieving multi-dimensional semantic enhancement with complementary functions and effectively improving the ability to discriminate foreground targets in small sample scenarios. The enhanced query feature map and supporting feature map are then subjected to feature interaction processing to obtain the image semantic segmentation result of the query image. The supporting feature map is determined based on a multi-scale support mid-level feature map, which can effectively preserve target details while improving the accuracy of foreground region localization and segmentation. Therefore, the above small sample image semantic segmentation method can improve the accuracy and robustness of image semantic segmentation with a small number of labeled samples. Attached Figure Description

[0060] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart illustrating a few-sample image semantic segmentation method based on prior information optimization provided in an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of a few-sample image semantic segmentation device based on prior information optimization provided in an embodiment of the present invention. Detailed Implementation

[0063] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0064] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0065] In this embodiment of the invention, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0066] Various exemplary embodiments, features, and aspects of the present invention will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0067] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0068] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0069] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without certain specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order to highlight the spirit of the invention.

[0070] Figure 1 This is a flowchart illustrating a few-sample image semantic segmentation method based on prior information optimization according to an embodiment of the present invention. This specification provides the method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server products, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 1 As shown, the above method may include:

[0071] S101: Input the support image and query image into the dual-branch encoder for feature extraction processing to obtain the multi-scale support mid-level feature map and multi-scale support high-level feature map of the support image, as well as the multi-scale query mid-level feature map and multi-scale query high-level feature map of the query image.

[0072] In one specific embodiment, the supporting image can be a labeled reference image, and the query image can be an unlabeled image to be segmented. Optionally, the number of supporting images and query images can be at least one. A dual-branch encoder can be used to perform visual feature extraction processing on the supporting and query images. Specifically, the dual-branch encoder can be used to extract mid-level and high-level visual features from the supporting and query images. Multi-scale supporting mid-level feature maps can include multiple scales of supporting mid-level feature maps corresponding to the supporting images, and these mid-level feature maps can be the mid-level visual feature maps corresponding to the supporting images. Multi-scale supporting high-level feature maps can include multiple scales of supporting high-level feature maps corresponding to the supporting images, and these high-level feature maps can be the high-level visual feature maps corresponding to the supporting images. Multi-scale query mid-level feature maps can include multiple scales of query mid-level feature maps corresponding to the query images, and these high-level feature maps can be the high-level visual feature maps corresponding to the query images.

[0073] In an optional embodiment, the dual-branch encoder described above may include a mid-level feature extraction encoder and a high-level feature extraction encoder; the above-mentioned inputting the support image and the query image into the dual-branch encoder for feature extraction processing to obtain multi-scale support mid-level feature maps and multi-scale support high-level feature maps of the support image, and multi-scale query mid-level feature maps and multi-scale query high-level feature maps of the query image may include:

[0074] A mid-level visual feature extraction encoder is used to extract mid-level visual features from the support image and the query image to obtain multi-scale support mid-level feature maps and multi-scale query mid-level feature maps.

[0075] High-level visual features are extracted from the support image and the query image by a high-level feature extraction encoder, resulting in multi-scale support high-level feature maps and multi-scale query high-level feature maps.

[0076] In one specific embodiment, the mid-level feature extraction encoder is used to extract mid-level visual features of the support image and the query image, and the high-level feature extraction encoder is used to extract high-level visual features of the support image and the query image. Optionally, the mid-level feature extraction encoder and the high-level feature extraction encoder can be configured according to the actual application; preferably, the mid-level feature extraction encoder can be a ResNet50 encoder, and the high-level feature extraction encoder can be a CLIP (Contrastive Language-Image Pre-training) visual encoder based on ViT-B-16.

[0077] In an optional embodiment, the aforementioned mid-level feature extraction encoder is a ResNet50 encoder, and the high-level feature extraction encoder is a CLIP visual encoder; the aforementioned inputting the support image and query image into the dual-branch encoder for feature extraction processing to obtain multi-scale support mid-level feature maps and multi-scale support high-level feature maps of the support image, as well as multi-scale query mid-level feature maps and multi-scale query high-level feature maps of the query image may include:

[0078] By extracting mid-level visual features from the support images using the ResNet50 encoder, the outputs of the second and third residual blocks in the ResNet50 encoder are determined to be mid-level support feature maps at multiple scales.

[0079] The query image is subjected to mid-level visual feature extraction using a ResNet50 encoder. The outputs of the second and third residual blocks in the ResNet50 encoder are determined to be mid-level feature maps of the query at multiple scales.

[0080] High-level visual features were extracted from the supporting images using the CLIP visual encoder, and the outputs of the 10th and 11th Transformer layers in the CLIP visual encoder were determined to be multi-scale support high-level feature maps.

[0081] High-level visual features of the query image are extracted using the CLIP visual encoder. The outputs of the 10th and 11th Transformer layers in the CLIP visual encoder are determined to be high-level feature maps of the query at multiple scales.

[0082] In one specific embodiment, the outputs of different residual blocks of the ResNet50 encoder can be determined as intermediate-level visual feature maps. Optionally, the different residual blocks can be determined in conjunction with the actual application. Preferably, the outputs of the second and third residual blocks of the ResNet50 encoder can be determined as intermediate-level visual feature maps of multiple scales, thereby extracting intermediate-level feature maps with a certain resolution and containing texture details. The outputs of different Transformer layers of the CLIP visual encoder can be determined as high-level visual feature maps. Optionally, the different Transformer layers can be determined in conjunction with the actual application. Preferably, the outputs of the 10th and 11th Transformer layers of the CLIP visual encoder can be determined as high-level visual feature maps of multiple scales, thereby extracting high-level feature maps containing rich semantics.

[0083] In one specific embodiment, a mid-level visual feature extraction is performed on the support image using a ResNet50 encoder to determine the outputs of the second and third residual blocks in the ResNet50 encoder. , Support for mid-level feature maps at multiple scales;

[0084] The query image is subjected to mid-level visual feature extraction using a ResNet50 encoder, and the outputs of the second and third residual blocks in the ResNet50 encoder are determined. , For query mid-level feature maps at multiple scales;

[0085] High-level visual feature extraction is performed on the supporting images using the CLIP visual encoder to determine the outputs of the 10th and 11th Transformer layers in the CLIP visual encoder. , To support high-level feature maps at multiple scales;

[0086] High-level visual feature extraction is performed on the query image using the CLIP visual encoder to determine the outputs of the 10th and 11th Transformer layers in the CLIP visual encoder. , This provides high-level feature maps for queries at multiple scales.

[0087] S102: Extract features from the preset foreground text prompts and preset background text prompts to obtain scene text features;

[0088] In one specific embodiment, the preset foreground text prompt can represent the target segmentation category of the current image segmentation task; the preset background text prompt can represent the non-target part in the image; optionally, the preset foreground text prompt and the preset background text prompt can be determined in combination with the image segmentation task.

[0089] In one specific embodiment, the CLIP text encoder extracts features from the preset foreground text prompts and preset background text prompts to obtain scene text features, which include foreground text features and background text features.

[0090] In the above embodiments, the mid-level visual features containing spatial details are extracted by the ResNet50 encoder, and the high-level visual features containing rich semantics are extracted by the CLIP visual encoder and the scene text features are extracted by the CLIP text encoder. This can achieve multi-dimensional semantic enhancement with complementary functions, thereby improving the ability to distinguish foreground targets in small sample scenarios and thus improving segmentation accuracy.

[0091] S103: Determine the foreground localization prior information of the query image based on the multi-scale support high-level feature map and the multi-scale query high-level feature map;

[0092] In one specific embodiment, foreground localization prior information can be used to locate the position of a target in an image.

[0093] In an optional embodiment, the prior information for foreground localization of the query image based on the multi-scale support high-level feature map and the multi-scale query high-level feature map may include:

[0094] Weighted global average pooling is performed on the support high-level feature map at each scale of the multi-scale support high-level feature map to determine the foreground prototype and background prototype at each scale.

[0095] Based on the query high-level feature map and foreground prototype at each scale in the multi-scale query high-level feature map, determine the foreground similarity map at each scale.

[0096] Based on the query high-level feature map and background prototype at each scale, determine the background similarity map at each scale;

[0097] Based on the foreground similarity map and background similarity map at each scale, the prior information for foreground localization is determined.

[0098] In a specific embodiment, the above-mentioned weighted global average pooling of the support high-level feature map at each scale of the multi-scale support high-level feature map to determine the foreground prototype and background prototype at each scale may include: using the ground truth mask M of the support image to perform weighted global average pooling on the support high-level feature map at each scale to obtain the foreground prototype at each scale; and using the background mask I-M of the support image to perform weighted global average pooling on the support high-level feature map at each scale to obtain the background prototype at each scale.

[0099] Specifically, for Perform weighted global average pooling to determine the foreground prototype. It can be represented as follows:

[0100] ;

[0101] in, Let (i,j) represent the foreground prototype corresponding to the 10th Transformer layer, and (i,j) represent its spatial position in the support high-level feature map, i.e., in the i-th row and j-th column of the support high-level feature map. This represents the truth mask corresponding to the spatial location (i,j). H represents the supporting high-level feature at spatial position (i,j) in the supporting high-level feature map corresponding to the 10th Transformer layer; H represents the height of the supporting high-level feature map, and W represents the width of the supporting high-level feature map.

[0102] In one specific embodiment, the foreground similarity map can characterize the similarity between each pixel and the foreground target, reflecting whether each pixel resembles the foreground target; the background similarity can characterize the similarity between each pixel and the background non-target. The query high-level feature map at each scale can be flattened in the spatial dimension, and the cosine similarity can be determined with the foreground prototype and background prototype at the corresponding scale, respectively, to obtain the foreground similarity map and background similarity map at each scale.

[0103] Specifically, determine and The foreground similarity graph between them can be represented by the following formula:

[0104] ;

[0105] in, It can represent the foreground similarity at spatial location (i,j). You can query the high-level features corresponding to the spatial position (i,j) in the high-level feature map corresponding to the 10th Transformer layer.

[0106] In an optional embodiment, determining the foreground localization prior information based on the foreground similarity map and background similarity map at each scale may include:

[0107] The maximum foreground similarity in the foreground similarity map at each scale is determined as the foreground confidence at each scale;

[0108] Compare the foreground confidence at each scale, and determine the foreground similarity map corresponding to the scale with the highest foreground confidence as the target foreground similarity map, and determine the background similarity map corresponding to the scale with the highest foreground confidence as the target background similarity map;

[0109] The target foreground similarity map and the target background similarity map are normalized and reshaped to obtain the foreground prior mask and the background prior mask, respectively.

[0110] Based on the foreground prior mask and the background prior mask, determine the foreground localization prior information.

[0111] In a specific embodiment, the maximum foreground similarity in the foreground similarity map at each scale is determined, and the maximum foreground similarity is determined as the foreground confidence at the corresponding scale. The foreground confidence at each scale is compared, and the foreground similarity map corresponding to the scale with the highest foreground confidence is taken as the target foreground similarity map, and the background similarity map corresponding to the scale with the highest foreground confidence is determined as the target background similarity map. Then, the target foreground similarity map and the target background similarity map are normalized and reshaped respectively to obtain the foreground prior mask and the background prior mask. Optionally, the foreground prior mask can represent the similarity between the query region and the foreground features of the supporting image, and the background prior mask can represent the similarity between the query region and the background features of the supporting image. Optionally, the foreground localization prior information is a foreground localization prior mask. The above determination of the foreground localization prior information based on the foreground prior mask and the background prior mask may include: subtracting the foreground prior mask and the background prior mask pixel by pixel to obtain a foreground-background difference map; truncating the foreground-background difference map to obtain the foreground localization prior mask; optionally, the foreground-background difference map can be truncated by setting the negative values ​​in the foreground-background difference map to zero, while retaining the positive values.

[0112] In the above embodiments, the optimized visual prior mask for foreground localization of the query image can be obtained through the above method, thereby optimizing the foreground region localization of the query image, improving the accuracy of visual prior information, and thus improving the accuracy and reliability of image semantic segmentation.

[0113] S104: Determine multimodal prior information based on multi-scale query high-level feature maps and scene text features;

[0114] In an optional embodiment, the aforementioned scene text features include foreground text features; the determination of multimodal prior information based on multi-scale querying of high-level feature maps and scene text features may include:

[0115] Extract the feature activation map of the query image at the target feature layer;

[0116] Determine the similarity map between the high-level feature map of the multi-scale query and the foreground text features;

[0117] Determine the gradient tensor of the similarity map to the feature activation map;

[0118] Global average pooling is performed on the gradient tensor to determine the channel weight of each channel in the feature activation graph.

[0119] By using channel weights, each channel of the query high-level feature map of the target feature layer is weighted to obtain a class activation map.

[0120] Based on the class activation map, determine the multimodal prior information.

[0121] In a specific embodiment, the target feature layer can be a deep feature layer that balances semantic features and spatial information. Specifically, the 11th Transformer layer of the CLIP visual encoder can be used as the target feature layer. Optionally, each channel of the query high-level feature map of the target feature layer is weighted using channel weights, and positive response regions can be preserved using the ReLU activation function to generate a class activation map. Optionally, the multimodal information can be a multimodal prior mask. Determining the multimodal prior information based on the class activation map can include: normalizing the class activation map and mapping its numerical range to [0,1], and using bilinear interpolation to restore its spatial size to be consistent with the size of the mid-level visual feature map of the query image, thus obtaining the multimodal prior mask. The multimodal prior mask can reflect the activation regions in the query image that are highly related to the foreground semantics, and can provide semantic alignment guidance information for subsequent feature fusion.

[0122] In the above embodiments, the gradient response of visual features relative to text features is determined based on the query high-level feature map and text features, thereby determining the multimodal prior information for visual-text semantic alignment. The dual-path prior collaborative constraint can make up for the deficiencies of traditional single-modal prior information, reduce the missegmentation rate of visually similar interference objects, and effectively suppress the interference of background on small sample semantic segmentation, thereby improving segmentation accuracy and reliability.

[0123] S105: The query feature map, foreground localization prior information, multimodal prior information, and supporting category prototypes are fused to obtain the enhanced query feature map;

[0124] In one specific embodiment, the query feature map can be determined based on the multi-scale query mid-level feature map; the support category prototype can be determined based on the multi-scale support mid-level feature map.

[0125] In an optional embodiment, the above-described fusion of the query feature map, foreground localization prior information, multimodal prior information, and supporting category prototypes to obtain the enhanced query feature map may include:

[0126] The support mid-level feature maps of multiple scales in the multi-scale support mid-level feature map and the query mid-level feature map in the multi-scale query mid-level feature map are concatenated along the channel dimension to obtain the support mid-level concatenated feature map and the query mid-level concatenated feature map.

[0127] Dimensionality reduction is performed on the support mid-level splicing feature map and the query mid-level splicing feature map respectively to obtain the support feature map and the query feature map;

[0128] We use the support mask corresponding to the support image to perform weighted global average pooling on the support feature map to generate support category prototypes.

[0129] The query feature map, supporting category prototypes, foreground localization prior information, and multimodal prior information are fused along the channel dimension to obtain the enhanced query feature map.

[0130] In a specific embodiment, the support mid-layer concatenated feature map and the query mid-layer concatenated feature map can be dimensionality-reduced using a 3×3 convolution operation to obtain the support feature map and the query feature map. Optionally, the query feature map, support category prototypes, foreground localization prior information, and multimodal prior information can be fused along the channel dimension to obtain the enhanced query feature map, as shown in the following formula:

[0131] ;

[0132] in, This represents an enhanced query feature map. This indicates support for category prototypes. Indicates query feature map, This represents prior information for foreground localization. Represents multimodal prior information, This indicates splicing along the channel dimension. The function is used to fuse and adjust the number of channels after splicing multi-source information; it is optional. The function can consist of a 1×1 convolutional layer and a ReLU activation function.

[0133] In the above embodiments, a fusion-optimized prior information that "supports prototype, foreground localization prior and multimodal prior" is constructed, which can provide target feature patterns, spatial location guidance and semantic alignment information respectively, thereby achieving multi-dimensional semantic enhancement with complementary functions, and thus improving the accuracy of image semantic segmentation.

[0134] S106: Perform feature interaction processing on the enhanced query feature map and the support feature map to obtain the image semantic segmentation result of the query image.

[0135] In one specific embodiment, the support feature map can be determined by a multi-scale support mid-level feature map. The enhanced query feature map and the support feature map can be input to the target decoder, where feature interaction and refinement are performed through a cross-attention mechanism to obtain the image semantic segmentation result. Optionally, the image semantic segmentation result can be a binary image of the same size as the query image.

[0136] In an optional embodiment, the above image semantic segmentation result includes a query segmentation mask; the enhanced query feature map and the support feature map are subjected to feature interaction processing to obtain the image semantic segmentation result of the query image, including:

[0137] The support feature map and the enhanced query feature map are input into the target decoder. The support feature map is processed through two independent linear projection layers to obtain the attention key vector and the attention value vector. The enhanced query feature map is processed through one linear projection layer to obtain the attention query vector.

[0138] The cross-attention weight matrix is ​​determined based on the first dot product of the attention query vector and the attention key vector;

[0139] Determine the second dot product between the cross-attention weight matrix and the attention value vector;

[0140] The query segmentation mask is obtained by processing the second dot product using the classification header.

[0141] In a specific embodiment, the target decoder can be configured according to the actual application; specifically, the target decoder can be a Transformer decoder. Determining the cross-attention weight matrix based on the first dot product of the attention query vector and the attention key vector can include: adjusting the first dot product using a scaling factor and normalizing it using the Softmax function to obtain the cross-attention matrix; optionally, the cross-attention matrix can represent the semantic similarity between each position in the enhanced query feature map and all positions in the supporting feature map. The query segmentation mask can be a pixel-level segmentation mask of the query image, where the value of each pixel indicates whether each pixel in the query image belongs to the foreground or the background; generally, the foreground can be represented by 1, and the background by 0.

[0142] In the above embodiments, based on the Transformer decoder, deep interaction and contextual reasoning can be performed on the enhanced query features that integrate multi-source priors. This can improve the accuracy of foreground region localization and segmentation while effectively preserving target details, thereby improving the accuracy and reliability of image semantic segmentation.

[0143] As can be seen from the technical solutions provided in the embodiments of this specification above, this specification inputs the supporting image and the query image into a dual-branch encoder for feature extraction processing, obtaining multi-scale support mid-level feature maps and multi-scale support high-level feature maps of the supporting image, as well as multi-scale query mid-level feature maps and multi-scale query high-level feature maps of the query image. This allows for the extraction of mid-level features rich in texture details and high-level features rich in semantics from the supporting image and the query image. Feature extraction is performed on the preset foreground text prompts and preset background text prompts to obtain scene text features, from which text semantic features can be extracted. Based on the multi-scale support high-level feature maps and multi-scale query high-level feature maps, the foreground localization prior information of the query image is determined to optimize the foreground region localization of the query image and improve the accuracy of visual prior information. Based on the multi-scale query high-level feature maps and scene text features, multi-modal prior information is determined, which can generate visual-text semantically aligned multi-modal prior information, thereby effectively suppressing background interference on semantic segmentation and reducing the missegmentation rate of visually similar interfering objects, thus improving segmentation accuracy. Finally, the query feature maps, foreground localization prior information, multi-modal prior information, and supporting category prototypes are fused. The method yields an enhanced query feature map, which is determined based on a multi-scale query mid-level feature map. The supporting category prototype is determined based on a multi-scale supporting mid-level feature map. This constructs a fusion-optimized query feature map that integrates supporting prototypes, foreground localization priors, and multi-modal priors. This provides target feature patterns, spatial location guidance, and semantic alignment information, achieving complementary multi-dimensional semantic enhancement and effectively improving the ability to discriminate foreground targets in small-sample scenarios. The enhanced query feature map and supporting feature map are then processed through feature interaction to obtain the image semantic segmentation result of the query image. The supporting feature map is determined based on a multi-scale supporting mid-level feature map, which effectively preserves target details while improving the accuracy of foreground region localization and segmentation. Therefore, this small-sample image semantic segmentation method can improve the accuracy, reliability, and robustness of image semantic segmentation with a small number of labeled samples, significantly reducing the dependence on large-scale labeled data and the high cost of manual annotation. Furthermore, this small-sample image semantic segmentation method can be applied in scenarios where labeled data is scarce, such as medical image analysis, remote sensing image interpretation, and industrial quality inspection, achieving efficient and reliable image semantic segmentation in these scenarios.

[0144] This invention also provides a few-sample image semantic segmentation device optimized based on prior information. Figure 2 This is a schematic diagram of a few-sample image semantic segmentation device based on prior information optimization provided in an embodiment of the present invention; as shown below. Figure 2 As shown, the above-mentioned device includes:

[0145] The feature extraction module 210 is used to input the support image and the query image into the dual-branch encoder for feature extraction processing, to obtain the multi-scale support mid-level feature map and the multi-scale support high-level feature map of the support image, and the multi-scale query mid-level feature map and the multi-scale query high-level feature map of the query image; and to extract features from the preset foreground text prompt and the preset background text prompt to obtain scene text features.

[0146] The foreground localization prior information optimization module 220 is used to determine the foreground localization prior information of the query image based on the multi-scale support high-level feature map and the multi-scale query high-level feature map;

[0147] The multimodal prior information determination module 230 is used to determine multimodal prior information based on the multi-scale query high-level feature map and the scene text features;

[0148] Feature enhancement module 240 is used to fuse the query feature map, the foreground localization prior information, the multimodal prior information and the support category prototype to obtain an enhanced query feature map. The query feature map is determined based on the multi-scale query mid-layer feature map, and the support category prototype is determined based on the multi-scale support mid-layer feature map.

[0149] The semantic segmentation determination module 250 is used to perform feature interaction processing on the enhanced query feature map and the support feature map to obtain the image semantic segmentation result of the query image, wherein the support feature map is determined based on the multi-scale support mid-layer feature map.

[0150] This invention also provides an electronic device, comprising: a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a few-sample image semantic segmentation method based on prior information optimization as described in any of the method embodiments.

[0151] Embodiments of the present invention also provide a computer storage medium, which may be disposed in a server to store at least one instruction, at least one program, code set, or instruction set for implementing the method embodiments. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the few-sample image semantic segmentation method based on prior information optimization as described in any of the method embodiments.

[0152] Optionally, in embodiments of the present invention, the storage medium may be located at at least one of a plurality of network servers in a computer network. Optionally, in embodiments of the present invention, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0153] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more flowcharts and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more flowcharts and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more flowcharts and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0158] Finally, it should be noted that the embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A small sample image semantic segmentation method based on prior information optimization, characterized in that, The method comprises: inputting a support image and a query image into a double-branch encoder for feature extraction processing to obtain multi-scale support middle-layer feature maps and multi-scale support high-layer feature maps of the support image, and multi-scale query middle-layer feature maps and multi-scale query high-layer feature maps of the query image; performing feature extraction on preset foreground text prompts and preset background text prompts to obtain scene text features; determining foreground positioning prior information of the query image according to the multi-scale support high-layer feature maps and the multi-scale query high-layer feature maps; determining multi-modal prior information based on the multi-scale query high-layer feature maps and the scene text features; performing fusion processing on query feature maps, the foreground positioning prior information, the multi-modal prior information and support category prototypes to obtain enhanced query feature maps, wherein the query feature maps are determined based on the multi-scale query middle-layer feature maps, and the support category prototypes are determined based on the multi-scale support middle-layer feature maps; performing feature interaction processing on the enhanced query feature maps and support feature maps to obtain image semantic segmentation results of the query image, wherein the support feature maps are determined based on the multi-scale support middle-layer feature maps.

2. The small sample image semantic segmentation method based on prior information optimization according to claim 1, characterized in that, The double-branch encoder comprises a middle-layer feature extraction encoder and a high-layer feature extraction encoder; the feature extraction processing of the support image and the query image into the double-branch encoder comprises: performing middle-layer visual feature extraction on the support image and the query image through the middle-layer feature extraction encoder to obtain the multi-scale support middle-layer feature maps and the multi-scale query middle-layer feature maps; performing high-layer visual feature extraction on the support image and the query image through the high-layer feature extraction encoder to obtain the multi-scale support high-layer feature maps and the multi-scale query high-layer feature maps.

3. The small sample image semantic segmentation method based on prior information optimization according to claim 2, characterized in that, The middle-layer feature extraction encoder is a ResNet50 encoder, and the high-layer feature extraction encoder is a CLIP visual encoder; the multi-scale support middle-layer feature maps comprise support middle-layer feature maps of multiple scales, the multi-scale query middle-layer feature maps comprise query middle-layer feature maps of multiple scales, the multi-scale support high-layer feature maps comprise support high-layer feature maps of multiple scales, and the multi-scale query high-layer feature maps comprise query high-layer feature maps of multiple scales; the feature extraction processing of the support image and the query image into the double-branch encoder comprises: performing middle-layer visual feature extraction on the support image through the ResNet50 encoder to determine the outputs of the second and third residual blocks in the ResNet50 encoder as the support middle-layer feature maps of multiple scales; The ResNet50 encoder is used for middle-level visual feature extraction of the query image, and the outputs of the second and third residual blocks in the ResNet50 encoder are determined as multi-scale query middle-level feature maps; The CLIP visual encoder is used for high-level visual feature extraction of the support image, and the outputs of the tenth and eleventh Transformer layers in the CLIP visual encoder are determined as multi-scale support high-level feature maps; The CLIP visual encoder is used for high-level visual feature extraction of the query image, and the outputs of the tenth and eleventh Transformer layers in the CLIP visual encoder are determined as multi-scale query high-level feature maps.

4. The small sample image semantic segmentation method based on prior information optimization according to claim 1, characterized in that, The foreground positioning prior information of the query image is determined according to the multi-scale support high-level feature maps and the multi-scale query high-level feature maps, including: The support high-level feature maps of each scale in the multi-scale support high-level feature maps are respectively subjected to weighted global average pooling to determine foreground prototypes and background prototypes of each scale; The foreground similarity maps of each scale are determined according to the query high-level feature maps of each scale in the multi-scale query high-level feature maps and the foreground prototypes; The background similarity maps of each scale are determined according to the query high-level feature maps of each scale and the background prototypes; The foreground positioning prior information is determined according to the foreground similarity maps and the background similarity maps of each scale.

5. The small sample image semantic segmentation method based on prior information optimization according to claim 4, characterized in that, The foreground positioning prior information is determined according to the foreground similarity maps and the background similarity maps of each scale, including: The maximum foreground similarity in the foreground similarity map of each scale is determined as the foreground confidence of each scale; The foreground similarity map corresponding to the scale with the maximum foreground confidence is determined as the target foreground similarity map, and the background similarity map corresponding to the scale with the maximum foreground confidence is determined as the target background similarity map by comparing the foreground confidence of each scale; The target foreground similarity map and the target background similarity map are respectively subjected to normalization and shape remodeling to obtain foreground prior masks and background prior masks; The foreground positioning prior information is determined according to the foreground prior masks and the background prior masks.

6. The small sample image semantic segmentation method based on prior information optimization according to claim 1, characterized in that, The scene text features include foreground text features; The multi-modal prior information is determined based on the multi-scale query high-level feature maps and the scene text features, including: The feature activation map of the query image in the target feature layer is extracted; The similarity map between the multi-scale query high-level feature maps and the foreground text features is determined; The gradient tensor of the similarity map to the feature activation map is determined; The global average pooling is performed on the gradient tensor to determine the channel weight of each channel in the feature activation map; The channel weight is used for weighted processing of each channel of the query high-level feature map of the target feature layer to obtain a class activation mapping map; The multi-modal prior information is determined according to the class activation mapping map.

7. The small sample image semantic segmentation method based on prior information optimization according to claim 1, characterized in that, The fusion processing of the query feature map, the foreground positioning prior information, the multi-modal prior information and the support class prototype comprises: respectively along the channel dimension, to obtain support middle layer splicing feature maps and query middle layer splicing feature maps; respectively along the channel dimension, to obtain support middle layer splicing feature maps and query middle layer splicing feature maps; respectively along the channel dimension, to obtain support middle layer splicing feature maps and query middle layer splicing feature maps; respectively along the channel dimension, to obtain support middle layer splicing feature maps and query middle layer splicing feature maps; 8. The small sample image semantic segmentation method based on prior information optimization according to claim 1, characterized in that, respectively along the channel dimension, to obtain support middle layer splicing feature maps and query middle layer splicing feature maps. The image semantic segmentation result comprises a query segmentation mask; the feature interaction processing of the enhanced query feature map and the support feature map to obtain the image semantic segmentation result of the query image comprises: inputting the support feature map and the enhanced query feature map into a target decoder, processing the support feature map through two independent linear projection layers to obtain an attention key vector and an attention value vector, and processing the enhanced query feature map through a linear projection layer to obtain an attention query vector; determining a cross-attention weight matrix according to a first dot product of the attention query vector and the attention key vector; determining a second dot product of the cross-attention weight matrix and the attention value vector; 9. An apparatus for small sample image semantic segmentation based on prior information optimization, characterized in that, processing the second dot product through a classification head to obtain a query segmentation mask. The device comprises: a feature extraction module configured to input a support image and a query image into a double-branch encoder for feature extraction processing to obtain multi-scale support middle layer feature maps and multi-scale support high layer feature maps of the support image, and multi-scale query middle layer feature maps and multi-scale query high layer feature maps of the query image; and extract features of a preset foreground text prompt and a preset background text prompt to obtain scene text features; a foreground positioning prior information optimization module configured to determine foreground positioning prior information of the query image according to the multi-scale support high layer feature maps and the multi-scale query high layer feature maps; a multi-modal prior information determination module configured to determine multi-modal prior information based on the multi-scale query high layer feature maps and the scene text features; a feature enhancement module configured to perform fusion processing of a query feature map, the foreground positioning prior information, the multi-modal prior information and a support class prototype to obtain an enhanced query feature map, the query feature map being determined based on the multi-scale query middle layer feature maps, and the support class prototype being determined based on the multi-scale support middle layer feature maps; a semantic segmentation determination module configured to perform feature interaction processing of the enhanced query feature map and a support feature map to obtain an image semantic segmentation result of the query image, the support feature map being determined based on the multi-scale support middle layer feature maps.

10. A computer storage medium, having stored therein at least one instruction and at least one program, the at least one instruction and the at least one program being loaded and executed by a processor to implement the small sample image semantic segmentation method based on prior information optimization according to any one of claims 1 to 8.

Images