A foreground-background proportion perception-based few-shot image segmentation method and system

Abstract

The application discloses a few-shot image segmentation method and system based on foreground-background proportion perception, and belongs to the technical field of computer vision and artificial intelligence, and comprises the following steps: acquiring a support set and a query set; constructing a foreground-background proportion perception segmentation model to perform the following operations: extracting support foreground features and high-level query features by using a deep feature extraction network according to the support set and the query set; obtaining foreground-background proportions of the query image under each support image; obtaining each calibration similarity matrix; and performing random disturbance enhancement and normalization on each calibration similarity matrix to form an attention weight matrix; obtaining the segmentation result of the query image by using an attention fusion module according to each support foreground feature, each attention weight matrix and the high-level query feature; and inputting an image data set into the trained foreground-background proportion perception segmentation model to obtain the segmentation result of the query image. The robustness, generalization ability and segmentation precision are greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and artificial intelligence technology, specifically relating to a few-sample image segmentation method and system based on foreground-background ratio perception. Background Technology

[0002] Semantic segmentation is used to classify each pixel in an image and is a fundamental technology in tasks such as industrial inspection, medical image analysis, remote sensing interpretation, autonomous driving, and ecological monitoring. Existing deep learning segmentation models typically rely on large amounts of pixel-level labeled data. Pixel-level labeling requires depicting target boundaries region by region, which is time-consuming, labor-intensive, and difficult to quickly obtain sufficient samples for different categories or scenarios.

[0003] Few-shot semantic segmentation aims to segment query images with unseen categories using a small number of masked support images. These methods typically employ contextual training, where each training or testing task is input with a support set (including several support images and corresponding masks) and a query set (including a query image and its corresponding mask). The model predicts the corresponding target region in the query image based on category cues provided by the support images.

[0004] Existing few-shot semantic segmentation methods typically include prototype-based methods and attention-based methods. Specifically: 1) Prototype-based methods first extract foreground features from the support image using a support mask, then compress all foreground features into a few prototype vectors using masked average pooling. These prototype vectors no longer carry spatial location and local fine-grained information; that is, target boundaries, local textures, and multi-part structural information are averaged, leading to a loss of spatial location and local fine-grained information. This results in degraded contour representation at the target boundary, and subsequent segmentation guided by the similarity between prototype and query features fails to reflect true detail differences, ultimately amplifying problems such as blurred boundaries, missing details, and inability to distinguish sub-parts. 2) Attention-based methods, by establishing pixel-level or feature unit-level correspondences between query features in the query image and support features in the support image, can retain more spatial details. However, in cross-attention computation, query features simultaneously include foreground and background features, while support features are typically more biased towards foreground features. The similarity between background features in the query image and foreground features in the supporting images is generally low. However, after normalization, a large number of low-similarity background locations may still accumulate to form non-negligible attention weights, misleading background regions into foreground regions and causing a mixture of foreground and background representations. Directly applying hard suppression to background attention will result in the loss of contextual information helpful for target boundary localization, and the suppressed regions are unlikely to obtain effective gradients, leading to low image segmentation accuracy. Furthermore, for few-shot segmentation tasks, the number of supporting images is limited, and the model easily relies on occasional background co-occurrence relationships in the training set, thus affecting its generalization ability to unseen categories.

[0005] Therefore, a few-sample image segmentation method and system based on foreground-background ratio perception is proposed. Summary of the Invention

[0006] The purpose of this invention is to address the above-mentioned problems by proposing a few-shot image segmentation method and system based on foreground-background ratio awareness. While preserving fine-grained information of the supporting image, it dynamically adjusts the attention allocation according to the foreground-background ratio of the query image to retain weak background context, which greatly improves the robustness, generalization ability and segmentation accuracy of the model.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] This invention proposes a few-shot image segmentation method based on foreground-background ratio awareness, comprising the following steps:

[0009] S1. Obtain the image dataset and divide it into a support set and a query set. The support set includes several support images and corresponding masks, and the query set includes the query image, i.e. the image to be segmented.

[0010] S2. Construct a foreground-background ratio-aware segmentation model to perform the following operations:

[0011] S21. Extract supporting foreground features and high-level query features using a deep feature extraction network based on the support set and query set;

[0012] S22. Flatten each supporting foreground feature along the spatial dimension into a corresponding supporting feature matrix, flatten the high-level query feature along the spatial dimension into a query feature matrix, and calculate the similarity between the query feature matrix and each supporting feature matrix to obtain the corresponding similarity matrix. Based on each similarity matrix, obtain the foreground-background ratio of the query image under the corresponding supporting image.

[0013] S23. Based on the foreground-background ratio of the query image under each supporting image, perform dynamic calibration of the corresponding similarity matrix to obtain the corresponding calibration similarity matrix;

[0014] S24. Randomly perturb and enhance each calibration similarity matrix to obtain the corresponding attention score matrix, and normalize each attention score matrix to form an attention weight matrix.

[0015] S25. Based on each supporting foreground feature, each attention weight matrix, and high-level query features, the attention fusion module is used to obtain the segmentation result of the query image;

[0016] S3. Input the image dataset into the trained foreground-background ratio-aware segmentation model to obtain the segmentation results of the query image.

[0017] Preferably, the deep feature extraction network adopts one of the following models: ResNet, ConvNeXt, Vision Transformer, and Swin Transformer.

[0018] Preferably, supporting foreground features and high-level query features are extracted using a deep feature extraction network based on the support set and query set, as follows:

[0019] Each support image in the support set is input into a deep feature extraction network to obtain the corresponding high-level support features. Then, the mask of each support image in the support set is transformed in dimension and multiplied element-wise with the corresponding high-level support features to obtain the corresponding support foreground features.

[0020] The query images in the query set are input into a deep feature extraction network to obtain high-level query features.

[0021] Preferably, the foreground-to-background ratio of the query image under the corresponding supporting image is obtained based on each similarity matrix, as follows:

[0022] S221. Obtain the soft prior mask for each similarity matrix. Specifically, average the similarity matrix along its column dimensions to obtain the activation score vector of the corresponding query feature matrix. Obtain the corresponding soft prior mask based on the activation score vector of the query feature matrix, calculated as follows:

[0023]

[0024] in, Indicates the query feature matrix and the first The soft prior mask of the similarity matrix of the support feature matrix of the supported image. Indicates the query feature matrix and the first The similarity matrix of the support feature matrix of the supported image The corresponding activation score vector of the query feature matrix. This indicates that the query has been converted to the same spatial size as the high-level query features. This represents the sigmoid function. Indicates temperature factor, , Indicates the number of images supported in the support set;

[0025] S222. Calculate the product of the spatial average of the soft prior mask of each similarity matrix and the scaling parameter as the foreground-background ratio of the query image under the corresponding support image. The scaling parameter... The value range is 0 to 1.

[0026] Preferably, the corresponding similarity matrix is ​​dynamically calibrated based on the foreground-to-background ratio of the query image under each supporting image to obtain the corresponding calibrated similarity matrix, as follows:

[0027] Flatten each similarity matrix into a one-dimensional similarity sequence, then sort the elements in the one-dimensional similarity sequence according to the size of the similarity, and use the similarity of the foreground-background ratio of the query image under the corresponding support image as the dynamic threshold. The position with similarity greater than or equal to the dynamic threshold is recorded as the pseudo-foreground region, and the position with similarity less than the dynamic threshold is recorded as the pseudo-background region.

[0028] A calibration similarity matrix is ​​established based on the dynamic threshold and the corresponding similarity matrix. Then, the first... Line number The elements of the column are represented as follows:

[0029]

[0030] in, To calibrate the similarity matrix The Middle Line number Column elements, For the corresponding similarity matrix, the first... Line number Column elements, , , This indicates the number of elements in the query feature matrix. This indicates the number of elements in the corresponding supporting feature matrix. For the corresponding dynamic threshold, This is a preset constant, and its value range is [value range missing]. ~ .

[0031] Preferably, each calibration similarity matrix is ​​enhanced by random perturbation to obtain the corresponding attention score matrix. Each attention score matrix is ​​then normalized to form an attention weight matrix, as shown in the following formula:

[0032]

[0033] Among them, the attention score matrix corresponds to the calibration similarity matrix. The Middle Line number Column elements The calculation is as follows:

[0034]

[0035] in, The attention weight matrix corresponding to the calibration similarity matrix, This indicates normalization along the column dimension. Indicates the first element in the corresponding calibration similarity matrix. Line number Disturbance noise of column elements, The value represents the disturbance intensity, ranging from 0.001 to 1.0.

[0036] Preferably, the proportional adjustment parameter and the disturbance intensity are obtained using one of the following methods:

[0037] 1) The proportional adjustment parameter is a first preset fixed value, and the disturbance intensity is a second preset fixed value;

[0038] 2) Both the proportional adjustment parameter and the perturbation intensity are adaptively adjusted according to the training process, that is, the proportional adjustment parameter increases with the training rounds, and the perturbation intensity decreases with the training rounds.

[0039] 3) Both the scaling parameter and the perturbation strength are adaptively adjusted based on the prior mask confidence level. Specifically, the prior mask confidence level is calculated based on the entropy of each soft prior mask. When the prior mask confidence level is less than the first confidence threshold, the scaling parameter is the first scaling threshold and the perturbation strength is the first perturbation threshold. When the prior mask confidence level is greater than or equal to the first confidence threshold and less than the second confidence threshold, the scaling parameter is the second scaling threshold and the perturbation strength is the second perturbation threshold. When the prior mask confidence level is greater than or equal to the second confidence threshold, the scaling parameter is the third scaling threshold and the perturbation strength is the third perturbation threshold. The first scaling threshold, the second scaling threshold, and the third scaling threshold increase sequentially, as do the first perturbation threshold, the second perturbation threshold, and the third perturbation threshold.

[0040] 4) The proportional adjustment parameter is adaptively adjusted according to the target area ratio, that is, it is positively correlated with the target area ratio. The disturbance intensity is dynamically adjusted according to the area ratio of the pseudo-background region, that is, it is negatively correlated with the area ratio of the pseudo-background region.

[0041] Preferably, the attention fusion module performs the following operations:

[0042] S251. After mapping each supporting foreground feature into a value matrix, multiply it by the attention weight matrix of the corresponding calibration similarity matrix to obtain the cross-attention output features of the corresponding supporting image.

[0043] S252. Perform self-attention calculation on the high-level query features to obtain the self-attention output features;

[0044] S253. The cross-attention output features of each support image are concatenated with the self-attention output features along the channel dimension, and mapped back to the same number of channels as the high-level query features through a convolutional layer. After passing through a non-linear activation function, residual fusion is performed with the high-level query features to obtain the fused features of the corresponding support images.

[0045] S254. Input the fusion features of each supporting image into the decoder to obtain the pixel-level segmentation result of the query image under the corresponding supporting image;

[0046] S255. The pixel-level segmentation results of the query image under each supporting image are weighted and fused to obtain the segmentation result of the query image. The weight of the pixel-level segmentation result of the query image under each supporting image is obtained by normalizing the average value of all elements of the similarity matrix between the query feature matrix and the support feature matrix of the corresponding supporting image.

[0047] Preferably, the foreground-background ratio-aware segmentation model is trained using a training image set, which is divided into several support-query pairs. Each support-query pair includes a support set and a query set. The support set of each support-query pair includes several support images and corresponding masks, and the query set of each support-query pair includes a query image and a corresponding mask. The segmentation loss during training is obtained by using Dice loss or by weighted summation of Dice loss and cross-entropy loss.

[0048] A few-shot image segmentation system based on foreground-background ratio awareness, comprising:

[0049] The data acquisition module is used to acquire image datasets and divide them into support sets and query sets. The support set includes several support images and corresponding masks, and the query set includes query images, i.e., the images to be segmented.

[0050] The model building module is used to build a foreground-background ratio-aware segmentation model, including:

[0051] The feature extraction module is used to extract supporting foreground features and high-level query features based on the support set and query set using a deep feature extraction network;

[0052] The ratio estimation module is used to flatten each supporting foreground feature along the spatial dimension into a corresponding supporting feature matrix, flatten the high-level query features along the spatial dimension into a query feature matrix, calculate the similarity between the query feature matrix and each supporting feature matrix to obtain the corresponding similarity matrix, and obtain the foreground-background ratio of the query image under the corresponding supporting image based on each similarity matrix.

[0053] The proportional dynamic calibration module is used to dynamically calibrate the corresponding similarity matrix based on the foreground-background ratio of the query image under each supported image, and obtain the corresponding calibration similarity matrix.

[0054] The perturbation enhancement module is used to randomly perturb and enhance each calibration similarity matrix to obtain the corresponding attention score matrix, and then normalize each attention score matrix to form an attention weight matrix.

[0055] The feature fusion module is used to obtain the segmentation result of the query image by using the attention fusion module based on each supporting foreground feature, each attention weight matrix and high-level query features;

[0056] The output module is used to input the image dataset into the trained foreground-background ratio-aware segmentation model to obtain the segmentation results of the query image.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] To address the problems of lost supporting foreground information, amplified background weights due to cross-attention, and the impact of hard background suppression on generalization ability in existing few-shot image segmentation methods, this invention utilizes a deep feature extraction network to extract supporting foreground features and high-level query features based on the support set and query set. Each supporting foreground feature is flattened along the spatial dimension into a corresponding supporting feature matrix, and the high-level query features are flattened along the spatial dimension into a query feature matrix. The similarity between the query feature matrix and each supporting feature matrix is ​​calculated to obtain the corresponding similarity matrix. Based on each similarity matrix, the foreground-to-background ratio of the query image under the corresponding supporting image is obtained. Subsequently, the similarity matrix is ​​dynamically thresholded based on the foreground-to-background ratio of the query image under the corresponding supporting image to obtain pseudo-foreground and pseudo-background regions. Controlled random perturbation is introduced into the pseudo-background region to avoid information loss caused by completely removing the background context. Finally, the segmentation result of the query image is obtained using an attention fusion module based on each supporting foreground feature, each attention weight matrix, and the high-level query features. Specifically:

[0059] 1) This invention establishes a fine-grained similarity relationship between high-level query features and supporting foreground features, which is beneficial for preserving information such as local texture, boundaries and component differences, rather than simply compressing the foreground information of the supporting image into a single prototype, thus avoiding the loss of fine-grained information;

[0060] 2) This invention estimates the foreground-background ratio from a soft prior mask and uses the foreground-background ratio for dynamic calibration of the similarity matrix. It can adaptively adjust the attention sparsity according to the target area changes in different images, reducing the inadequacy of fixed thresholds in small targets, large targets or complex background scenes.

[0061] 3) Before normalizing each attention score matrix to form the attention weight matrix, the present invention processes the pseudo-background region, which can reduce the probability that the low similarity response of the background is amplified after normalization, thereby alleviating the problem of mixed foreground and background features.

[0062] 4) This invention introduces controlled random perturbation enhancement in the pseudo-background region, so that the model does not rely excessively on the fixed background pattern in the training samples, while retaining some background context information, which helps to improve the model's generalization ability on unseen categories and cross-scene images.

[0063] 5) The scale-aware attention module of the present invention (including scale estimation module, scale dynamic calibration module and perturbation enhancement module) can be used as a plug-and-play module for existing few-shot segmentation networks, and is not limited to training datasets or categories. The scale-aware attention module and the foreground-background scale-aware segmentation model as a whole are easy to deploy in segmentation scenarios such as industrial defect images, medical images, remote sensing images and natural scene images. Attached Figure Description

[0064] Figure 1 This is a flowchart of the few-sample image segmentation method based on foreground-background ratio awareness of the present invention.

[0065] Figure 2 This is a flowchart of the attention fusion module of the present invention;

[0066] Figure 3 This is a schematic diagram of the foreground-background ratio-aware segmentation model of the present invention;

[0067] Figure 4 This is a comparison chart of the segmentation results of the present invention and existing technologies under different proportions of the actual foreground area. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] It should be noted that when a component is referred to as being "connected" to another component, it can be directly connected to the other component or there may be an intervening component. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application.

[0070] Example 1:

[0071] like Figures 1-4 As shown, a few-shot image segmentation method based on foreground-background ratio awareness includes the following steps:

[0072] S1. Obtain the image dataset and divide it into a support set and a query set. The support set includes several support images and corresponding masks, and the query set includes the query image, i.e. the image to be segmented.

[0073] The few-shot segmentation task employs contextual input, meaning that each training iteration provides the few-shot segmentation model with a set of supporting and query sets as input to create a training scenario. Specifically, the image dataset corresponding to each few-shot segmentation task includes a supporting set... and query set ,in, Indicates support for the centralized first Zhang supports images, Indicates support for the centralized first Zhang supports image masks. , This indicates the number of images supported in the set. The query image is the image to be segmented. This is used to query the mask of the image. It should be noted that the mask in this embodiment is a binary mask. When the mask value is 0, it represents the background; when the mask value is 1, it represents the foreground. In this embodiment, the pixel size of the image being queried is 473×473.

[0074] This embodiment does not limit the number of images that can be supported in a centralized manner. When = 1, this few-shot segmentation task is a 1-shot few-shot segmentation task; when When the value is greater than 1, this few-shot segmentation task is a K-shot few-shot segmentation task. A mask of the query image can be used during the training phase. Calculate the supervised loss; do not use the mask of the query image during the testing phase. This invention provides target category hints for query images using only masks that support all supported images in the dataset. It can be used for target segmentation in industrial defect images, medical lesion area images, remote sensing ground object images, agricultural pest and disease area images, microscopic images, and other scenarios requiring pixel-level segmentation and recognition with a limited number of labeled samples.

[0075] S2. Construct a foreground-background ratio-aware segmentation model to perform the following operations:

[0076] S21. Based on the support set and query set, use a deep feature extraction network to extract supporting foreground features and high-level query features.

[0077] In one embodiment, the deep feature extraction network employs one of the following models: ResNet, ConvNeXt, VisionTransformer, or Swing Transformer. Alternatively, other feature extraction networks with image representation capabilities may also be used.

[0078] In one embodiment, supporting foreground features and high-level query features are extracted using a deep feature extraction network based on the support set and the query set, as follows:

[0079] Each support image in the support set is input into a deep feature extraction network to obtain the corresponding high-level support features. Then, the mask of each support image in the support set is transformed in dimension and multiplied element-wise with the corresponding high-level support features to obtain the corresponding support foreground features.

[0080] The query images in the query set are input into a deep feature extraction network to obtain high-level query features.

[0081] Specifically, record For the first Zhang supports high-level support features of the image. This is for high-level query features. It will support centralized first-level... Zhang supports the image mask After performing dimensional transformation, the result is multiplied element-wise with the corresponding high-level support feature to obtain the _th_ ... Zhang supports foreground features of the image :

[0082]

[0083] in, This represents the Hadamard product. To ensure the transformation dimension is the same as the dimension of the high-level support features, this step is used to reduce the impact of background regions in the support image on subsequent similarity calculations.

[0084] S22. Flatten each supporting foreground feature along the spatial dimension into a corresponding supporting feature matrix, flatten the high-level query feature along the spatial dimension into a query feature matrix, and calculate the similarity between the query feature matrix and each supporting feature matrix to obtain the corresponding similarity matrix. Based on each similarity matrix, obtain the foreground-background ratio of the query image under the corresponding supporting image.

[0085] In one embodiment, the foreground-to-background ratio of the query image under the corresponding supporting image is obtained based on each similarity matrix, as follows:

[0086] S221. Obtain the soft prior mask for each similarity matrix. Specifically, average the similarity matrix along its column dimensions to obtain the activation score vector of the corresponding query feature matrix. Obtain the corresponding soft prior mask based on the activation score vector of the query feature matrix, calculated as follows:

[0087]

[0088] in, Indicates the query feature matrix and the first The soft prior mask of the similarity matrix of the support feature matrix of the supported image. Indicates the query feature matrix and the first The similarity matrix of the support feature matrix of the supported image The corresponding activation score vector of the query feature matrix. This indicates that the query has been converted to the same spatial size as the high-level query features. This represents the sigmoid function. Indicates temperature factor, , Indicates the number of images supported in the support set;

[0089] S222. Calculate the product of the spatial average of the soft prior mask of each similarity matrix and the scaling parameter as the foreground-background ratio of the query image under the corresponding support image. The scaling parameter... The value range is 0 to 1.

[0090] Specifically, the first Zhang supports foreground features of the image Flattened along the spatial dimension to the first Zhang's support feature matrix High-level query features Flattened along the spatial dimension into a query feature matrix , Indicates the first The first in the support feature matrix of the supported image One element, Represents the first element in the query feature matrix. One element, This indicates the number of elements in the corresponding supporting feature matrix. This indicates the number of elements in the query feature matrix, and calculates the query feature matrix and the number of elements in the first feature matrix. The similarity between the support feature matrices of the supporting images is obtained by comparing the query feature matrix with the first... The similarity matrix of the support feature matrix of the supported image Then query the first element in the feature matrix. The element and the first The first in the support feature matrix of the supported image Similarity of elements Represented as:

[0091]

[0092] in, for The Middle Line number The elements of the column are calculated using cosine similarity in this embodiment, that is, the elements in the query feature matrix are the elements of the column. The element and the first The first in the support feature matrix of the supported image The normalized value of the cosine similarity between elements. It is a norm 2. For transpose, , .

[0093] Obtain the soft prior masks for each similarity matrix, as follows:

[0094] The activation score vector of the corresponding query feature matrix is ​​obtained by averaging the similarity matrices along the column dimensions. Based on the activation score vector of the query feature matrix, the corresponding soft prior mask is obtained. Then, the query feature matrix and the... Soft prior mask of the similarity matrix of the support feature matrix of the supported image. The calculation is as follows:

[0095]

[0096] but The activation score vector of the corresponding query feature matrix The Middle element The calculation is as follows:

[0097]

[0098] in, This represents the temperature factor, and its value ranges from 1 to 10. In this embodiment, =5, This represents the sigmoid function. express The activation score vector corresponding to the query feature matrix. This indicates a transformation to the same spatial dimension as the high-level query features (transformation dimension). The pixel value of each pixel at each spatial location is between 0 and 1, which can represent the probability that the corresponding pixel in the query image belongs to the foreground.

[0099] The spatial average of the soft prior mask for each similarity matrix is ​​calculated as the foreground-to-background ratio of the query image under the corresponding support image. The foreground-to-background ratio of the query image under the corresponding support image is calculated as follows:

[0100]

[0101] in, This indicates the foreground-to-background ratio of the queried image under the corresponding supported image. It is the global arithmetic mean function. This represents the soft prior mask corresponding to the similarity matrix. This is a proportional adjustment parameter, with a value range of 0 to 1. In this embodiment... For a fixed value, take =0.75. Proportional adjustment parameter. This is used to correct the blending effect between the foreground boundary and the background in the feature space, and can also be adaptively adjusted according to actual needs, including but not limited to the following schemes:

[0102] 1) Adaptive adjustment based on the training progress, meaning the value is gradually increased as the number of training rounds increases, with smaller values ​​in the early stages and gradually larger values ​​in the later stages. Specifically, the training stage is divided according to the proportion of the current training round to the total number of training rounds. For example, if the current training round accounts for less than 30% of the total training rounds, it is classified as the early stage of training; 30% to 70% is classified as the middle stage of training; and more than 70% is classified as the late stage of training. In the early stage of training, the value is... Set to 0.50, during the middle of training. Gradually increase from 0.50 to 0.75, in the later stages of training. Let's set it to 0.75. For example, when the total number of training rounds is 100, rounds 1 to 29 are considered the early training phase, so we take... =0.50; Rounds 30 to 70 constitute the middle of the training period. The value was linearly increased from 0.50 to 0.75; the period from round 71 to round 100 was the later stage of training, and the value was taken as... =0.75.

[0103] 2) Adaptive adjustment based on prior mask confidence, as follows:

[0104] The confidence level of the corresponding prior mask is calculated based on each soft prior mask, using the following formula:

[0105]

[0106] in, This represents the prior mask confidence level. Indicates the first in the soft prior mask The element value at each spatial location, , Indicates the height of high-level query features. Indicates the width of the high-level query feature. This represents the total number of spatial locations of high-level query features. The constant is a very small constant that avoids abnormal logarithmic calculations. In this embodiment, The soft prior mask has the same spatial size as the high-level query features and corresponds one-to-one with their spatial positions in the query feature matrix.

[0107] The corresponding proportional adjustment parameters are adaptively adjusted based on the prior mask confidence levels. Specifically, the preset first confidence threshold is 0.60, and the second confidence threshold is 0.75. For example, when the prior mask confidence level... When the value is less than 0.60, it is considered that the pixel value of most spatial locations in the soft prior mask is close to 0.5, indicating that the reliability of the current soft prior mask is low. Therefore, a proportional adjustment parameter is used. =0.50; when 0.60≤ When <0.75, take =0.65; when When the value is ≥0.75, it is considered that the pixel values ​​of most spatial locations in the soft prior mask are close to 0 or 1, and the reliability of the current prior mask is relatively high. =0.75. As the confidence of the prior mask increases, the scaling parameter is increased to enhance the guiding role of reliable foreground priors on high-level query features; when the confidence of the prior mask is low, the scaling parameter is decreased to reduce the impact of erroneous priors on feature fusion and subsequent segmentation.

[0108] 3) Adaptive adjustment based on target area ratio: The target area ratio is determined by dividing the foreground and background according to a preset threshold (e.g., 0.5). In the soft prior mask, pixels with spatial positions greater than the preset threshold are classified as foreground, and pixels with spatial positions less than or equal to the preset threshold are classified as background. The target area ratio is the ratio of foreground pixels to all pixels; the larger the target area ratio, the better. The larger the value, specifically, the smaller the target area percentage (less than 20%). Using a value of 0.50, the target area ratio is 20%–50%. Taking 0.65, the target area accounts for more than 50%. Take 0.75. All the specific values ​​used in the above divisions can be adjusted according to actual needs.

[0109] S23. Based on the foreground-background ratio of the query image under each supporting image, perform dynamic calibration of the corresponding similarity matrix to obtain the corresponding calibrated similarity matrix.

[0110] In one embodiment, the corresponding similarity matrix is ​​dynamically calibrated based on the foreground-to-background ratio of the query image under each supporting image to obtain the corresponding calibrated similarity matrix, as follows:

[0111] Flatten each similarity matrix into a one-dimensional similarity sequence, then sort the elements in the one-dimensional similarity sequence according to the size of the similarity, and use the similarity of the foreground-background ratio of the query image under the corresponding support image as the dynamic threshold. The position with similarity greater than or equal to the dynamic threshold is recorded as the pseudo-foreground region, and the position with similarity less than the dynamic threshold is recorded as the pseudo-background region.

[0112] A calibration similarity matrix is ​​established based on the dynamic threshold and the corresponding similarity matrix. Then, the first... Line number The elements of the column are represented as follows:

[0113]

[0114] in, To calibrate the similarity matrix The Middle Line number Column elements, For the corresponding similarity matrix, the first... Line number Column elements, , , This indicates the number of elements in the query feature matrix. This indicates the number of elements in the corresponding supporting feature matrix. For the corresponding dynamic threshold, This is a preset constant, and its value range is [value range missing]. ~ .

[0115] S24. Randomly perturb and enhance each calibration similarity matrix to obtain the corresponding attention score matrix, and normalize each attention score matrix to form an attention weight matrix.

[0116] In one embodiment, each calibration similarity matrix is ​​enhanced by random perturbation to obtain the corresponding attention score matrix. Each attention score matrix is ​​then normalized to form an attention weight matrix, as shown in the following formula:

[0117]

[0118] Among them, the attention score matrix corresponds to the calibration similarity matrix. The Middle Line number Column elements The calculation is as follows:

[0119]

[0120] in, The attention weight matrix corresponding to the calibration similarity matrix, This indicates normalization along the column dimension. Indicates the first element in the corresponding calibration similarity matrix. Line number Disturbance noise of column elements, The value represents the disturbance intensity, ranging from 0.001 to 1.0.

[0121] In one embodiment, the proportional adjustment parameter and the disturbance intensity are obtained using one of the following methods:

[0122] 1) The proportional adjustment parameter is a first preset fixed value, and the disturbance intensity is a second preset fixed value;

[0123] 2) Both the proportional adjustment parameter and the perturbation intensity are adaptively adjusted according to the training process, that is, the proportional adjustment parameter increases with the training rounds, and the perturbation intensity decreases with the training rounds.

[0124] 3) Both the scaling parameter and the perturbation strength are adaptively adjusted based on the prior mask confidence level. Specifically, the prior mask confidence level is calculated based on the entropy of each soft prior mask. When the prior mask confidence level is less than the first confidence threshold, the scaling parameter is the first scaling threshold and the perturbation strength is the first perturbation threshold. When the prior mask confidence level is greater than or equal to the first confidence threshold and less than the second confidence threshold, the scaling parameter is the second scaling threshold and the perturbation strength is the second perturbation threshold. When the prior mask confidence level is greater than or equal to the second confidence threshold, the scaling parameter is the third scaling threshold and the perturbation strength is the third perturbation threshold. The first scaling threshold, the second scaling threshold, and the third scaling threshold increase sequentially, as do the first perturbation threshold, the second perturbation threshold, and the third perturbation threshold.

[0125] 4) The proportional adjustment parameter is adaptively adjusted according to the target area ratio, that is, it is positively correlated with the target area ratio. The disturbance intensity is dynamically adjusted according to the area ratio of the pseudo-background region, that is, it is negatively correlated with the area ratio of the pseudo-background region.

[0126] Specifically, the similarity matrix is ​​flattened into a one-dimensional similarity sequence. Then, the elements in the one-dimensional similarity sequence are sorted by similarity. The similarity at the quantile of the foreground-background ratio of the query image under the corresponding support image is used as a dynamic threshold. Positions with similarity greater than or equal to the dynamic threshold are designated as pseudo-foreground regions, and positions with similarity less than the dynamic threshold are designated as pseudo-background regions. The foreground-background ratio is used to determine the proportion of images with the highest similarity. For example, sorting them from largest to smallest, when... When the similarity ratio is 0.2, it means that the 20% of positions with the highest similarity in the similarity matrix are retained as pseudo-foreground regions, and the rest are retained as pseudo-background regions. In other words, 20% of the positions in the query image under the similarity matrix belong to the foreground, and the similarity of the quantile positions of the foreground and background ratios is the dynamic threshold.

[0127] A calibration similarity matrix is ​​established based on the dynamic threshold and the corresponding similarity matrix. Then, the first... Line number The elements of the column are represented as follows:

[0128]

[0129] in, To calibrate the similarity matrix The Middle Line number Column elements, For the corresponding similarity matrix, the first... Line number Column elements, For the corresponding dynamic threshold, It is a preset constant, and its value range is [value range missing]. ~ In this embodiment, Set the similarity of the pseudo-background regions to a preset constant. , This can reduce the weight contribution of pseudo-background regions in subsequent attention weight matrix calculations. Compared to a fixed threshold, this dynamic threshold adjusts according to the target area ratio in the query image, adapting to foreground targets of different sizes.

[0130] S5. Based on the calibration similarity matrix, perform random perturbation enhancement on the corresponding pseudo-background region to obtain the attention score matrix. Normalize the attention score matrix to form the attention weight matrix, as shown in the following formula:

[0131]

[0132] Among them, the attention score matrix corresponds to the calibration similarity matrix. The Middle Line number Column elements The formula is as follows:

[0133]

[0134] in, The attention weight matrix corresponding to the calibration similarity matrix, This indicates normalization along the column dimension. Indicates the first element in the corresponding calibration similarity matrix. Line number Disturbance noise of column elements, The disturbance intensity ranges from 0.001 to 1.0. In this embodiment, Disturbance intensity It can also be dynamically adjusted according to actual needs, including but not limited to the following options:

[0135] 1) Dynamically adjust by gradually decreasing the decay as the training progresses; for example, initially take... , The perturbation value gradually decreases to 0.001 with each training epoch, giving the model a strong regularization effect in the early stages of training and reducing the intensity of perturbation in the later stages of training. To facilitate convergence. For example, when the total number of training rounds is 100, the perturbation strength... The value is set to 0.05 in the first round and gradually decreases with each training round using a linear decay method, decaying to 0.001 in the 100th round.

[0136] 2) Dynamically adjust based on prior mask confidence; that is, when prior mask confidence is high, background suppression strength (i.e., perturbation strength) can be increased. When the prior mask confidence is low, the background suppression strength can be reduced to avoid excessive suppression of foreground information due to erroneous priors. (For example, the prior mask confidence...) When <0.60, take =0.10; when 0.60≤ When <0.75, take =0.30; when When ≥0.75, take =0.50.

[0137] 3) Dynamically adjust based on the area ratio of the pseudo-background region. For example, when the area ratio of the pseudo-background region is large, the disturbance intensity can be reduced according to actual needs. To avoid introducing excessive background noise. For example, when the area of ​​the pseudo-background region is less than 40%, the perturbation intensity... Set to 0.50; when the area of ​​the pseudo-background region is greater than or equal to 40% and less than 70%, the disturbance intensity is... Set to 0.30; when the area of ​​the pseudo-background region is greater than or equal to 70%, the disturbance intensity is... Take 0.10.

[0138] In this embodiment, Indicates the first element in the corresponding calibration similarity matrix. Line number Disturbance noise of column elements It follows a Gaussian distribution with a mean of 0 and a variance of 1. It is easy to understand that the perturbation noise is not limited to the Gaussian noise of this embodiment; uniform noise, learnable perturbations, and random mask perturbations can also be used. Among them, 1) when using uniform noise, it satisfies... , Indicates the first element in the corresponding calibration similarity matrix. Line number Uniform noise of column elements 1) It follows a uniform distribution on the interval [-1, 1]; 2) When a learnable perturbation is used, The learning function is obtained by inputting high-level query features and supporting foreground features into a learnable function, which is a neural network consisting of a 3×3 convolutional layer, a ReLU activation function, a 1×1 convolutional layer, and a Sigmoid activation function connected in sequence; 3) random mask perturbation is used. For the corresponding calibration similarity matrix, the first... Line number Uniform noise of column elements The result of multiplying with a random mask allows some pseudo-background interaction locations to retain the perturbation response.

[0139] Completely removing interactions from the pseudo-background region might result in the loss of contextual information near the target boundary. Therefore, controlled random perturbation enhancement is introduced into the pseudo-background region to obtain the attention weight matrix corresponding to the calibrated similarity matrix. This random perturbation enhancement allows the pseudo-background region to retain a small amount of random response in the attention matrix, thus reducing its dependence on fixed background patterns in a regularized manner.

[0140] S25. Based on each supporting foreground feature, each attention weight matrix, and high-level query features, the attention fusion module is used to obtain the segmentation result of the query image.

[0141] In one embodiment, the attention fusion module performs the following operations:

[0142] S251. After mapping each supporting foreground feature into a value matrix, multiply it by the attention weight matrix of the corresponding calibration similarity matrix to obtain the cross-attention output features of the corresponding supporting image.

[0143] S252. Perform self-attention calculation on the high-level query features to obtain the self-attention output features;

[0144] S253. The cross-attention output features of each support image are concatenated with the self-attention output features along the channel dimension, and mapped back to the same number of channels as the high-level query features through a convolutional layer. After passing through a non-linear activation function, residual fusion is performed with the high-level query features to obtain the fused features of the corresponding support images.

[0145] S254. Input the fusion features of each supporting image into the decoder to obtain the pixel-level segmentation result of the query image under the corresponding supporting image;

[0146] S255. The pixel-level segmentation results of the query image under each supporting image are weighted and fused to obtain the segmentation result of the query image. The weight of the pixel-level segmentation result of the query image under each supporting image is obtained by normalizing the average value of all elements of the similarity matrix between the query feature matrix and the support feature matrix of the corresponding supporting image.

[0147] Specifically, the attention fusion module performs the following operations:

[0148] 1) Map the support foreground features of each support image to a value matrix, then multiply it by the attention weight matrix of the corresponding calibration similarity matrix to obtain the cross-attention output features of the corresponding support image:

[0149]

[0150]

[0151] in, To support the cross-attention output features of the corresponding images, For the value matrix corresponding to the supported images, To support the foreground features of the corresponding supporting image, It is a 1×1 convolutional layer. For flattening operation.

[0152] 2) To preserve the structural information of the query image itself, higher-level query features are also added. Perform self-attention computation to obtain self-attention output features. The calculation is as follows:

[0153]

[0154]

[0155]

[0156]

[0157] in, For querying the matrix, The key matrix, For value matrices, To query the feature matrix, For learnable query mapping parameters, These are learnable key mapping parameters. For learnable value mapping parameters, For transpose, In this embodiment, the scaling factor is... , This refers to the Softmax function, which is a normalization operation. These are learnable mapping parameters that are automatically updated during training via backpropagation.

[0158] 3) Concatenate the cross-attention output features of each supporting image with the self-attention output features along the channel dimension, and then map them back to the high-level query features through a convolutional layer. With the same number of channels, after passing through a non-linear activation function, and then compared with high-level query features... Residual fusion is performed to obtain the fusion features of the corresponding supporting images, as shown in the following formula:

[0159]

[0160] in, To support the fusion features of the images, This indicates splicing along the channel dimension. It is a 1×1 convolutional layer. This represents the ReLU nonlinear activation function.

[0161] 4) Input the fused features of each supporting image into the decoder to obtain the pixel-level segmentation result of the query image under the corresponding supporting image. In this embodiment, the decoder adopts the U-Net model decoder, or the decoder may also adopt the FCN-style model decoder, the DeepLab model decoder, the SegFormer model decoder, or other semantic decoders.

[0162] 5) The pixel-level segmentation results of the query image under each supporting image are weighted and fused to obtain the segmentation result of the query image, as shown in the following formula:

[0163]

[0164]

[0165] in, To query the segmentation results of the image, For the first Zhang supports querying pixel-level segmentation results of images. Indicates the first Zhang supports the weighting of pixel-level segmentation results for the query image under the image. In this embodiment, From the query feature matrix and the first The similarity matrix of the support feature matrix of the supported image The average value of all elements is obtained by normalization. The higher the similarity between the supporting image and the query image, the better. The larger the value, the lower the similarity between the supported image and the query image. The smaller the value, the better. When the support set contains multiple support images, weights can be assigned according to different support samples. This weighted fusion method can make full use of reliable support information under multiple support samples and reduce the impact of abnormal support samples on the segmentation results of the query image.

[0166] It should be noted that the proportion-aware attention module of the present invention (including steps S22, S23, and S24, corresponding to the proportion estimation module, the proportion dynamic calibration module, and the perturbation enhancement module, respectively) can also be embedded in existing few-shot segmentation networks as an independent attention calibration unit. Specifically, the embedding method is as follows: after the existing few-shot segmentation network has extracted supporting foreground features and high-level query features, the processing flow of the proportion-aware attention module is executed. That is, the backbone network (deep feature extraction network, which can be a convolutional neural network, a Transformer network, or other semantic segmentation backbone network to obtain supporting foreground features and high-level query features) and decoder of the existing few-shot segmentation network can remain unchanged. The backbone network, the proportion-aware attention module, and the decoder are connected sequentially. The proportion-aware attention module of the present invention only performs dynamic calibration and background perturbation enhancement on the attention allocation between supporting foreground features and high-level query features to obtain an attention weight matrix. Based on each supporting foreground feature, each attention weight matrix, and the high-level query features, the attention fusion module is used to obtain the segmentation result of the query image. Furthermore, if existing few-shot segmentation networks have already calculated a similarity matrix based on supporting foreground features and high-level query features after the backbone network, then the proportion-aware attention module of this invention does not need to calculate the similarity matrix. It can directly calculate the soft prior mask based on the similarity matrix to obtain the foreground-background ratio, thereby completing dynamic thresholding, pseudo-background suppression, and perturbation enhancement. Therefore, the proportion-aware attention module of this invention can serve as a plug-and-play attention calibration unit to improve the problems of background weight amplification and foreground-background mixing in existing few-shot segmentation networks.

[0167] S3. Input the image dataset into the trained foreground-background ratio-aware segmentation model to obtain the segmentation results of the query image.

[0168] In one embodiment, the foreground-background ratio-aware segmentation model is trained using a training image set, which is divided into several support-query pairs. Each support-query pair includes a support set and a query set. The support set of each support-query pair includes several support images and corresponding masks, and the query set of each support-query pair includes a query image and a corresponding mask. The segmentation loss during training is obtained by using Dice loss or by weighted summation of Dice loss and cross-entropy loss.

[0169] Specifically, the foreground-background ratio-aware segmentation model is trained using a training image set to obtain a trained foreground-background ratio-aware segmentation model, as follows:

[0170] During the training phase, multiple few-shot segmentation tasks are sampled from the training image set. Each few-shot segmentation task includes a support set and a query set. The support set includes at least one support image and its corresponding mask, while the query set includes one query image and its corresponding mask. The support image and corresponding mask provide foreground cues to the model regarding the category to be segmented. The query image is used to generate the segmentation result. The mask of the query image is only used during the training phase to supervise loss calculation and ensures that the training and test categories do not overlap. The model predicts the segmentation result of the query image based on the support image and its corresponding mask, and then compares it with the mask of the query image to calculate the segmentation loss. After training, the model is deployed to servers, workstations, embedded devices, or edge computing devices for few-shot image segmentation.

[0171] In this embodiment, Dice loss is used. As segmentation loss The formula is as follows:

[0172]

[0173] in, This indicates the queried image number under the corresponding supported image. Pixel-level segmentation results for each pixel. Represents the first image in the query image The true label of the nth pixel, i.e., the nth pixel in the query image. A mask of 1 pixel, Represents the first image in the query image Each pixel belongs to the foreground. Represents the first image in the query image Each pixel belongs to the background. To avoid smoothing terms with a denominator of zero, the value is taken as... ~1, in this embodiment , , Indicates the height of the queried image. Indicates the width of the query image. This indicates the total number of pixels in the queried image.

[0174] Alternatively, the loss can be divided according to actual needs. For Dice's loss The weighted summation with the cross-entropy loss is used to adapt to different few-shot segmentation tasks, as shown in the following formula:

[0175]

[0176]

[0177] in, Represents cross-entropy loss, The weight coefficients representing the cross-entropy loss range from 0 to 1. In this embodiment... , This is a constant used to avoid logarithmic calculation errors. In this embodiment, .

[0178] The following is a detailed explanation based on a specific experiment.

[0179] During the training of the foreground-background ratio-aware segmentation model, only the attention fusion module is trained, while all other modules are frozen. =0.75, The total training rounds are 100. The optimizer can be the AdamW optimizer, the SGD optimizer, or a combination of both. In this embodiment, the attention fusion module uses the AdamW optimizer for updates, with a learning rate of [missing value]. to Preferred The weight decay coefficient can be taken as to The model allows for fine-tuning of hyperparameters and backbone network parameters individually to adapt to image segmentation tasks of different scales, demonstrating strong generalization ability. Local fine-tuning of the backbone network can be performed using the SGD optimizer, with a learning rate that can be set to a suitable value. to Preferred The momentum coefficient can be taken from 0.8 to 1.0, and the momentum coefficient in this embodiment is 0.9.

[0180] This experiment used the PASCAL-5i and COCO-20i datasets for validation. As shown in Tables 1 and 2, the segmentation evaluation metric was the mean Intersection over Union (mIoU). The backbone network was the deep feature extraction network, which in this experiment used ResNet-50 and ResNet-101 models respectively, and the decoder was the U-Net model. Figure 4 As shown, the image presents a comparison of the segmentation results of the present invention and the prior art under different proportions of the real foreground area. For example, the images in columns (a), (b), (c), and (d) correspond to the proportions of the real foreground area (obtained by statistically querying the foreground of the real label of the image) as 0%~25%, 26%~50%, 51%~75%, and 76%~100%, respectively.

[0181] (1) On the PASCAL-5i dataset, the existing methods (including PFENet, SRPNet, AFANet, QGNet, DCAMA and NTRENet) are compared with the method of this invention, as shown in Table 1.

[0182] Table 1 Performance comparison on the PASCAL-5i dataset (mIoU, %)

[0183]

[0184] In Table 1, bold text indicates the best results, and underlined text indicates the second-best results. Category 0 contains image samples of airplanes, bicycles, birds, boats, and bottles; Category 1 contains image samples of buses, cars, cats, chairs, and cows; Category 2 contains image samples of dining tables, dogs, horses, motorcycles, and people; and Category 3 contains image samples of potted plants, sheep, sofas, trains, and televisions (or monitors).

[0185] On the PASCAL-5i dataset, the method of this invention achieved high mean intersection-over-union (mIoU) ratios under different backbone networks and single-shot and five-shot settings. When using the ResNet-50 backbone network, the method of this invention achieved mean mIoU ratios of 67.0% and 69.6% in the 1-shot and 5-shot settings, respectively; the best comparison method in the same period, AFANet, achieved mIoU ratios of 64.0% and 66.2%, respectively. Therefore, the method of this invention improved the mIoU ratio by 3.0 and 3.4 percentage points in the 1-shot and 5-shot settings, respectively. In particular, for category 2, the method of this invention achieved a mean mIoU ratio of 66.5% in the 1-shot setting, which is 5.9 percentage points higher than the 60.6% of the AFANet method; and a mean mIoU ratio of 69.8% in the 5-shot setting, which is 8.5 percentage points higher than the 61.3% of the AFANet method. This indicates that the method of this invention has strong feature representation and target recognition capabilities for this category.

[0186] Even after adopting a stronger ResNet-101 model for the backbone network, the advantages of the proposed method remain stable. In the 1-shot setting, the average crossover and union ratio (CUP) of the proposed method is 67.5%, higher than the best comparison method (NTRENet) of 63.7%, representing an improvement of 3.8 percentage points. In the 5-shot setting, the average CUP of the proposed method reaches 71.3%, an improvement of 2.4 percentage points compared to the best comparison method (DCAMA) of 68.9%. Further observation of each category shows that the proposed method outperforms all comparison methods in all four categories under the 1-shot setting of the ResNet-101 model, and also achieves the highest results in all four categories under the 5-shot setting, demonstrating good inter-category stability.

[0187] The overall results on the PASCAL-5i dataset demonstrate that the advantage of the proposed method does not stem from a chance improvement in a single category, but rather from maintaining its superiority across different backbone networks, sample sizes, and multiple class divisions, resulting in high segmentation accuracy. This indicates that the method possesses good generalization ability and exhibits stable segmentation performance across different target categories.

[0188] (2) On the COCO-20i dataset, the existing methods (including PFENet, SCL, ABCNet, QGNet and NTRENet) are compared with the method of the present invention, as shown in Table 2.

[0189] Table 2 Performance comparison on the COCO-20i dataset (mIoU, %)

[0190]

[0191] In Table 2, bold text indicates the best results, and underlined text indicates the second-best results. Category 0 includes image samples of people, airplanes, boats, parking meters, dogs, elephants, backpacks, suitcases, sports balls, skateboards, wine glasses, spoons, sandwiches, hot dogs, chairs, dining tables, mice, microwave ovens, refrigerators, and scissors; Category 1 includes image samples of bicycles, buses, traffic lights, benches, horses, bears, umbrellas, frisbees, kites, surfboards, cups, bowls, oranges, pizzas, sofas, toilets, remote controls, ovens, books, and teddy bears; Category 2 includes image samples of cars, trains, fire hydrants, birds, sheep, zebras, handbags, skis, baseball bats, tennis rackets, forks, bananas, broccoli, donuts, potted plants, televisions, keyboards, toasters, clocks, and hair dryers; Category 3 includes image samples of motorcycles, trucks, stop signs, cats, cows, giraffes, ties, snowboards, baseball gloves, bottles, knives, apples, carrots, cakes, beds, laptops, mobile phones, sinks, vases, and toothbrushes.

[0192] Compared to the PASCAL-5i dataset, the COCO-20i dataset contains more categories and more complex target scenes, thus better demonstrating the generalization ability of few-shot segmentation methods under complex category distributions. When the backbone network uses the ResNet-50 model, the method of this invention achieves an average intersection-over-union (IoU) ratio of 42.9% in the 1-shot setting, which is 3.6 percentage points higher than the best comparison method (NTRENet) of 39.3%. In the 5-shot setting, the average IoU ratio of the method of this invention reaches 49.9%, which is a significant improvement of 9.6 percentage points compared to the NTRENet method of 40.3%.

[0193] The improvement in class-level accuracy under the 5-shot setting is particularly significant: Class 0 improved from 40.1% (ABCNet method) to 43.5%, an increase of 3.4 percentage points; Class 1 improved from 44.1% (NTRENet method) to 55.0%, an increase of 10.9 percentage points; Class 2 improved from 40.4% (NTRENet method) to 53.4%, an increase of 13.0 percentage points; and Class 3 improved from 38.4% (NTRENet method) to 47.3%, an increase of 8.9 percentage points. This demonstrates that the method of this invention can more fully utilize the effective information in the support set after introducing more support samples, especially when there are large class differences and complex changes in the appearance of the target.

[0194] The advantages were further validated when the backbone network adopted the ResNet-101 model. The method of this invention achieved an average crossover ratio (CVR) of 44.9% in the 1-shot setting, a 5.8 percentage point improvement compared to the best comparison method (NTRENet) of 39.1%; in the 5-shot setting, it achieved an average CVR of 52.6%, a 7.8 percentage point improvement compared to the best comparison method (QGNet) of 44.8%. Moreover, the method of this invention achieved the highest results in all four categories: in the 5-shot setting, the CVRs for the four categories were 47.7%, 55.6%, 56.1%, and 51.3%, respectively, representing improvements of approximately 5.4, 7.5, 9.8, and 7.7 percentage points compared to the corresponding best comparison results.

[0195] In the above table, the reference for the PFENet method: TIAN Z, ZHAO H, SHU M, et al. Priori-guided feature enrichment network for few-shot segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(2): 1050-1065. (Chinese translation: Tian Zhuotao, Zhao Hengshuang, Michel Shu, Yang Zhicheng, Li Ruiyu, Jia Jiaya. Prior-guided feature enrichment network for few-shot segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(2): 1050-1065.); The reference for the SRPNet method: DING H, ZHANG H, JIANG X. Self-regularized prototypical network for few-shot semantic segmentation[J]. Pattern Recognition, 2023, 133: 109018. (Chinese translation: Ding Henghui, Zhang Hui, Jiang Xudong. Self-regularized prototypical network for few-shot semantic segmentation[J]. Pattern Recognition, 2023, 133: 109018.); The reference for the AFANet method: MA J, XIE G S, ZHAO F, et al. AFANet: Adaptive frequency-aware network for weakly-supervised few-shot semantic segmentation[J]. IEEE Transactions on Multimedia, 2025, 27: 4018-4028. (Chinese translation: Ma Jiaqi, Xie Guosen, Zhao Fang, Li Zechao. AFANet: Adaptive frequency-aware network for weakly-supervised few-shot semantic segmentation[J]. IEEE Transactions on Multimedia, 2025, 27: 4018-4028.); The reference for the QGNet method: LIU W, WU Z, DING H, et al. Exploiting independent query information for few-shot image segmentation[J]. Displays, 2026, 91: 103179. (Chinese translation: Liu Weide, Wu Zhonghua, Ding Henghui, Liu Fuyao, Lin Jie, Lin Guosheng, Zhou Wei. Exploiting independent query information for few-shot image segmentation[J].Display Technology, 2026, 91: 103179.); Reference for DCAMA method: SHI X, WEI D, ZHANG Y, et al. Dense cross-query-and-support attention weighted mask aggregation for few-shot segmentation[C] / / Computer Vision – ECCV 2022. Cham: Springer, 2022: 151-168. (Chinese Translation: Shi Xinyu, Wei Dong, Zhang Yu, Lu Donghuan, Ning Munan, Chen Jiashun, Ma Kai, Zheng Yefeng. Dense Query-Support Cross-Attention Weighted Mask Aggregation for Few-Shot Segmentation[C] / / Computer Vision — ECCV 2022: Proceedings of the 17th European Conference on Computer Vision. Cham, Switzerland: Springer, 2022: 151-168.); Reference for NTRENet method: LIU Y, LIU N, CAO Q, et al. Learning non-target knowledge for few-shot semantic segmentation[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 11573-11582. (Chinese Translation: Liu Yuanwei, Liu Nian, Cao Qinglong, Yao Xiwen, Han Junwei, Shao Ling. Non-Target Knowledge Learning for Few-Shot Semantic Segmentation[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 11573-11582.); Reference for ABCNet method: WANG Y, SUN R, ZHANG T. Rethinking the Correlation in Few-Shot Segmentation: A Buoys View[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2023: 7183-7192. (Chinese Translation: Wang Yuan, Sun Rui, Zhang Tianzhu. Rethinking Correlation in Few-Shot Segmentation from a Buoy Perspective[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2023: 7183-7192.); Reference for SCL method: ZHANG B, XIAO J, QIN T.Self-Guided and Cross-Guided Learning for Few-Shot Segmentation [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 8308-8317. (Chinese translation: Zhang Bingfeng, Xiao Jimin, Qin Yinghua. Self-Guided and Cross-Guided Learning for Few-Shot Segmentation [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 8308-8317.)

[0196] In summary, the foreground-background ratio-aware segmentation model of this invention first extracts high-level supporting features and high-level query features through a deep feature extraction network, and then obtains supporting foreground features based on the mask of the supporting image. Subsequently, each supporting foreground feature is flattened along the spatial dimension into a corresponding supporting feature matrix, and the high-level query features are flattened along the spatial dimension into a query feature matrix. The similarity between the query feature matrix and each supporting feature matrix is ​​calculated to obtain the corresponding similarity matrix, and a soft prior mask for each similarity matrix is ​​obtained. Based on the soft prior masks of each similarity matrix, the proportion of the defect region (target) in the query image (foreground-background ratio) is estimated, and this proportion is used to determine the dynamic threshold of the similarity matrix. Regions with high similarity are designated as pseudo-defect regions (pseudo-foreground regions), and regions with low similarity are designated as pseudo-background regions. Attention scores in pseudo-background regions are suppressed, and small random perturbations are introduced into the pseudo-background regions. After normalization, an attention weight matrix is ​​formed, which is used to aggregate supporting foreground features. Finally, based on each supporting foreground feature, each attention weight matrix, and the high-level query features, an attention fusion module is used to obtain the segmentation result of the query image. It can complete the segmentation of new defect categories with a small number of defect-labeled samples and reduce the influence of background texture on defect region judgment. For images to be segmented with small defect areas, indistinct boundaries, or complex background structures, the dynamic threshold can adjust the attention distribution according to the prior area of ​​defects in the query image, thereby reducing the interference of background response on the prediction results.

[0197] Therefore, this invention estimates the foreground-to-background ratio of the query image using a soft prior mask and uses this ratio for dynamic calibration of the similarity matrix. Simultaneously, controlled random perturbation enhancement is added to the pseudo-background region to balance background suppression and context preservation. This can alleviate the problems of foreground information loss and background weight amplification in few-sample image segmentation tasks and can be applied to image segmentation tasks in various few-label scenarios, demonstrating strong generalization ability.

[0198] Example 2:

[0199] A few-shot image segmentation system based on foreground-background ratio awareness includes a data acquisition module for acquiring an image dataset and dividing it into a support set and a query set. The support set includes several support images and corresponding masks, while the query set includes the query image, i.e., the image to be segmented.

[0200] The model building module is used to build a foreground-background ratio-aware segmentation model, including:

[0201] The feature extraction module is used to extract supporting foreground features and high-level query features based on the support set and query set using a deep feature extraction network;

[0202] The ratio estimation module is used to flatten each supporting foreground feature along the spatial dimension into a corresponding supporting feature matrix, flatten the high-level query features along the spatial dimension into a query feature matrix, calculate the similarity between the query feature matrix and each supporting feature matrix to obtain the corresponding similarity matrix, and obtain the foreground-background ratio of the query image under the corresponding supporting image based on each similarity matrix.

[0203] The proportional dynamic calibration module is used to dynamically calibrate the corresponding similarity matrix based on the foreground-background ratio of the query image under each supported image, and obtain the corresponding calibration similarity matrix.

[0204] The perturbation enhancement module is used to randomly perturb and enhance each calibration similarity matrix to obtain the corresponding attention score matrix, and then normalize each attention score matrix to form an attention weight matrix.

[0205] The feature fusion module is used to obtain the segmentation result of the query image by using the attention fusion module based on each supporting foreground feature, each attention weight matrix and high-level query features;

[0206] The output module is used to input the image dataset into the trained foreground-background ratio-aware segmentation model to obtain the segmentation results of the query image.

[0207] The system comprises the following modules: a data acquisition module, a query image, and a support image; a feature extraction module, and a scale estimation module. The support image and its corresponding mask, along with a query image, are used to acquire the foreground and high-level query features from the support and query images. The scale estimation module generates a soft prior mask for the query image and calculates the foreground-background ratio. The scale dynamic calibration module determines a dynamic threshold based on the foreground-background ratio to segment the pseudo-foreground and pseudo-background regions. The perturbation enhancement module injects controlled random perturbations into the pseudo-background regions. The feature fusion module fuses cross-attention output features, self-attention output features, and high-level query features to obtain the fused features of the corresponding support images and outputs the pixel-level segmentation result of the query image under the corresponding support images. Weighted fusion is then used to obtain the segmentation result of the query image. The model training module updates the model parameters based on the training image set. The result output module inputs the image dataset into the trained foreground-background ratio-aware segmentation model to obtain the segmentation result of the query image, which may include a segmentation mask and a corresponding visualization image.

[0208] In practical applications, users can input a small number of labeled samples as a support set into a few-shot image segmentation system based on foreground-background ratio awareness, and then input the image to be segmented. The few-shot image segmentation system based on foreground-background ratio awareness automatically outputs the segmentation results for the query image. For new categories of samples such as industrial defects and medical images, only a small number of masked images are needed to complete the segmentation of the target region.

[0209] Specific limitations regarding the few-shot image segmentation system based on foreground-background ratio awareness can be found in Example 1 regarding the limitations of the few-shot image segmentation method based on foreground-background ratio awareness, and will not be repeated here. Each module in the aforementioned few-shot image segmentation system based on foreground-background ratio awareness can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0210] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor. By running the computer program stored in the memory, the processor implements the few-sample image segmentation method based on foreground-background ratio perception in Embodiment 1 of the present invention.

[0211] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). The memory stores computer programs, and the processor executes these programs after receiving execution instructions.

[0212] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in Embodiment 1 of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0213] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0214] The embodiments described above are merely specific and detailed examples of the embodiments described in this application, and should not be construed as limiting the scope of the application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A few-shot image segmentation method based on foreground-background ratio awareness, characterized in that: Includes the following steps: S1. Obtain the image dataset and divide it into a support set and a query set. The support set includes several support images and corresponding masks, and the query set includes the query image, i.e. the image to be segmented. S2. Construct a foreground-background ratio-aware segmentation model to perform the following operations: S21. Extract supporting foreground features and high-level query features using a deep feature extraction network based on the support set and query set; S22. Flatten each supporting foreground feature along the spatial dimension into a corresponding supporting feature matrix, flatten the high-level query feature along the spatial dimension into a query feature matrix, and calculate the similarity between the query feature matrix and each supporting feature matrix to obtain the corresponding similarity matrix. Based on each similarity matrix, obtain the foreground-background ratio of the query image under the corresponding supporting image. S23. Based on the foreground-background ratio of the query image under each supporting image, perform dynamic calibration of the corresponding similarity matrix to obtain the corresponding calibration similarity matrix; S24. Randomly perturb and enhance each calibration similarity matrix to obtain the corresponding attention score matrix, and normalize each attention score matrix to form an attention weight matrix. S25. Based on each supporting foreground feature, each attention weight matrix, and high-level query features, the attention fusion module is used to obtain the segmentation result of the query image; S3. Input the image dataset into the trained foreground-background ratio-aware segmentation model to obtain the segmentation results of the query image.

2. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 1, characterized in that: The deep feature extraction network adopts one of the following models: ResNet, ConvNeXt, Vision Transformer, or SwinTransformer.

3. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 1, characterized in that: The extraction of support foreground features and high-level query features using a deep feature extraction network based on the support set and query set is as follows: Each support image in the support set is input into a deep feature extraction network to obtain the corresponding high-level support features. Then, the mask of each support image in the support set is transformed in dimension and multiplied element-wise with the corresponding high-level support features to obtain the corresponding support foreground features. The query images in the query set are input into a deep feature extraction network to obtain high-level query features.

4. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 1, characterized in that: The foreground-to-background ratio of the query image under the corresponding supporting image is obtained based on each similarity matrix, as follows: S221. Obtain the soft prior mask for each similarity matrix. Specifically, average the similarity matrix along its column dimensions to obtain the activation score vector of the corresponding query feature matrix. Obtain the corresponding soft prior mask based on the activation score vector of the query feature matrix, calculated as follows: in, Indicates the query feature matrix and the first The soft prior mask of the similarity matrix of the support feature matrix of the supported image. Indicates the query feature matrix and the first The similarity matrix of the support feature matrix of the supported image The corresponding activation score vector of the query feature matrix. This indicates that the query has been converted to the same spatial size as the high-level query features. This represents the sigmoid function. Indicates temperature factor, , Indicates the number of images supported in the support set; S222. Calculate the product of the spatial average of the soft prior mask of each similarity matrix and the scaling parameter as the foreground-background ratio of the query image under the corresponding support image. The scaling parameter... The value range is 0 to 1.

5. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 4, characterized in that: The dynamic calibration of the corresponding similarity matrix is ​​performed based on the foreground-to-background ratio of the query image under each supporting image to obtain the corresponding calibrated similarity matrix, as detailed below: Flatten each similarity matrix into a one-dimensional similarity sequence, then sort the elements in the one-dimensional similarity sequence according to the size of the similarity, and use the similarity of the foreground-background ratio of the query image under the corresponding support image as the dynamic threshold. The position with similarity greater than or equal to the dynamic threshold is recorded as the pseudo-foreground region, and the position with similarity less than the dynamic threshold is recorded as the pseudo-background region. A calibration similarity matrix is ​​established based on the dynamic threshold and the corresponding similarity matrix. Then, the first... Line number The elements of the column are represented as follows: in, To calibrate the similarity matrix The Middle Line number Column elements, For the corresponding similarity matrix, the first... Line number Column elements, , , This indicates the number of elements in the query feature matrix. This indicates the number of elements in the corresponding supporting feature matrix. For the corresponding dynamic threshold, This is a preset constant, and its value range is [value range missing]. ~ .

6. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 5, characterized in that: The attention score matrix is ​​obtained by randomly perturbing each calibration similarity matrix. Each attention score matrix is ​​then normalized to form an attention weight matrix, as shown in the following formula: Among them, the attention score matrix corresponds to the calibration similarity matrix. The Middle Line number Column elements The calculation is as follows: in, The attention weight matrix corresponding to the calibration similarity matrix, This indicates normalization along the column dimension. Indicates the first element in the corresponding calibration similarity matrix. Line number Disturbance noise of column elements, The value represents the disturbance intensity, ranging from 0.001 to 1.

0.

7. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 6, characterized in that: The proportional adjustment parameter and the disturbance intensity are obtained using one of the following methods: 1) The proportional adjustment parameter is a first preset fixed value, and the disturbance intensity is a second preset fixed value; 2) Both the proportional adjustment parameter and the perturbation intensity are adaptively adjusted according to the training process, that is, the proportional adjustment parameter increases with the training rounds, and the perturbation intensity decreases with the training rounds. 3) Both the scaling parameter and the perturbation strength are adaptively adjusted based on the prior mask confidence level. Specifically, the prior mask confidence level is calculated based on the entropy of each soft prior mask. When the prior mask confidence level is less than the first confidence threshold, the scaling parameter is the first scaling threshold and the perturbation strength is the first perturbation threshold. When the prior mask confidence level is greater than or equal to the first confidence threshold and less than the second confidence threshold, the scaling parameter is the second scaling threshold and the perturbation strength is the second perturbation threshold. When the prior mask confidence level is greater than or equal to the second confidence threshold, the scaling parameter is the third scaling threshold and the perturbation strength is the third perturbation threshold. The first scaling threshold, the second scaling threshold, and the third scaling threshold increase sequentially, as do the first perturbation threshold, the second perturbation threshold, and the third perturbation threshold. 4) The proportional adjustment parameter is adaptively adjusted according to the target area ratio, that is, it is positively correlated with the target area ratio. The disturbance intensity is dynamically adjusted according to the area ratio of the pseudo-background region, that is, it is negatively correlated with the area ratio of the pseudo-background region.

8. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 1, characterized in that: The attention fusion module performs the following operations: S251. After mapping each supporting foreground feature into a value matrix, multiply it by the attention weight matrix of the corresponding calibration similarity matrix to obtain the cross-attention output features of the corresponding supporting image. S252. Perform self-attention calculation on the high-level query features to obtain the self-attention output features; S253. The cross-attention output features of each support image are concatenated with the self-attention output features along the channel dimension, and mapped back to the same number of channels as the high-level query features through a convolutional layer. After passing through a non-linear activation function, residual fusion is performed with the high-level query features to obtain the fused features of the corresponding support images. S254. Input the fusion features of each supporting image into the decoder to obtain the pixel-level segmentation result of the query image under the corresponding supporting image; S255. The pixel-level segmentation results of the query image under each supporting image are weighted and fused to obtain the segmentation result of the query image. The weight of the pixel-level segmentation result of the query image under each supporting image is obtained by normalizing the average value of all elements of the similarity matrix between the query feature matrix and the support feature matrix of the corresponding supporting image.

9. The few-shot image segmentation method based on foreground-background ratio awareness as described in claim 1, characterized in that: The foreground-background ratio-aware segmentation model is trained using a training image set, which is divided into several support-query pairs. Each support-query pair includes a support set and a query set. The support set of each support-query pair includes several support images and corresponding masks, and the query set of each support-query pair includes a query image and a corresponding mask. The segmentation loss during training is obtained by using Dice loss or by weighted summation of Dice loss and cross-entropy loss.

10. A few-shot image segmentation system based on foreground-background ratio awareness, characterized in that: include: The data acquisition module is used to acquire an image dataset and divide it into a support set and a query set. The support set includes several support images and corresponding masks, and the query set includes a query image, i.e., the image to be segmented. The model building module is used to build a foreground-background ratio-aware segmentation model, including: The feature extraction module is used to extract supporting foreground features and high-level query features based on the support set and query set using a deep feature extraction network; The ratio estimation module is used to flatten each supporting foreground feature along the spatial dimension into a corresponding supporting feature matrix, flatten the high-level query features along the spatial dimension into a query feature matrix, calculate the similarity between the query feature matrix and each supporting feature matrix to obtain the corresponding similarity matrix, and obtain the foreground-background ratio of the query image under the corresponding supporting image based on each similarity matrix. The proportional dynamic calibration module is used to dynamically calibrate the corresponding similarity matrix based on the foreground-background ratio of the query image under each supported image, and obtain the corresponding calibration similarity matrix. The perturbation enhancement module is used to randomly perturb and enhance each calibration similarity matrix to obtain the corresponding attention score matrix, and then normalize each attention score matrix to form an attention weight matrix. The feature fusion module is used to obtain the segmentation result of the query image by using the attention fusion module based on each supporting foreground feature, each attention weight matrix and high-level query features; The output module is used to input the image dataset into the trained foreground-background ratio-aware segmentation model to obtain the segmentation results of the query image.

Images