Priori mask optimization method based on similarity measurement

Abstract

The invention discloses a priori mask optimization method based on similarity measurement. The overall process is as follows: foreground mask extraction based on a fusion prototype, background mask extraction based on a gating mechanism, and foreground mask optimization based on loop feedback. A trainable weighted fusion channel is established between a support prototype and a query prototype, the semantic deviation problem of traditional single prototype matching is solved, a background probability confidence coefficient matrix and cyclic consistency check are combined, cross-sample positioning of false positive prediction is achieved, temperature parameters are dynamically adjusted in the mask correction stage, and the accuracy of the temperature correction is improved. Compared with a fixed temperature strategy, the method is more suitable for confidence distribution characteristics of complex scenes; and the segmentation mask is optimized by fusing prototype similarity measurement and a cyclic feedback mechanism, so that false positive prediction is effectively reduced, and the distinguishing precision of the foreground and the background is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of few-shot learning, and specifically provides a prior mask optimization method based on similarity measurement. Background Art

[0002] In recent years, deep learning has made remarkable progress in the field of semantic segmentation. In particular, fully supervised methods (such as FCN, DeepLab series) rely on large-scale labeled data to achieve high-precision segmentation. However, in practical scenarios such as medical image analysis and autonomous driving scene understanding, obtaining a large amount of pixel-level labeled data is costly and time-consuming, giving rise to the application demand of few-shot learning in image segmentation. The core of such methods is to guide the segmentation of new scenes (query sets) through a small number of labeled samples (support sets). However, the existing technologies still face the following challenges: (1) Limitations of prototype matching: Traditional prototype networks extract prototype features of the support set through average pooling. However, complex background interference easily leads to blurred prototype representations. Especially when the support samples contain noise or the foreground regions are sparse, the prototype offset problem is significant, reducing the segmentation robustness.

[0003] (2) Insufficient handling of background complexity: Existing methods (such as PANet, PFENet) often regard the background as a single category, ignoring the multi-modal distribution characteristics of the background regions in real scenes, resulting in calculation biases in background similarity and causing misjudgments of the foreground.

[0004] (3) Cumulative false positive predictions: During the iterative optimization process, mis-matched pixels continue to propagate due to the lack of a feedback mechanism. For example, foreground regions in the query image that are similar to the support background features are easily mis-suppressed, forming segmentation holes. Summary of the Invention

[0005] The purpose of the present invention is to provide a prior mask optimization method based on similarity measurement to solve the problems raised in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A prior mask optimization method based on similarity measurement, including the following steps: Step 1, foreground mask extraction based on fused prototypes. Adjust the support set mask to the feature map size, extract foreground prototypes and background prototypes through average pooling, fill with zero vectors when there is no foreground, perform L 2 normalization on the query features and the support prototypes, calculate the preliminary foreground cosine and background cosine similarities, generate a preliminary confidence mask through temperature coefficient soft pooling, use the preliminary mask to perform weighted pooling on the query features to extract the query foreground prototype, and re-normalize after dynamically weighted fusion with the support prototype to generate a fused foreground confidence mask; Step 2: Generate a background prototype by weighted pooling the query features. After L normalization, multiply it with the query feature matrix to generate a background similarity probability matrix. Use the probability matrix as a gate to correct the preliminary background confidence. Finally, concatenate the foreground mask and the background mask along the channel and output. Step 3: Generate a query binary mask through softmax, divide the foreground features and background features, calculate their similarity with the support features, generate a reverse support prediction mask, compare the reverse mask with the original support background area, identify inconsistent areas to generate a subtraction mask, adaptively reduce the false positive confidence and enhance background supplementation, and concatenate the corrected masks as the final segmentation result.

[0007] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Bidirectional prototype fusion: Establish a trainable weighted fusion channel between the support prototype and the query prototype, overcoming the semantic deviation problem of traditional single-prototype matching.

[0008] (2) Gating-feedback coupling mechanism: Combine the background probability confidence matrix with cyclic consistency checking to achieve cross-sample localization of false positive predictions.

[0009] (3) Adaptive temperature adjustment: Dynamically adjust the temperature parameter in the mask correction stage, which is more adaptable to the confidence distribution characteristics of complex scenarios compared with the fixed temperature strategy. Description of the Drawings

[0010] Figure 1 is the flowchart of the present invention; Figure 2 is the effect diagram of the present invention. Detailed Embodiment

[0011] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0012] Embodiment: Please refer to Figure 1 , the present invention provides a technical solution: A prior mask optimization method based on similarity measurement, and the overall process is as follows: 1.1 Foreground mask extraction Adjust the size of the support mask → Calculate the support foreground / background prototype → Calculate the similarity mask → Generate the fused foreground prototype → Calculate the fused similarity mask; 1.2 Background mask extraction Computational query background prototype → Computational background gating matrix → Generate background similarity mask; 1.3 Foreground mask optimization Generate query foreground region → Compute reverse support prediction mask → Cycle consistency check → Generate query subtraction mask → Adjust mask foreground / background confidence; Input support features, support mask, query features, output query prior mask.

[0013] 2.1 Foreground mask extraction based on fused prototype For a given support feature ( represents the number of channels, , represent the length and width of the feature size respectively) and its binary mask (the mask element values are 0 or 1, and the size is ), first adjust the foreground mask to the feature map size through bilinear interpolation (that is, adjust the binary mask to the feature size). Define the foreground region and the background region , where represents the feature pixel coordinates. Use the average mask pooling operation to extract the support feature foreground prototype and the background prototype . At the same time, to avoid numerical instability problems caused by empty prototypes, this method introduces an indicator function in the calculation process of the query foreground prototype , where is the number of feature channels, is the zero vector: (2.1) (2.2) This method normalizes the query feature and the support foreground prototype respectively to obtain and and , and use the traditional cosine similarity to calculate the preliminary query foreground similarity and the preliminary query background similarity , where represents matrix transpose: (2.3) (2.4) Through the soft pooling operation introducing the temperature coefficient for the preliminary query foreground similarity And the similarity with the preliminary query background Perform probability distribution enhancement to obtain the preliminary query foreground confidence mask respectively And the preliminary query background confidence mask : (2.5) Utilize the preliminary query foreground confidence mask through weighted mask pooling operation And the query feature Extract the foreground prototype of the query feature , where Prevent division-by-zero errors, Indicates the query feature value with pixel coordinates , Indicates the query foreground confidence corresponding to the coordinate, Represents the per-pixel accumulation operation, and : (2.6) With trainable parameter ( )Fuse the support foreground prototype by adding weights And the query foreground prototype , and perform on this fused foreground prototype L 2 normalization to obtain .

[0014] (2.7) Consistent with formulas (2.3) and (2.5), use the normalized fused foreground prototype And the normalized query feature Calculate the fused query foreground similarity , and then obtain the query foreground confidence mask and query background confidence mask based on the fused foreground prototype through the soft pooling operation with temperature coefficient , where And the query background confidence mask , where Represents the transpose of the fused foreground prototype

[0015] (2.8) (2.9) 2.2 Background Mask Extraction Based on Gating Mechanism To address the problem of complex background region environments, this method uses the query background prototype to refine the query background confidence mask . First, utilize the query background confidence mask through weighted mask pooling operation And the query feature Obtain the query background prototype , and perform L 2 normalization on this background prototype to obtain the normalized query background prototype , where to prevent division by zero errors, represents the query eigenvalue with pixel coordinates , represents the query background confidence at the corresponding coordinates, and .

[0016] (2.10) Immediately use the normalized query background prototype and the normalized query feature to screen the background possibility of the query feature in the form of a matrix product, and normalize the screening result to obtain the background similarity probability matrix , where to prevent division by zero errors, represents the transpose of the background prototype.

[0017] (2.11) Using the background similarity probability matrix as a gate, perform probability adjustment on the query background confidence mask to obtain the query background similarity mask based on gate adjustment .

[0018] (2.12) Finally, concatenate the query background similarity mask and the query foreground confidence mask along the channel as the overall query prediction mask for output.

[0019] (2.13) 2.3 Foreground Mask Optimization Based on Circular Feedback For the given query feature , support feature and query prediction mask , first obtain the query binary mask through the softmax operation, where the query foreground region , and the query background region . The query feature can be divided into the query foreground feature and the query background feature through this binary mask.

[0020] (2.14) Subsequently, calculate the normalized query foreground features and the normalized support features for similarity , and for each pixel coordinate predicted to be in the query foreground region , find the most similar position in the normalized support features to obtain the inverted support prediction mask , where represents the transpose of the normalized foreground features.

[0021] (2.15) (2.16) After obtaining the inverted support prediction mask , check each of the most similar positions to see if it is located in the support background region , identify potential inconsistent predictions, and generate a query subtraction mask to identify the positions where confidence operations are needed.

[0022] (2.17) To achieve smoothing of the prediction mask, the method corrects the query mask by setting a temperature parameter ( ), using adaptive mean subtraction to reduce the confidence of false positive predictions and increase the background confidence in the complementary region.

[0023] (2.18) (2.19) where and are the means of the foreground and background probabilities respectively, , represent the foreground probability and background probability of the query prediction mask respectively, , represent the corrected foreground probability and background probability respectively. Finally, the corrected prediction masks are concatenated along the channels to obtain the final optimized output .

[0024] (2.20) Please refer to Figure 2 , the present invention specifies the fusion parameter and the temperature parameter The intuitive effects before and after the optimization of the similarity mask when both are 0.5 can be seen. It can be seen that steps 2.1 and 2.2 can effectively enhance the expression of the query foreground type, while effectively distinguishing the background area, and the boundary between the foreground and the background is more accurate; step 2.3 can effectively suppress the background expression in the foreground confidence area, enhance the confidence of the foreground area, and reduce false positive predictions.

[0025] In this embodiment, a small number of trainable / fine-tunable prototype fusion weights and temperature control coefficients replace the empirical weight setting in the traditional algorithm, and the softmax operation is used to distinguish the foreground / background areas of the query image, enhancing the adaptability to new domain tasks.

[0026] In this embodiment, a background confidence gating mechanism is constructed based on the query background prototype, replacing the dense attention mechanism for background pixels in the traditional method, and significantly reducing the background recognition deviation caused by noise, inter-class similarity, and feature distribution.

[0027] In this embodiment, through a three-stage linkage mechanism of the reverse support prediction mask, the feedback identification mask, and the query subtraction mask, a pixel-level bidirectional closed loop from the query space to the support space and then back is constructed, realizing a cyclic matching strategy for cross-space consistency verification.

[0028] In this embodiment, an adaptive mean smoothing and dynamic confidence compensation mechanism for the foreground confidence through cross-space cyclic matching replaces the traditional hard truncation strategy, suppressing noise interference while retaining effective features.

[0029] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A prior mask optimization method based on similarity measurement, characterized in that , including the following steps: Step 1, foreground mask extraction based on the fusion prototype. Adjust the support set mask to the feature map size, extract the foreground prototype and background prototype through average pooling. When the foreground is empty, fill it with a zero vector. Perform L 2 normalization on the query feature and the support prototype, calculate the preliminary foreground cosine and background cosine similarities, generate the preliminary confidence mask through temperature coefficient soft pooling, use the preliminary mask to perform weighted pooling on the query feature to extract the query foreground prototype, and re-normalize after dynamically weighted fusion with the support prototype to generate the fused foreground confidence mask; Step 2: Generate a background prototype from the weighted pooling query features. After L 2 normalization, multiply it with the query feature matrix to generate a background similarity probability matrix. Use the probability matrix as a gate to correct the preliminary background confidence. Finally, concatenate the foreground mask and the background mask along the channels and output. Step 3: Generate a query binary mask through softmax, divide foreground features and background features, calculate their similarities with support features, generate a reverse support prediction mask, compare the reverse mask with the original support background region, identify inconsistent regions to generate a subtraction mask, adaptively reduce the false positive confidence and enhance background supplementation, and splice the corrected masks into the final segmentation result.

2. The prior mask optimization method based on similarity measurement according to claim 1, wherein: In Step 1, , where is the background prototype, is the background area, is the given support feature, is the coordinate of the feature pixel point; , where is the foreground prototype, is the foreground area, is the zero vector.

3. The prior mask optimization method based on similarity measurement according to claim 2, wherein: In step one, , where is the preliminary query foreground confidence mask, is the preliminary query foreground confidence mask, is the temperature coefficient of soft pooling, is the preliminary query foreground similarity, is the preliminary query background similarity; , where is the foreground prototype of the query feature, to prevent division-by-zero errors, is the pixel coordinate of the query feature value, is the query foreground confidence of the corresponding coordinate, represents a per-pixel accumulation operation.

4. A prior mask optimization method based on similarity measurement according to claim 3, characterized in that: In step one, , where is to query the foreground similarity, is to fuse the foreground prototype and the query feature for normalization , represents the transpose of the fused foreground prototype; , where is the query background confidence mask, is the query foreground confidence mask.

5. A prior mask optimization method based on similarity measurement according to claim 1, characterized in that: In step two, , where is to query the background prototype,[[]] is the pixel coordinate of the query eigenvalue; , where is the query background similarity mask, is the background similarity probability matrix, is the query background confidence mask.

6. A prior mask optimization method based on similarity measurement according to claim 5, characterized in that: In step two, , where is the query prediction mask, is the refined query background similarity mask.

7. A prior mask optimization method based on similarity measurement according to claim 1, characterized in that: In step three, , where is a reverse support prediction mask, is a query subtraction mask, is a support background area; , , where and are the means of the foreground probability and the background probability respectively, , represent the foreground probability and the background probability of the query prediction mask respectively, , represent the corrected foreground probability and the background probability respectively.

8. A prior mask optimization method based on similarity measurement according to claim 7, characterized in that: In step three, , where is the optimized output.

Images