An interactive saliency detection method and system

Abstract

The application discloses an interactive saliency detection method and system. The detection method comprises the following steps: acquiring a light field image, acquiring a full-focus image and a focus stack of a target scene based on the light field image, acquiring multi-scale multi-focus plane features based on the acquired focus stack, performing weighted aggregation based on the acquired multi-focus plane features to acquire a focus representation aligned with a full-focus feature, acquiring a fusion feature, encoding a user point / box / ink interaction prompt into a prompt feature based on the user point / box / ink interaction prompt, acquiring an enhanced feature, performing gradient feature extraction based on the enhanced feature to acquire a gating weight, and performing boundary enhancement based on the gradient gating. Through effective processing of the focus stack and the full-focus image, the application outputs a saliency map or a segmentation result through multi-scale decoding, and realizes improvement of boundary consistency and robustness and reduction of interaction cost.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a saliency detection method and system based on full-focus images, focus stack data, and user interaction prompts. Background Technology

[0002] Saliency detection has significant application value in scenarios such as industrial defect area localization, anomaly area extraction, intelligent quality inspection, and auxiliary annotation. Most existing saliency detection methods use a single image as input, primarily relying on texture and semantic information for target region identification. In complex industrial scenarios, when the target area is similar in appearance to the background area, or when the target boundary is weak or thin, false positives, false negatives, and problems such as boundary discontinuities and loss of detail easily occur. Furthermore, while existing interactive methods introduce user interaction information such as point hints, box hints, and stroke hints, the integration of interactive hints and visual features is still insufficient, resulting in the need for numerous interactive operations to obtain accurate detection results.

[0003] Focus stack data is an image sequence composed of multiple images acquired at different focal planes. It can provide richer cues of focal length and depth-of-field variations than a single image, aiding in the differentiation of foreground and background regions. However, focus stack data differs from full-focus images in feature representation and information organization. How to effectively integrate focus stack data with full-focus images and further collaborate with user interaction prompts to improve saliency detection accuracy and boundary quality in complex industrial scenarios remains a pressing technical problem to be solved in the current technology. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] This invention provides an interactive focusing stack saliency detection method, comprising the following steps:

[0006] Acquire light field images;

[0007] Based on the light field image, obtain the full-focus image of the target scene and focus stack data;

[0008] Based on the full-focus image and the focus stack data, obtain the multi-scale image features of the full-focus image and the multi-focal plane features of the focus stack data;

[0009] Based on the multi-focal plane features, a focus representation aligned with the multi-scale image features is obtained;

[0010] Based on the multi-scale image features and the focus representation, fusion features are obtained;

[0011] Based on user interaction prompts, the fused features are encoded with prompts and subjected to dynamic attention interaction to obtain interaction enhancement features. The user interaction prompts include at least one of point prompts, box prompts, and stroke prompts.

[0012] Gradient features are extracted based on the interaction enhancement features to obtain gating weights;

[0013] Based on the gating weights, the interaction enhancement features are subjected to boundary enhancement to obtain boundary enhancement features;

[0014] Based on the boundary enhancement features, semantic decoupling is performed to generate foreground region representation, background region representation, and uncertain region representation;

[0015] Multi-scale decoding is performed based on the boundary enhancement features, foreground region representation, background region representation, and uncertain region representation to output a saliency map with the same spatial size as the fully focused image.

[0016] The organization of the focus stack data includes: grouping the focus stack data according to the focal plane order and reconstructing it into a focal plane sequence tensor, so that each focal plane image is input into the feature extraction network as a sequence element, or mapping each focal plane image into a channel stack form according to a predetermined rule and inputting it into the feature extraction network.

[0017] Feature extraction for full-focus images and focus stack data employs a first backbone network and a second backbone network, respectively, with the first backbone network and the second backbone network outputting at least two levels of multi-scale features.

[0018] The step of obtaining a focus representation aligned with the multi-scale image features based on the multi-focal plane features includes: aggregating the multi-focal plane features using average aggregation or attention-weighted aggregation; wherein, the attention-weighted aggregation includes: generating corresponding focal plane weights based on each focal plane feature, and performing weighted summation on the multi-focal plane features based on the focal plane weights.

[0019] The step of obtaining fused features based on the multi-scale image features and the focus representation includes: splicing and fusing the full-focus image features and the focus representation at the same scale, or adding and fusing them together, and adaptively weighting the fused features through a channel attention mechanism.

[0020] The user interaction prompt information includes at least one of dot prompts, box prompts, and stroke prompts; the encoding of the user interaction prompt information includes mapping the user interaction prompt information into a prompt feature map with the same size as the fused feature space.

[0021] The dynamic attention interaction adopts a multi-head attention mechanism, and uses the fused features as the query vector and the prompt features as the key vector and value vector to achieve feature updates guided by prompts.

[0022] The multi-head attention mechanism involves mapping fused visual features and cue features to multiple independent feature subspaces. Within each subspace, the correlation between the query vector and the key vector is calculated. The value vectors are then weighted and aggregated based on this correlation. Finally, the weighted aggregation results from each feature subspace are concatenated and mapped to obtain the attention interaction result. By performing feature interactions within multiple feature subspaces, the correlation information between fused visual features and user interaction cuees can be extracted from different representational dimensions.

[0023] The term "dynamic" refers to the fact that attention weights are not preset or fixed, but are adaptively calculated based on the currently fused visual features and current prompt features. When the input image content, user prompt type, prompt location, prompt range, or prompt semantics change, the corresponding query vector, key vector, and value vector change accordingly, thereby dynamically updating the attention weights and feature responses at each spatial location. This enhances the response in areas relevant to the user prompt and suppresses the response in areas unrelated to the user prompt.

[0024] The step of extracting gradient features based on the interaction enhancement features and obtaining gating weights based on the gradient features includes: calculating gradient features on the interaction enhancement features, generating gating weights based on the gradient features, and using the gating weights to weight the interaction enhancement features to enhance the target boundary.

[0025] The generation of the gating weights includes: sequentially performing depthwise convolution and pointwise convolution on the gradient features, and then... Activation yields gating weights with values ​​ranging from 0 to 1; the boundary enhancement includes: weighted fusion of the gating enhancement features and the original interactive enhancement features using a learnable fusion coefficient α, wherein the fusion coefficient α is... Function constraints are set to 0 to 1.

[0026] The multi-scale decoding outputs a main saliency map and at least one auxiliary scale output, and the main saliency map is upsampled or reconstructed to a resolution consistent with the full-focus image.

[0027] The present invention also provides an interactive focusing stack saliency detection system, comprising:

[0028] The acquisition module is used to acquire the full-focus image and focus stack data of the target scene;

[0029] The sorting module is used to sort the focal plane sequence of the focus stack data;

[0030] The feature extraction module is used to extract the multi-scale image features of the fully focused image and the multi-focal plane features of the focus stack data, respectively.

[0031] An aggregation module is used to aggregate the multi-focal plane features to obtain a focused representation aligned with the multi-scale image features;

[0032] A fusion module is used to fuse the multi-scale image features and the focus representation at multiple scales to obtain fused features;

[0033] The prompt encoding and interaction module is used to encode user interaction prompt information into prompt features, and to make the prompt features interact with the fused features based on a dynamic attention mechanism to obtain interaction enhancement features;

[0034] A boundary enhancement module is used to enhance the boundaries of the interactive enhancement features based on gradient features and gating weights to obtain boundary enhanced features;

[0035] A semantic decoupling module is used to generate foreground region representation, background region representation, and uncertain region representation based on the boundary enhancement features.

[0036] The decoding module is used to perform multi-scale decoding based on the boundary enhancement features, the foreground region representation, the background region representation, and the uncertain region representation, and output a saliency map.

[0037] The aggregation module is configured to perform average aggregation or attention-weighted aggregation.

[0038] The fusion module includes a channel attention unit, which is used to perform channel weighting on the fused features.

[0039] The prompt encoding and interaction module supports at least one of point prompts, box prompts, and stroke prompts, and maps them to prompt feature maps.

[0040] The boundary enhancement module includes a gradient extraction unit and a gating generation unit. The gating generation unit outputs gating weights with values ​​ranging from 0 to 1 to modulate the interactive enhancement features.

[0041] The decoding module outputs a main saliency map and at least one auxiliary scale output.

[0042] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the above-described interactive focusing stack saliency detection method.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the above-described interactive focusing stack saliency detection method is implemented.

[0044] Compared with the prior art, the present invention has at least the following beneficial effects:

[0045] 1. This invention introduces focus stack data and aggregates and fuses multi-focal plane features at multiple scales, enabling focal length and depth-of-field variation information to participate in the distinction between target and background regions, thereby improving the robustness and accuracy of saliency detection in complex industrial scenarios.

[0046] 2. By encoding user interaction prompts into prompt features and using a dynamic attention mechanism to enable the prompt features to interact with the fused features, the model's ability to respond to a small number of interaction prompts is improved, thereby reducing the number of manual interactions required to obtain accurate detection results;

[0047] 3. By generating gating weights based on gradient features and performing boundary enhancement on interactive enhancement features, the representation of target edges and detailed structures is strengthened, thereby improving the problems of boundary breakage, blurry contours and excessive smoothing;

[0048] 4. By semantically decoupling the boundary enhancement features, foreground region representation, background region representation, and uncertain region representation are generated, which enhances the separability between the target region and the background region, thereby improving the regional consistency and boundary quality of the detection results in complex scenes;

[0049] 5. By decoding the main saliency map and outputting auxiliary scales at multiple scales, the stability of the model training process is improved, and the adaptability to target regions at different scales is enhanced, thereby improving the stability and generalization ability of the saliency detection results. Attached Figure Description

[0050] Figure 1 : Schematic diagram of the overall process of the method of the present invention. Detailed Implementation

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0053] like Figure 1As shown, this embodiment provides an interactive focus stack saliency detection method, which mainly includes the following steps: S1 Acquire full-focus image and focus stack data, extract multi-scale features and construct fused visual features;

[0054] S2 performs cue encoding and dynamic attention interaction on the fused visual features to obtain interaction enhancement features; S3 performs gradient gating boundary enhancement and semantic decoupling on the interaction enhancement features; S4 performs multi-scale decoding based on the boundary enhancement features and semantic decoupling results to output a saliency map.

[0055] In this application, the two information sources participating in dynamic attention interaction are fused visual features obtained by fusing full-focus image features and focus stack features, and prompt features obtained by encoding user interaction prompt information. Specifically, the fused visual features are mapped to query vectors, and the prompt features are mapped to key vectors and value vectors. Attention weights are calculated based on the correlation between the query vector and the key vector, and the value vectors are weighted and aggregated based on the attention weights, thereby enabling user interaction prompt information to participate in the selection and updating of the fused visual features.

[0056] In the interactive focusing stack saliency detection method of this application, step S1 mainly corresponds to Figure 1 The left-hand side features the extraction of dual branches, focused representation aggregation, and multi-scale fusion; step S2 mainly corresponds to... Figure 1 The interactive prompt module section; step S3 mainly corresponds to Figure 1 The gradient enhancement module and semantic decoupling module are part of the middle section; step S4 mainly corresponds to Figure 1 The saliency plot output section.

[0057] S1 acquires full-focus images and focus stack data, extracts multi-scale features, and constructs fused visual features.

[0058] This step is used to acquire full-focus images and focus stack data from the target scene, and extract multi-scale visual features respectively; further, the focus stack features are aggregated on the focal plane and fused with the full-focus image features at multiple scales to construct fused visual features for subsequent interactive guidance.

[0059] S11. Obtain the full-focus image and focus stack data:

[0060] Acquire a fully focused image of the target scene and focus stack data .

[0061] The focus stack data includes at least two images acquired at different focal planes. The fully focused image is used to provide complete scene texture information and overall semantic information, while the focus stack data is used to provide focal length variation information and depth of field variation information at different focal planes.

[0062] In one embodiment, let the batch size be B, the number of focal planes be N, the number of input image channels be C, and the input image spatial size be [missing information]. Then the focused stack data can be represented as: In the formula, B represents the sample batch size, N represents the number of focal planes in the focus stack, C represents the number of image channels, and H and W represent the height and width of the input image, respectively.

[0063] S12. Organize the focal plane sequence of the focus stack data:

[0064] The focal plane sequence of the focus stack data is sorted to obtain a focus stack input arranged in focal plane order.

[0065] In one embodiment, the focus stack data can be organized into a focal plane sequence form. In another embodiment, the focus stack data can also be expanded as follows: Alternatively, map the images of each focal plane as a channel stack: The above-mentioned sorting method can be adapted to the input organization requirements of different feature extraction networks.

[0066] S13. Extract multi-scale features from full-focus images and focus stack data:

[0067] Based on the full-focus image and the focus stack data, multi-scale image features of the full-focus image and multi-focal plane features of the focus stack data are extracted respectively.

[0068] Let the first backbone network used for feature extraction of fully focused images be... The second backbone network used for focused stack data feature extraction ,but: , In the formula, K represents the total number of feature scales output by the first backbone network and the second backbone network, K represents the scale index, and K = 1, 2, ..., K; This represents the full-focus image features at the k-th scale. This represents the focused stack feature at the k-th scale.

[0069] S14. Aggregate the features of the multi-focal plane to obtain the focused representation.

[0070] Based on the multi-focal-plane features, features corresponding to different focal planes at the same scale are aggregated along the focal-plane dimension, and spatial size and channel dimension mapping is performed as needed to obtain a focus representation that matches the features of a fully focused image at the same scale. The focus representation refers to the feature representation formed by integrating features corresponding to multiple focal planes at the same scale using an average aggregation method or an attention-weighted aggregation method. It is used to centrally represent the focal length and depth-of-field changes of the target scene at different focal planes.

[0071] For the focused stack feature at the k-th scale, it can be reconstructed as: ; Focused representation at the k-th scale under the average aggregation method for:

[0072]

[0073] In the attention-weighted aggregation approach, focal plane weights are first generated based on the features of each focal plane:

[0074]

[0075]

[0076] The focusing representation at the k-th scale can then be expressed as:

[0077]

[0078] In the formula, This represents the feature of the nth focal plane at the kth scale. This represents the focal plane weight generation function. This represents the contribution weight of the nth focal plane to the focus representation.

[0079] This sub-step is a crucial technical foundation of the present invention. By aggregating multi-focal-plane features along the focal plane dimension, focal change information scattered across different focal planes can be integrated into a unified focus representation. This allows focus stack data to participate in the saliency detection process in a structured manner, rather than merely serving as additional image sequence input.

[0080] S15. Fusion of multi-scale image features and focus representations

[0081] Based on the multi-scale image features and the focus representation, a fusion feature is obtained. The fusion feature refers to the joint feature obtained by combining the multi-scale image features of the full-focus image with the corresponding focus representation through concatenation or addition at the same feature scale, followed by convolutional mapping and / or channel attention weighting. The fusion feature simultaneously includes texture and semantic information from the full-focus image, as well as focal length and depth-of-field variation information from the focus stack data.

[0082] In one embodiment, a splicing and fusion method can be used:

[0083]

[0084] In another embodiment, an additive fusion method can also be used:

[0085]

[0086] Furthermore, a channel attention mechanism can be used to adaptively weight the fused features:

[0087]

[0088]

[0089] In the formula, This represents the initial fused features at the k-th scale. This indicates a global average pooling operation. This represents a multilayer perceptron. express Activation function Indicates channel attention weights. This represents the fused visual feature at the k-th scale.

[0090] This sub-step is another important foundation of the present invention. By aligning and fusing the focus representation with the multi-scale image features of the full-focus image, it is possible to introduce focal length variation information to assist in distinguishing the foreground and background regions while maintaining global semantic information, thereby improving the target representation capability in complex backgrounds.

[0091] Step S1 can unify the texture and semantic information in the full-focus image and the focal length and depth cues in the focus stack data into the same feature space, thereby providing a reliable fusion visual foundation for subsequent interactive prompts and boundary enhancement.

[0092] S2 performs cue encoding and dynamic attention interaction on the fused visual features to obtain interactive enhancement features.

[0093] This step encodes user interaction prompts into prompt features and enables dynamic attention interaction between these prompt features and fused visual features. This allows for explicit guidance of the user's prior knowledge of the salient target region detection process. Compared to directly stitching prompts to the input image or simply overlaying them onto the feature map, this step, through unified prompt encoding and a dynamic attention update mechanism, allows different types of interactive prompts to function within a unified feature space. This improves the utilization efficiency of a small amount of prompt information and reduces repetitive interaction operations.

[0094] S21. Obtain user interaction prompts.

[0095] Obtain user interaction prompts G.

[0096] The user interaction prompts include at least one of dot prompts, box prompts, and stroke prompts. Dot prompts indicate key locations within the target or background area, box prompts define the range of the candidate target area, and stroke prompts roughly outline the target or background area.

[0097] In one embodiment, dot hints may include foreground dot hints and background dot hints; box hints may represent the area roughly enclosed by the target; and stroke hints may represent the user's rough drawing trajectory of the foreground or background area. Since different types of hints differ in their spatial distribution and semantic function, they require unified encoding.

[0098] S22. Standardize the encoding of user interaction prompts.

[0099] The user interaction prompt information is mapped to a prompt feature map with the same size as the fused visual feature space.

[0100] Let the prompt encoding function be Then the cue feature at the k-th scale can be represented as:

[0101] In the formula, , representing the cue feature map at the k-th scale.

[0102] In one embodiment, dot hints can be encoded as location response maps, box hints can be encoded as region mask maps, and stroke hints can be encoded as path response maps. Furthermore, different types of hints can be uniformly mapped to a spatial resolution and channel dimension consistent with the fused visual features through convolutional mapping, embedding mapping, or location encoding.

[0103] By using the unified encoding method described above, the originally heterogeneous point, box, and stroke prompts can be transformed into prompt features with a unified representation, thereby providing a consistent input basis for subsequent dynamic attention interactions. This sub-step is one of the key technical points of this invention, and its function is to solve the problem that different interactive prompt forms are difficult to participate in feature updates in a unified manner.

[0104] S23. Feature Interaction Based on Dynamic Attention Mechanism

[0105] Dynamic attention interaction is performed based on the cue features and the fused visual features.

[0106] Let the fused visual features at the k-th scale be... Then it is mapped to a query vector:

[0107]

[0108] Map the cue features to key vectors and value vectors:

[0109]

[0110] The attention interaction result at the k-th scale is:

[0111]

[0112] In the formula, , and These represent the query, key, and value mapping parameters, respectively. This represents the feature dimension at the k-th scale. This represents the result of dynamic attention interaction.

[0113] Compared to existing technologies that directly concatenate prompt information to images or simply add it to feature maps, this embodiment uses fused visual features as the query vector and prompt features as the key and value vectors. This allows the model to proactively select feature response regions relevant to the user prompts and suppress background responses unrelated to the prompts, based on the current visual semantic understanding. Thus, the prompt information is no longer merely an additional input but directly participates in the feature selection and update process.

[0114] This sub-step is one of the core innovations of this invention. Through the dynamic attention mechanism, a small number of interactive prompts can have a stronger guiding effect on the response of the target area, thereby achieving effective focusing on the target area even in complex backgrounds, when the target and background are similar, or when the target boundary is blurred.

[0115] S24. Generate interactive enhancement features

[0116] The attention interaction results are combined with the original fused visual features to obtain the interaction enhancement features.

[0117] In one embodiment, a residual connection method can be used: In the formula, This represents the interactive enhancement feature at the k-th scale. Employing residual connections preserves stable semantic information in the original fused visual features while introducing cue guidance results, avoiding excessive shift in the original visual representation due to overly strong cue signals, thereby improving the stability and robustness of feature updates.

[0118] Step S2 allows a small number of user prompts to be directly injected into the fused visual features, enabling the model to obtain target region responses that better match the user's intent with fewer interactions, thereby improving the efficiency and accuracy of interactive saliency detection.

[0119] S3 performs gradient-gated boundary enhancement and semantic decoupling on interactive enhancement features.

[0120] This step is used to enhance the boundaries of interactive enhancement features and further improve the separability between the target region, background region and uncertain region through semantic decoupling.

[0121] S31. Extract gradient features: Extract gradient features based on the interaction enhancement features.

[0122] Let the gradient extraction function be... Then the gradient feature at the k-th scale can be expressed as:

[0123] In one embodiment, the gradient extraction function may be adopted. Gradient extraction operators, or other gradient extraction operators that can characterize edge and local structural changes.

[0124] S32. Generate gating weights

[0125] Gating weights are obtained based on the gradient features.

[0126] In one embodiment, gated weights can be generated using depthwise convolution and pointwise convolution:

[0127]

[0128] In the formula, Represents depthwise convolution. Represents point convolution. Let represent the gating weight at the k-th scale, and .

[0129] The gating weights are used to characterize the degree of response of different spatial locations and / or different feature channels in the interactive enhancement features to the target boundary. The gradient features reflect the degree of change of the interactive enhancement features in local regions, where locations with strong gradient responses usually correspond to target contours, detailed structures, or abrupt changes in the region. The local gradient structure in each channel is extracted by depthwise convolution, and the gradient responses of different channels are fused by point convolution. The fused result is then mapped to a value range of 0 to 1 by the Sigmoid function, thereby converting the gradient features into gating weights that can modulate the interactive enhancement features.

[0130] Positions with larger gating weights indicate a higher correlation between that position and the target boundary or local structural changes, retaining a higher proportion of feature responses during gating modulation. Conversely, positions with smaller gating weights indicate a lower correlation with the boundary, resulting in relatively suppressed feature responses. Therefore, the gating weights establish a mapping relationship between gradient features and boundary enhancement features, enabling the target boundary region to obtain more prominent feature representation compared to non-boundary regions.

[0131] S33. Perform boundary reinforcement

[0132] The interaction enhancement features are enhanced by applying the gating weights to obtain the boundary enhancement features.

[0133] First, the interaction enhancement features are gated and modulated using the gate weights:

[0134]

[0135] Furthermore, learnable fusion coefficients can be used. Adaptive fusion of gated enhancement features and original interaction enhancement features:

[0136]

[0137] In the formula, Indicates gating enhancement features, Indicates boundary enhancement features, For learnable parameters, This represents the Sigmoid function. For the The fusion coefficient is constrained to 0 to 1.

[0138] The fusion coefficient This is used to adjust the contribution ratio of gated augmentation features and original interactive augmentation features to boundary augmentation features. When As the size increases, boundary enhancement features focus more on the boundary and detail information after gradient gating; when When reduced, the boundary enhancement features place greater emphasis on the overall semantic information in the original interaction enhancement features. The learnable parameters... As model parameters, they are updated together with other network parameters through backpropagation based on the target loss function during model training, enabling the model to adaptively determine the fusion ratio between boundary detail information and overall semantic information based on the training data.

[0139] S34. Perform semantic decoupling

[0140] Semantic decoupling is performed based on the boundary enhancement features. The target foreground semantics, background semantics, and uncertain region semantics that coexist in the boundary enhancement features are mapped to three independent feature branches, respectively, to obtain foreground region representation, background region representation, and uncertain region representation. Specifically:

[0141]

[0142]

[0143]

[0144] In the formula, These represent the foreground semantic mapping branch, the background semantic mapping branch, and the uncertain region semantic mapping branch with mutually independent parameters at the Kth scale, respectively. , , These represent the foreground region, background region, and uncertain region, respectively.

[0145] In one embodiment, each semantic mapping branch includes at least one layer of convolution, normalization, and nonlinear activation operations to extract feature representations with different semantic tendencies from the same boundary enhancement features. The parameters of each semantic mapping branch are independent of each other and are updated separately during model training, thereby achieving decoupling of the hybrid semantic features from the foreground region, background region, and uncertain region.

[0146] In one embodiment, independent auxiliary output branches can be set based on the foreground region representation, background region representation, and uncertain region representation respectively to generate foreground region auxiliary output, background region auxiliary output, and uncertain region auxiliary output:

[0147]

[0148]

[0149]

[0150] In the formula, and These represent the foreground auxiliary output head, the background auxiliary output head, and the uncertain region auxiliary output head, respectively. Represents the Sigmoid function; , and These represent the foreground region response map, background region response map, and uncertain region response map, respectively, with values ​​ranging from 0 to 1.

[0151] In one embodiment, each auxiliary output head may include convolution mapping operations and upsampling operations to map the corresponding semantic region representation into a single-channel response map that matches the spatial size of the input image. By setting foreground auxiliary output heads, background auxiliary output heads, and uncertain region auxiliary output heads with independent parameters respectively, the three types of semantic representations are made into explicit outputs, thereby constraining the semantic decoupling process and improving the separability between the foreground region, background region, and uncertain region.

[0152] This sub-step constitutes the third important technical point of the present invention. By first performing gradient-gated boundary enhancement and then semantically decoupling the enhanced features, the continuity of the target boundary and the stability of region partitioning can be improved simultaneously, thereby mitigating the problems of boundary breakage, region spillover, and easy confusion in complex backgrounds.

[0153] Step S3 enhances the model's ability to perceive target boundaries and detailed structures, and explicitly distinguishes between foreground, background, and uncertain regions.

[0154] S4 performs multi-scale decoding based on boundary enhancement features and semantic decoupling results, and outputs a saliency map.

[0155] This step is used to perform multi-scale decoding based on boundary enhancement features and semantic decoupling results to obtain a saliency map with a spatial size consistent with the input fully focused image.

[0156] S41. Constructing the decoding input

[0157] Based on the boundary enhancement features and the foreground region representation, background region representation, and uncertain region representation, the decoding input features are constructed.

[0158] In one embodiment, the decoding input can be formed in the following manner:

[0159]

[0160] In the formula, This represents the decoding input feature at the k-th scale.

[0161] S42. Perform multi-scale decoding

[0162] Multi-scale decoding is performed based on the aforementioned decoding input features.

[0163] Let the decoding function be The step-by-step decoding process can then be represented as:

[0164]

[0165] In the formula, This represents the decoding feature at the k-th scale.

[0166] In one embodiment, the decoding function It can include upsampling and convolutional reconstruction operations to achieve a stepwise recovery of high-level semantic information and low-level detailed information.

[0167] S43. Output the significance plot

[0168] Generate a saliency plot using the main output header: ,in, In the formula, S represents the main output header, and S represents the main saliency graph.

[0169] In one embodiment, at least one auxiliary scale output may also be output: In the formula, This represents the prediction result at the k-th auxiliary scale.

[0170] S44. Applied to industrial defect area detection tasks

[0171] The saliency map can be used for tasks such as locating industrial defect areas, extracting abnormal areas, intelligent quality inspection, or auxiliary annotation.

[0172] Step S4 yields saliency detection results that are continuous in boundary, complete in region, and robust to complex backgrounds, thus meeting the application requirements for rapid localization and high-quality separation of target areas in industrial scenarios.

[0173] Further explanation

[0174] The above formulas are used to illustrate the feature flow relationships and main calculation relationships in each step of the present invention, and do not constitute a limitation on the specific network parameter form, convolution kernel size, number of layers, number of channels, or specific function form. Without departing from the concept of the present invention, those skilled in the art can make equivalent substitutions for the described feature extraction method, aggregation method, cue encoding method, dynamic attention interaction method, gradient gating method, and multi-scale decoding method.

[0175] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An interactive focusing stack saliency detection method, characterized in that, Includes the following steps: Acquire light field images; Based on the light field image, obtain the full-focus image of the target scene and focus stack data; Based on the full-focus image and the focus stack data, obtain the multi-scale image features of the full-focus image and the multi-focal plane features of the focus stack; Based on the multi-focal plane features, a focus representation aligned with the multi-scale image features is obtained; Based on the multi-scale image features and the focus representation, fusion features are obtained; Based on user interaction prompts, the fused features are prompted with encoding and dynamic attention interaction to obtain interaction enhancement features; Gradient features are extracted based on the interaction enhancement features to obtain gating weights; Based on the gating weights, the interaction enhancement features are subjected to boundary enhancement to obtain boundary enhancement features; Based on the boundary enhancement features, semantic decoupling is performed to generate foreground region representation, background region representation, and uncertain region representation; Multi-scale decoding is performed based on the boundary enhancement features, foreground region representation, background region representation, and uncertain region representation to output a saliency map with the same spatial size as the fully focused image.

2. The interactive focusing stack saliency detection method as described in claim 1, characterized in that, The organization of the focus stack data includes: The focus stack data is grouped according to the focal plane order and reconstructed into a focal plane sequence tensor, so that each focal plane image is used as a sequence element input to the feature extraction network, or each focal plane image is mapped as a channel stack input to the feature extraction network according to a predetermined rule.

3. The interactive focusing stack saliency detection method as described in claim 1, characterized in that, Feature extraction for full-focus images and focus stacks employs a first backbone network and a second backbone network, respectively, with the first backbone network and the second backbone network outputting at least two levels of multi-scale features.

4. The interactive focusing stack saliency detection method as described in claim 1, characterized in that, The step of obtaining a focus representation aligned with the multi-scale image features based on the multi-focal plane features includes: aggregating the multi-focal plane features using average aggregation or attention-weighted aggregation; wherein, the attention-weighted aggregation includes: generating corresponding focal plane weights based on each focal plane feature, and performing weighted summation on the multi-focal plane features based on the focal plane weights.

5. The interactive focusing stack saliency detection method as described in claim 1, characterized in that, The step of obtaining fused features based on the multi-scale image features and the focus representation includes: splicing and fusing the full-focus image features and the focus representation at the same scale, or adding and fusing them together, and adaptively weighting the fused features through a channel attention mechanism.

6. The interactive focusing stack saliency detection method as described in claim 1, characterized in that, The user interaction prompts include at least one of dot prompts, box prompts, and stroke prompts; Encoding the user interaction prompt information includes: mapping the user interaction prompt information into a prompt feature map with the same size as the fused feature space; The dynamic attention interaction adopts a multi-head attention mechanism, and uses the fused features as the query vector and the prompt features as the key vector and value vector to achieve feature updates guided by prompts.

7. The interactive focusing stack saliency detection method as described in claim 1, characterized in that, The step of extracting gradient features based on the interaction enhancement features and obtaining gating weights based on the gradient features includes: calculating gradient features on the interaction enhancement features, generating gating weights based on the gradient features, and using the gating weights to weight the interaction enhancement features to enhance the target boundary; the generation of the gating weights includes: sequentially performing depthwise convolution and pointwise convolution on the gradient features, and then... Activation yields a gating weight with a value between 0 and 1; The boundary enhancement employs a learnable fusion coefficient α to weight and fuse the gated enhancement features and the original interactive enhancement features, and the fusion coefficient α is... Constrained to 0 to 1; the multi-scale decoding simultaneously outputs a main saliency map and at least one auxiliary scale output, and upsamples or reconstructs the main output to a resolution consistent with the full-focus image.

8. An interactive focusing stack saliency detection system, characterized in that, include: The acquisition module is used to acquire the full-focus image and focus stack data of the target scene; The sorting module is used to sort the focal plane sequence of the focus stack data; The feature extraction module is used to extract the multi-scale image features of the fully focused image and the multi-focal plane features of the focus stack data, respectively; the aggregation module is used to aggregate the multi-focal plane features to obtain a focus representation aligned with the multi-scale image features. The fusion module is used to fuse the multi-scale image features and focus representations at multiple scales to obtain fused features; The prompt encoding and interaction module is used to encode user interaction prompt information into prompt features, and to make the prompt features interact with the fused features based on a dynamic attention mechanism to obtain interaction enhancement features; A boundary enhancement module is used to enhance the boundaries of the interactive enhancement features based on gradient features and gating weights to obtain boundary enhanced features; A semantic decoupling module is used to generate foreground region representation, background region representation, and uncertain region representation based on the boundary enhancement features. The decoding module is used to perform multi-scale decoding based on the boundary enhancement features, the foreground region representation, the background region representation, and the uncertain region representation, and output a saliency map.

9. The interactive focusing stack saliency detection system as described in claim 8, characterized in that, The aggregation module is configured to perform average aggregation or attention-weighted aggregation; the fusion module includes a channel attention unit for channel-weighting the fused features; the prompt encoding and interaction module supports at least one of point prompts, box prompts, or stroke prompts and maps them to a prompt feature map.

10. The interactive focusing stack saliency detection system as described in claim 8, characterized in that, The boundary enhancement module includes a gradient extraction unit and a gating generation unit. The gating generation unit outputs gating weights of 0 to 1 to modulate the interactive enhancement features. The decoding module outputs a main saliency map and at least one auxiliary scale output.

Images