Image feature enhancement method and system based on learnable unary function gating

Abstract

This invention relates to the field of image feature enhancement technology, providing an image feature enhancement method and system based on learnable unary function gating. The method includes: dividing the input image into image patches and mapping them to visual tokens; in a multi-head attention layer, calculating a visual attention aggregation score based on an attention weight matrix to quantify the degree of abnormal attention received by the image patch; inputting the normalized visual token, aggregation score, and two-dimensional position code into a gating module composed of a learnable unary function to generate a gating matrix; after each attention head completes SDPA output and before multi-head stitching, performing element-wise gating modulation using the gating matrix, and stitching and projecting to obtain the enhanced image features. This invention, through the synergy of aggregation score and learnable unary function, suppresses abnormal attention propagation from background noise, enhances the expression of key features of small targets, and improves the feature discriminativeness and task adaptability of the visual Transformer.

Description

Technical Field

[0001] This invention relates to the field of image feature enhancement technology, and in particular to an image feature enhancement method and system based on learnable univariate function gating. Background Technology

[0002] The Vision Transformer (ViT) transforms image data into a sequence for processing by dividing the input image into fixed-size patches and embedding each patch into a high-dimensional visual token using linear or convolutional mapping. Let's assume the input image, after patching and embedding, yields a sequence of visual tokens: Where B is the batch size, N is the number of patch tokens, and D is the hidden dimension.

[0003] The core of the visual Transformer lies in the Multi-Head Self-Attention (MHSA) mechanism. For the h-th attention head, the calculation process is as follows: ; ; ; ; ; in, These are the query matrix, key matrix, and value matrix, respectively. Let be the attention weight matrix, and d be the dimension of a single attention head. This is the output of the Scaled Dot-Product Attention (SDPA) for this attention head. The outputs of multiple attention heads are concatenated and projected to form the output of the current attention layer.

[0004] The MHSA mechanism captures long-range dependencies and global contextual information in images by calculating fully connected attention relationships between patch tokens. However, this global attention calculation method has inherent structural defects in visual scenes: after an image is segmented into a large number of patches, background areas, repetitive textures, reflective areas, occluded areas, noisy areas, or non-target salient areas may receive abnormally high attention weights in the attention calculation, causing the visual information of these areas to be redundantly propagated to the global features, thereby weakening the feature representation ability of target areas, small defect areas, small lesion areas, or remote sensing small target areas.

[0005] To improve the information selection capability of attention mechanisms, traditional approaches have proposed various gating attention schemes. In linear gating attention, the gating matrix is ​​typically generated after the SDPA output and before the output projection, and is modulated element-wise with the attention output: ; ; ; in, It is the Sigmoid activation function. It is a learnable linear projection matrix. This represents element-wise multiplication. This type of scheme filters the information after attention aggregation through input-related gating mechanisms, which can, to some extent, suppress ineffective attention output and improve training stability.

[0006] Furthermore, Kolmogorov-Arnold Networks (KAN) proposes replacing scalar weights or simple linear mappings in traditional neural networks with learnable unary functions, such as spline functions, piecewise linear functions, or other trainable unary functions, to improve nonlinear expressiveness and interpretability. Traditional approaches have used KAN to replace some linear transformations in the Multilayer Perceptron (MLP) layers or attention mechanisms in Transformers, but a systematic solution specifically designed for visual scenarios and addressing the modulation requirements after SDPA output has not yet been developed.

[0007] While the aforementioned techniques have alleviated the information selection problem of attention mechanisms to some extent, the following deep-seated technical shortcomings still exist in visual Transformers for image input: First, the nonlinear expressive power of gating generation methods is limited. In traditional linear gating schemes, gating logits are generated by linear projections such as XWg, which are essentially weighted sums of the input features across different dimensions. This linear mapping struggles to characterize the nonlinear suppression or enhancement relationships of image patch features across different numerical ranges. For example, linear gating may produce similar gating outputs for high activation values ​​in the background region and high activation values ​​in the target region, failing to provide a fine-grained distinction based on differences in visual semantics.

[0008] Second, there is a lack of explicit perception of the degree of abnormal concentration of visual attention. Traditional gating schemes typically rely solely on the hidden states of the current layer for input, failing to fully utilize the rich structural information contained in the attention weight matrix Ah. The attention weight matrix Ah directly reflects the attention relationships between various patch tokens: if a background patch receives high-weight attention from a large number of other patches, or if a local region abnormally dominates global features, this attention concentration anomaly should be explicitly perceived and used to guide gating decisions. However, traditional schemes do not establish a direct mapping path from the attention weight matrix to gating generation, resulting in the gating module lacking the ability to perceive anomalies in attention propagation.

[0009] Third, there is a lack of effective utilization of two-dimensional visual location information. Image patches have a clear spatial arrangement structure in the original image, and patches at different spatial locations have different visual prior meanings (e.g., the central region of the image is more likely to contain the target subject, while the edge region is more likely to belong to the background). Traditional gating schemes do not use the two-dimensional spatial location information of the patches as gating input, resulting in the gating decision not being able to utilize the spatial structure prior of the image, making it difficult to target and modulate the attention output of specific spatial regions.

[0010] Fourth, the independent modulation capability of multi-head attention heads is insufficient. In visual tasks, different attention heads typically learn different visual patterns, such as edge detection, texture analysis, local object focusing, and long-range region relationship modeling. If gating is applied uniformly after concatenating multiple attention heads, the functional differences between them will be smoothed out, resulting in the outputs of edge-sensitive heads and region relationship heads being modulated by the same strategy, thus undermining the original design intent of the multi-head mechanism. Traditional solutions do not fully consider the functional differentiation characteristics of attention heads in visual Transformers and lack the ability to independently and adaptively modulate each attention head.

[0011] Fifth, the interpretability and deployability of gating mechanisms are insufficient. In linear gating, the projection matrix Wg maps input features to gating values ​​in a black-box manner, making it difficult to explain why certain visual features, spatial locations, or attention patterns trigger suppression or enhancement. Furthermore, linear gating lacks structural sparsity, making it difficult to achieve lightweight deployment by pruning low-contribution parameters after training. While KAN-like networks improve interpretability, traditional solutions have not organically integrated them with visual attention modulation scenarios, failing to leverage their potential advantages in gating generation. Summary of the Invention

[0012] The purpose of this invention is to solve at least one technical problem in the background art and to provide an image feature enhancement method and system based on learnable univariate function gating.

[0013] To achieve the above objectives, this invention provides an image feature enhancement method based on learnable univariate function gating, comprising: The input image is divided into multiple image blocks, and each image block is mapped to a visual token to obtain a visual token sequence. The two-dimensional positional encoding of the image blocks is embedded in the visual token sequence. The visual token sequence is normalized to obtain normalized visual tokens; For each attention head in the multi-head attention layer, the query matrix, key matrix, and value matrix corresponding to the attention head are calculated based on the normalized visual token, and the attention weight matrix and SDPA output of the attention head are obtained by scaling dot product attention calculation. The visual attention clustering score is calculated based on the attention weight matrix. The visual attention clustering score is used to quantify the degree of clustering of the corresponding image patch with attention from other image patches, so as to identify whether the image patch belongs to a target region that is abnormally focused, a background region that generates redundant propagation, or a local region that over-dominates global features. The normalized visual token, the visual attention aggregation score, and the two-dimensional position code are input into the gating module. The gating module is composed of learnable unary functions. Each channel of each attention head corresponds to a set of learnable unary functions. Each set of learnable unary functions contains the same number of learnable unary functions as the gating input components. Each learnable unary function is composed of at least one function edge, which is used to generate the gating matrix. After each attention head completes its SDPA output and before multiple attention heads are concatenated, the SDPA output of the attention head is modulated element-wise using the gating matrix to obtain the gated attention head output. The outputs of multiple gated attention heads are stitched together and then projected onto the output to obtain enhanced image features.

[0014] According to one aspect of the present invention, calculating a visual attention concentration score based on the attention weight matrix includes one or a combination of the following methods: For the h-th attention head, the average intensity of attention given to the j-th image patch by other image patches is calculated as the visual attention aggregation score of that image patch; A unified clustering score is obtained by averaging, taking the maximum value, or weighted summing the visual attention clustering scores of multiple attention heads. The visual attention concentration score is constructed by combining image region priors, saliency estimation, or background estimation.

[0015] According to one aspect of the invention, the learnable univariate function includes a spline function, a piecewise linear function, or a polynomial function; For the h-th attention head and the c-th channel, the gated logits are obtained by summing the components of the gated input after transforming each learnable unary function corresponding to that channel, and then obtaining the gated value through Sigmoid mapping, thus forming the gated matrix.

[0016] According to one aspect of the present invention, element-wise gated modulation of the SDPA output of the attention head is performed using the gate matrix, comprising: The gating matrix is ​​multiplied element-wise with the SDPA output of the corresponding attention head. When the gating value approaches 1, the corresponding attention output is retained. When the gating value approaches 0, the corresponding attention output is suppressed. When the gating value is between 0 and 1, the corresponding attention output is partially retained.

[0017] According to one aspect of the invention, during the training phase, a sparse constraint regularization term is applied to the gating matrix or the learnable univariate function, the sparse constraint regularization term including at least one of the following: gating value average activation regularization term, gating entropy regularization term, function edge contribution regularization term, or channel activation sparse regularization term.

[0018] According to one aspect of the present invention, after training is completed, the contribution of each learnable univariate function edge in the gating module to the gating output is calculated based on the activation statistics of the gating matrix, and the function edges that meet the preset pruning conditions are pruned. The preset pruning conditions are determined based on at least one of the following indicators: the long-term activation mean of the function edge is lower than a first threshold, the activation variance of the function edge is lower than a second threshold, or the contribution of the function edge output to the gated logits is lower than a third threshold.

[0019] To achieve the above objectives, the present invention also provides an image feature enhancement system based on learnable univariate function gating, comprising: The image block embedding module divides the input image into multiple image blocks, maps each image block to a visual token to obtain a visual token sequence, and embeds the two-dimensional positional encoding of the image blocks into the visual token sequence; The normalization module normalizes the visual token sequence to obtain a normalized visual token. The multi-head attention calculation module calculates the query matrix, key matrix, and value matrix corresponding to each attention head in the multi-head attention layer based on the normalized visual token, and obtains the attention weight matrix and SDPA output of the attention head through scaled dot product attention calculation. The aggregation score calculation module calculates a visual attention aggregation score based on the attention weight matrix. The visual attention aggregation score is used to quantify the degree of aggregation of the corresponding image patch by other image patches, so as to identify whether the image patch belongs to a target region that is abnormally focused, a background region that generates redundant propagation, or a local region that over-dominates global features. The gating module receives the normalized visual token, the visual attention aggregation score, and the two-dimensional position code. The gating module is composed of learnable unary functions. Each channel of each attention head corresponds to a set of learnable unary functions. Each set of learnable unary functions contains the same number of learnable unary functions as the gating input components. Each learnable unary function is composed of at least one function edge, which is used to generate a gating matrix. The gated modulation module uses the gate matrix to perform element-wise gated modulation on the SDPA output of each attention head after it completes its SDPA output and before multiple attention heads are spliced ​​together, so as to obtain the gated attention head output. The output module stitches together the outputs of multiple gated attention heads and then projects the outputs to obtain enhanced image features.

[0020] To achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the image feature enhancement method based on learnable univariate function gating as described above.

[0021] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image feature enhancement method based on learnable univariate function gating as described above.

[0022] According to the present invention, visual attention aggregation scoring, learnable univariate function gating, and head-to-head modulation after SDPA output are organically integrated to achieve refined adaptive control of attention propagation and produce multiple technical effects: Visual attention aggregation scoring is calculated based on the attention weight matrix, explicitly quantifying the degree of abnormal attention to each image patch to identify background redundancy propagation or local over-dominance, thereby suppressing ineffective propagation of background and noise patches; a gating module composed of learnable univariate functions such as spline functions, piecewise linear functions, or polynomial functions is constructed, using the normalized visual token, aggregation score, and two-dimensional position code as input, breaking through the nonlinear expression bottleneck of linear gating, generating differentiated responses for different value ranges, and the response curve of the learnable univariate function can be intuitively plotted. This approach analyzes how specific visual features, spatial locations, or clustering scores influence gating decisions, significantly improving interpretability. Element-wise gating modulation is applied independently after each attention head completes its SDPA output and before multi-head stitching, preserving multi-head functional differentiation and enabling differentiated control over different visual modes such as edges, textures, shapes, and long-distance dependencies. This enhances the representation of small targets and defect regions, reducing the intrusion of key features in scenes such as industrial defects, medical lesions, and remote sensing small targets by large background areas. The gating value is restricted to the (0,1) interval by a Sigmoid function, suppressing abnormally large attention outputs and improving training and inference stability. After training, the contribution of function edges is evaluated based on the activation statistics of the gating matrix. Function edges with long-term low activation, low variance, or low contribution are pruned, achieving lightweight structure and low-overhead deployment. These core mechanisms work synergistically to significantly improve the feature discriminative power and task adaptability of the visual Transformer in complex visual scenes. Attached Figure Description

[0023] Figure 1 The flowchart illustrates an image feature enhancement method based on learnable univariate function gating according to one embodiment of the present invention. Detailed Implementation

[0024] The invention will now be discussed with reference to exemplary embodiments. It should be understood that the described embodiments are merely intended to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0025] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment".

[0026] Figure 1The flowchart schematically illustrates an image feature enhancement method based on learnable univariate function gating according to an embodiment of the present invention. Figure 1 As shown, in this embodiment, the image feature enhancement method based on learnable univariate function gating includes: The input image is divided into multiple image blocks, and each image block is mapped to a visual token to obtain a visual token sequence. The two-dimensional positional encoding of the image blocks is embedded in the visual token sequence. The visual token sequence is normalized to obtain normalized visual tokens; For each attention head in the multi-head attention layer, the query matrix, key matrix, and value matrix corresponding to the attention head are calculated based on the normalized visual token, and the attention weight matrix and SDPA output of the attention head are obtained by scaling dot product attention. The visual attention clustering score is calculated based on the attention weight matrix. The visual attention clustering score is used to quantify the degree of clustering of the corresponding image patch with attention from other image patches, so as to identify whether the image patch belongs to the target region that is abnormally focused, the background region that generates redundant propagation, or the local region that over-dominates the global features. The normalized visual token, visual attention aggregation score, and two-dimensional position code are input into the gating module. The gating module is composed of learnable unary functions. Each channel of each attention head corresponds to a set of learnable unary functions. Each set of learnable unary functions contains the same number of learnable unary functions as the gating input components. Each learnable unary function is composed of at least one function edge, which is used to generate the gating matrix. After each attention head completes its SDPA output and before multiple attention heads are concatenated, the SDPA output of the attention head is modulated element-wise using a gating matrix to obtain the gated attention head output. The outputs of multiple gated attention heads are stitched together and then projected onto the output to obtain enhanced image features.

[0027] In this embodiment, in the multi-head self-attention mechanism, the attention weight matrix It contains the complete attention relationships between image patches, but traditional methods usually only use it as an intermediate calculation result to generate SDPA output, without further mining its structural semantic information. This invention calculates the visual attention aggregation score based on the attention weight matrix, specifically by calculating the average intensity of attention of the j-th image patch by all other image patches. This method compresses the high-dimensional attention relation matrix into a scalar of clustering degree with explicit physical meaning. This score directly reflects the centrality of the corresponding image patch within the attention propagation network: when a background patch receives high weight attention from many other patches due to repetitive textures or reflective properties, its clustering score will significantly increase; when a local region abnormally dominates global features, the clustering scores of patches within that region will also exhibit an abnormal distribution. Thus, the visual attention clustering score establishes an explicit mapping channel from the attention weight matrix to gating decisions, enabling the gating module to adaptively modulate based on structural anomalies in attention propagation, rather than relying solely on the semantic representation of the hidden state for black-box filtering.

[0028] Traditional linear gating generates gated logits from linear projections of the form XWg. Its output exhibits a monotonically linear response to each dimension of the input features, failing to generate differentiated gating strategies for inputs with varying numerical ranges. This scheme's gating module consists of learnable unary functions. For the h-th attention head and the c-th channel, the gated logits are expressed as... ,in, For learnable univariate functions, Let m be the m-th component of the gating input. This structure maps each component of the gating input independently and then aggregates them. Each unary function can learn differentiated response curves for different value ranges of its corresponding input component. For example, for the visual attention concentration score component, the learnable unary function can learn an approximate identity mapping in the low concentration score range to preserve the target region features, a smooth transition in the medium concentration score range, and a strong suppression mapping in the high concentration score range to suppress background redundancy propagation. For the two-dimensional location coding component, the learnable unary function can learn differentiated gating biases for different spatial locations, such as suppressing attention output in image edge regions and enhancing feature representation in the central region. This component-wise and interval-wise nonlinear learning capability is impossible for linear projection, enabling the gating module to generate refined gating decisions based on the ternary combination of visual token features, attention concentration degree, and spatial location.

[0029] In this implementation, the learnable unary function is a nonlinear scoring function independently set for a single dimension of the feature vector within the gating module. For the h-th attention head, c-th channel, and m-th input feature component, the corresponding unary function is denoted as: This function accepts only one scalar input (i.e., the value of the m-th dimension of the gated feature vector). It outputs a scalar contribution value, which represents the degree of gating contribution of this dimension feature to the current attention head and channel.

[0030] In this embodiment, the gating module controls the input feature vector. Each dimension applies a corresponding unary function, and the dimensions are not cross-coupled.

[0031] same Parameters are shared across different batches of samples and different tokens, meaning that the m-th component of all images and all tokens is processed by the same set of unary functions.

[0032] Different unary functions can be set for different input components m, different attention heads h, or different channels c. Therefore, even if the same input feature has the same value, it can produce differentiated gating contributions for different attention heads and channels.

[0033] The core function of learnable univariate functions is to provide a dimension-wise nonlinear scoring mechanism for the calculation of gated logits, specifically including: Feature importance assessment: Determine how much gating contribution the current value of each dimension of the gating feature vector should make to the corresponding attention head and channel.

[0034] Gated logits synthesis: combining the outputs of M univariate functions with a learnable bias b h,c Add them together to get the gated logit of the current token on a specific attention head and channel: ; Refined feature control: The gate value G is obtained through subsequent Sigmoid transformation. h,c It is then multiplied element-wise with the attention output to achieve fine control over the degree of attention feature retention for each sample, each token, each attention head, and each channel—the larger the gate value, the more features are retained; the smaller the gate value, the stronger the suppression of the corresponding features.

[0035] In this embodiment, to avoid excessive suppression of attention output, a lower gating limit g can be further introduced. min The gate value is calculated as follows: ; Where, 0≤g min <1 represents a preset or learnable lower limit parameter.

[0036] In this embodiment, the learnable univariate function can be implemented using a piecewise linear function with learnable parameters, a spline interpolation function, or a lightweight multilayer perceptron (with an input dimension of 1 and an output dimension of 1), whose parameters are jointly optimized with other parts of the model through end-to-end gradient descent. In one embodiment, when using a spline interpolation function or a lightweight multilayer perceptron, L2 regularization can be applied to the function output to control smoothness.

[0037] This invention concatenates the normalized visual token representation H, the visual attention concentration score S, and the two-dimensional position code P into a gated input. This approach enables the gating module to simultaneously perceive three complementary types of information: normalized visual tokens provide semantic representations of image patches, visual attention aggregation scores provide structural anomalies in attention propagation, and two-dimensional positional encoding provides spatial location priors. This ternary fusion mechanism addresses the problem of singular gating inputs in traditional schemes. Specifically, relying solely on visual tokens, the gating module struggles to distinguish whether high activation values ​​originate from the target region or background noise; relying solely on aggregation scores, the gating module lacks understanding of the patch's semantic content; and relying solely on positional information, the gating module cannot adapt to changes in image content. After fusing these three elements, the gating module can make comprehensive decisions: for example, for patches located at image edges with abnormally high aggregation scores but moderate visual token activation values, the gating module can comprehensively determine them as background redundancy regions and apply suppression; for patches located at the image center with moderate aggregation scores but significant visual token activation values, the gating module can determine them as target regions and retain them. This multi-dimensional perception capability significantly improves the accuracy and robustness of gating decisions.

[0038] In visual Transformers, different attention heads typically learn differentiated visual patterns spontaneously through training. For example, some heads focus on local edges and texture details, some model long-range region relationships, and some focus on specific semantic categories. If a uniform gating is applied after concatenating multiple heads, the outputs of all heads are modulated by the same gating matrix, causing the fine features of edge-sensitive heads and the global features of region relationship heads to be treated equally, thus smoothing out the functional differentiation of the multi-head mechanism. This invention explicitly limits the gating position to after each attention head completes its SDPA output and before concatenating multiple attention heads, meaning that a gating matrix is ​​generated independently for each attention head. and independently apply element-wise modulation This location selection ensures that the SDPA aggregation result of each attention head is first fully preserved, and then independently modulated according to the head's unique attention aggregation pattern, visual token distribution, and spatial location information. For example, for attention heads focused on edge detection, the gating module can suppress abnormal edge responses caused by complex background textures, preserving the true object contour edges; for attention heads focused on long-distance dependencies, the gating module can suppress the interfering attention of local noise patches to distant patches, maintaining the clarity of global contextual relationships. This head-by-head independent modulation mechanism maintains the original design intent of the multi-head attention mechanism, enabling different visual modes to be effectively enhanced or suppressed under their respective optimal gating strategies.

[0039] The gating matrix generated by this invention The dimension is This fine-grained modulation extends to each sample, each image patch, each attention head, and each channel dimension. The gating value is constrained to the (0,1) range using a sigmoid mapping. When the gating value approaches 1, the corresponding attention output is largely preserved; when it approaches 0, the output is completely suppressed; and at intermediate values, partial preservation is achieved. This element-wise modulation granularity is far coarser than strategies that uniformly scale the entire attention head output or uniformly suppress the entire patch. It enables selective filtering along the channel dimension: for the same image patch, some channels may encode background texture information and be suppressed, while others may encode target semantic information and be preserved. This fine-grained control allows gating modulation to precisely target specific dimensions in the feature space, avoiding the loss of useful information caused by the "one-size-fits-all" approach of traditional coarse-grained gating.

[0040] In this implementation, the visual attention aggregation score provides anomaly perception signals at the attention structure level for the learnable unary function, enabling its nonlinear response to learn from real attention propagation anomalies. The nonlinear expressive power of the learnable unary function allows the gating decision of the ternary fusion input to transcend the limitations of linear combinations. The ternary fusion input ensures that the gating decision simultaneously considers semantic, structural, and spatial dimensions. The head-by-head gating position selection after SDPA output allows the above-mentioned fine-grained decision to be applied in multi-head independent spaces. The element-by-element modulation granularity implements the decision results to the finest feature dimension. Ultimately, this collaborative mechanism effectively suppresses anomalous attention propagation in background regions, repetitive textures, reflective regions, occluded regions, noisy regions, or non-target salient regions, while enhancing the feature representation capabilities of target regions, small defect regions, small lesion regions, or remotely sensed small target regions. This significantly improves the feature discrimination and task adaptability of the visual Transformer in complex visual scenes.

[0041] Furthermore, according to one embodiment of the present invention, calculating a visual attention concentration score based on an attention weight matrix includes one or a combination of the following methods: For the h-th attention head, the average intensity of attention given to the j-th image patch by other image patches is calculated as the visual attention aggregation score of that image patch; A unified clustering score is obtained by averaging, taking the maximum value, or weighted summing the visual attention clustering scores of multiple attention heads. The visual attention concentration score is constructed by combining image region priors, saliency estimation, or background estimation.

[0042] In this embodiment, for the h-th attention head, the average intensity of attention of the j-th image patch by all other image patches is calculated. This calculation method directly mines the attention weight matrix. The column-oriented statistical properties. In the attention weight matrix, the elements in the j-th column... This column represents the attention level of the i-th image patch to the j-th image patch. Averaging this column yields the global attention level of that image patch within that attention head. This column averaging strategy has a clear physical meaning: when an image patch receives high-weight attention from a large number of other image patches due to its significant visual features (such as high-contrast edges, strong reflections, or repetitive textures), its column average is accumulated and amplified, thus identifying it as an attention aggregation anomaly. This method is computationally simple, requiring only one column-wise reduction operation on the attention weight matrix, resulting in extremely low computational overhead. Furthermore, it preserves the independent aggregation pattern perception capability of each attention head, enabling subsequent gating modulation to accurately respond to specific anomalies in different attention heads.

[0043] A unified clustering score S(j) is obtained by averaging, maximizing, or weighted summing the visual attention clustering scores of multiple attention heads, elevating single-head perception to a multi-head collaborative perception level. Different attention heads spontaneously learn different visual patterns during training: some heads may be sensitive to edge features, some to color distribution, and some to semantic categories. Clustering anomalies within a single head may be normal responses to that head's specific visual pattern (e.g., edge heads showing high attention to strong edge patches) rather than genuine anomaly propagation; however, when the clustering scores of multiple heads simultaneously show anomalies, the credibility of the image patch exhibiting attention clustering anomalies is significantly increased. The averaging strategy smooths out the specific biases of individual heads and is suitable for identifying anomaly clusters with cross-modal consistency; the maximizing strategy preserves sensitivity to the most significant anomalies and is suitable for detecting extreme anomalies; the weighted summing strategy assigns differentiated weights based on the task contribution or reliability of each head, achieving adaptive multi-head information fusion. This cross-head statistical mechanism effectively reduces the false positive rate of single-head perception, making the clustering score more robust in distinguishing genuine attention propagation anomalies.

[0044] This paper constructs a visual attention clustering score by combining image region priors, saliency estimation, or background estimation, deeply integrating data-driven statistical computation with knowledge-driven visual priors. Image region priors can utilize structured information in the image, such as center priors (the central region of an image is more likely to contain the target subject), edge priors (the boundary regions of an image are more likely to belong to the background), or task-specific spatial distribution priors (such as the prior importance of the eye region in face detection). Saliency estimation can introduce external saliency models or internal bottom-up saliency computation to identify visually prominent regions in the image, avoiding misclassification of these regions as abnormal clusters due to high saliency. Background estimation can reduce or correct the clustering scores of known background regions through foreground-background segmentation or scene parsing, reducing the interference of background patches on gating decisions. This prior fusion mechanism transforms the clustering score from a purely mathematical statistical quantity into a discriminative index with visual semantic interpretability, adapting to the specific needs of different visual tasks and scenarios, and improving the discrimination accuracy and task relevance of the clustering score in complex visual environments.

[0045] The three calculation methods described above, provided in one or a combination, enable the construction of visual attention concentration scores to possess high flexibility and scalability. In deployment scenarios with limited computing resources, a single-head column averaging strategy can be used to maintain the lowest computational overhead. In scenarios with high accuracy requirements, cross-head statistics can be superimposed to improve the reliability of anomaly detection. In specific tasks with clear visual priors (such as the prior location distribution of lesion regions in medical images or the prior range of target scale in remote sensing images), image priors can be further integrated to achieve task-customized anomaly perception. The combination of multiple strategies is not a simple superposition, but rather a synergistic enhancement achieved through the fusion of complementary information: single-head column averaging provides fine-grained intra-head anomaly signals, cross-head statistics provide robust multi-head consistency verification, and image priors provide task-related semantic constraints. The combination of these three enables visual attention concentration scores to comprehensively characterize the abnormal state of attention propagation at different levels and dimensions, providing high-quality, multi-perspective decision-making basis for subsequent learnable univariate function gating modules.

[0046] Furthermore, according to one embodiment of the present invention, the learnable univariate function includes spline functions, piecewise linear functions, or polynomial functions; For the h-th attention head and the c-th channel, the gated logits are obtained by summing the components of the gated input after transforming each learnable unary function corresponding to that channel, and then obtaining the gated value through Sigmoid mapping, thus forming the gated matrix.

[0047] In this embodiment, the spline function, through piecewise polynomials and node continuity constraints, can precisely fit the local characteristics of different input intervals while maintaining overall smoothness. In the gated generation scenario, the spline function can learn differentiated gated response curves for low, medium, and high value intervals of visual attention aggregation scores. For example, it maintains an approximate identity mapping in the low aggregation score interval to preserve target region features, and learns a steep descent in the high aggregation score interval to suppress background redundancy propagation. Furthermore, the continuity between nodes ensures smooth changes in the gate value at interval transitions, avoiding gradient instability issues caused by hard thresholds. The piecewise linear function achieves piecewise nonlinear response with low computational complexity by splicing multiple linear segments. The slope and intercept of each segment are independently learnable, enabling the gated module to form sharp transition characteristics near specific input values. This is suitable for scenarios that require a clear distinction between the decision boundaries of preservation and suppression, and its forward computation only requires finding the corresponding linear segment, resulting in significantly lower inference overhead than higher-order nonlinear functions. Polynomial functions generate globally smooth nonlinear curves through linear combinations of monomials. The coefficients of each order collectively determine the overall shape of the curve, enabling the establishment of a continuous nonlinear relationship between the input and the gate value across the entire input domain. This makes them suitable for scenarios where the gated response needs to smoothly transition monotonically or non-monotonicly with the input value. These three function types cover different needs, from local fine-fitting to globally smooth mapping, and from low computational overhead to high expressive power. This allows the gating module to be flexibly selected based on the computational constraints and task characteristics of the deployment environment, overcoming the expressive power bottleneck of a single linear gating mapping form.

[0048] For the h-th attention head and the c-th channel, the gated logits are represented as follows: ,in Let m be the learnable univariate function corresponding to the m-th gated input component. This refers to the specific value of the component. The core of this structure lies in the independent transformation of each component: each gated input component (a certain dimension of the normalized visual token, the visual attention aggregation score, or a certain dimension of the two-dimensional position encoding) is first independently nonlinearly mapped through its own learnable unary function, and then the mapping results are summed to obtain the final gated logits. This structure differs from the traditional approach of linearly projecting the concatenated gated input as a whole. Its technical advantages are reflected in three aspects: First, the physical meaning and numerical distribution characteristics of each input component differ significantly (e.g., visual token components typically follow a specific distribution of semantic activation values, cluster scores are probabilistically normalized scalars, and positional encodings are periodic or learned spatial coordinates). Independent transformation allows each component to have a function shape and parameter space that matches its characteristics, avoiding numerical scale conflicts and semantic confusion caused by unified projection. Second, independent transformation makes the contributions of each component separable, quantifiable, and interpretable. After training, the response curves of each univariate function can be visualized to directly analyze how specific visual feature dimensions, cluster score intervals, or spatial locations affect gating opening or closing, significantly improving the interpretability of the gating mechanism. Third, the summation and aggregation method maintains the linear additivity of the gating logits to the transformation results of each component, allowing gradients to smoothly propagate back along independent paths during backpropagation. This avoids gradient vanishing or gradient explosion problems that may occur with complex nested structures, ensuring training stability.

[0049] The gated logits are mapped to the gated value using the Sigmoid function. The technical effects of this mapping process are reflected in three aspects: numerical constraints, decision smoothing, and gradient characteristics. At the numerical constraint level, Sigmoid compresses unbounded logits into the open interval (0,1), giving the gated value the inherent physical semantics of complete preservation (approaching 1), complete suppression (approaching 0), and partial modulation (intermediate value), without requiring additional normalization or truncation operations. Furthermore, this constraint aligns perfectly with the mathematical form of element-wise gated modulation. The high degree of fit ensures the stability of the modulated feature values. In terms of decision smoothing, the Sigmoid exhibits an approximately linear response when the absolute value of logits is small, making the gate value sensitive to small changes in the input; when the absolute value of logits is large, it exhibits saturation characteristics, causing the gate value to approach extreme states (0 or 1), forming a clear retention / suppression decision boundary. This boundary is a continuous and smooth transition rather than a hard threshold jump, avoiding oscillations in the gate decision near the input critical value. In terms of gradient characteristics, the derivative of the Sigmoid... Automatic gradient decay when the output approaches 0 or 1, applying gradient suppression to gating units that have made clear decisions to prevent their parameters from being over-updated; maintaining a large gradient magnitude when the output is in the middle range to promote the continued learning of gating units that have not yet converged. This adaptive gradient characteristic naturally realizes the gradual formation of gating sparsity, forming a synergistic effect with the sparse constraint regularization term in the training phase.

[0050] Each channel of each attention head corresponds to a set of learnable unary functions. This configuration strategy refines the gating generation mechanism to the head-channel level. Its technical effect is that different attention heads focus on differentiated visual patterns (such as edges, textures, and semantic regions), while different channels within the same head encode differentiated feature dimensions (such as color, shape, and spatial frequency). The head-channel level function configuration allows gating generation to learn its optimal gating response strategy for each head-channel combination. For example, for the channel encoding high-frequency textures in an attention head focused on texture analysis, its corresponding unary function can learn strong suppression in high-aggregation score intervals to filter out noisy textures; while for the channel encoding semantic activation in an attention head focused on object detection, its corresponding unary function can maintain a high gating value in medium-aggregation score intervals to preserve potential target features. This ultra-fine-grained configuration matches the number of parameters in the gating module with the expressive power of the visual Transformer, trading moderate parameter overhead for the accuracy and adaptability of gating decisions, avoiding the loss of useful information or residual noise information caused by a one-size-fits-all approach to a uniform gating strategy.

[0051] Furthermore, according to one embodiment of the present invention, element-wise gated modulation of the SDPA output of the attention head is performed using a gating matrix, including: The gating matrix is ​​multiplied element-wise with the SDPA output of the corresponding attention head. When the gating value approaches 1, the corresponding attention output is retained; when the gating value approaches 0, the corresponding attention output is suppressed; and when the gating value is between 0 and 1, the corresponding attention output is partially retained.

[0052] In this embodiment, the gating matrix With SDPA output Element-wise multiplication While mathematically concise, its technical essence lies in precisely implementing gating decisions down to each scalar element of the tensor. This operation is performed independently across the batch dimension B, the image patch dimension N, the attention head dimension h (implicit context), and the channel dimension d, ensuring that each image patch in each sample receives independent modulation coefficients in each channel of each attention head. This four-dimensional fine-grained granularity differs from coarse-grained strategies that uniformly scale the entire head output or uniformly suppress the entire feature block: uniform scaling fails to distinguish the information value of different channels within the same head, potentially treating channels encoding key semantics and those encoding noise equally; uniform suppression fails to distinguish the spatial aliasing of different visual attributes within the same block, potentially treating target and background regions in a bundled manner. Element-wise multiplication breaks this bundling, enabling gating modulation to precisely locate specific scalar elements in the feature space, achieving surgical selective filtering of the attention output.

[0053] After being mapped by the Sigmoid algorithm, the gate values ​​are distributed in the open interval (0,1). This continuous value space is naturally divided into three functional regions in terms of modulation semantics, and the transition between the three states is smooth rather than abrupt. When the gate value approaches 1, the multiplication result of the corresponding element is approximately equal to the original SDPA output. This attention feature is essentially lossless and can be passed to subsequent layers, making it suitable for preserving target regions, key edges, or semantically significant features. When the gate value approaches 0, the multiplication result of the corresponding element is approximately zero, and this attention feature is effectively blocked, making it suitable for suppressing background redundancy, noise interference, or abnormal cluster propagation. When the gate value is in the intermediate range between 0 and 1, the corresponding element is partially scaled, achieving a gradual attenuation or enhancement of the attention feature, which is suitable for soft decision processing in regions of uncertain confidence. The technical advantages of this three-state continuous decision-making mechanism are: it avoids the gradient non-differentiability and information cliff loss caused by hard threshold gating (such as only taking 0 or 1), enabling the optimization of gating parameters during backpropagation to be based on the complete gradient signal; at the same time, it avoids the noise accumulation problem caused by single-state gating (such as only normalizing without suppression), enabling the gating module to truly have information filtering function. The continuous decision space also provides a mathematical basis for the gradual formation of gating sparsity. In the early stage of training, the gating values ​​are mostly distributed in the middle interval to explore the optimal decision boundary. In the later stage of training, some gating values ​​gradually migrate to the extreme states at both ends under the action of the sparse constraint regularization term, eventually forming a stable and interpretable retention / suppression pattern.

[0054] The execution position of element-wise gated modulation is limited to after each attention head completes its SDPA output and before multiple attention heads are concatenated. This positional constraint ensures that element-wise multiplication is performed in the independent subspace of each attention head. Instead of being executed on a unified space after multi-head concatenation, each attention head performs its own gating decision independently. This technical detail ensures that the gating decisions of different attention heads do not interfere with each other: heads focused on edge detection can independently suppress false edge responses caused by complex background textures while preserving the true object contours; heads focused on long-range dependencies can independently block the interfering attention of local noise patches to distant patches while maintaining the integrity of global contextual relationships; heads focused on semantic activation can independently enhance the feature responses of the target category while suppressing the confusing activation of similar-looking background categories. If gating is placed after multi-head concatenation, the outputs of all heads are compressed into the same feature space, and a single gating matrix cannot adaptively modulate the differentiated visual patterns of different heads, resulting in the functional differentiation of the multi-head mechanism being smoothed out. Independent modulation for each attention head makes each attention head an "expert unit" with autonomous information filtering capabilities. Its output is independently gated before concatenation, which not only preserves the diversity advantages of the multi-head mechanism, but also achieves quality screening of each expert's output through gating.

[0055] Furthermore, according to one embodiment of the present invention, during the training phase, a sparse constraint regularization term is applied to the gating matrix or the learnable univariate function. The sparse constraint regularization term includes at least one of the following: gating value average activation regularization term, gating entropy regularization term, function edge contribution regularization term, or channel activation sparse regularization term.

[0056] In this implementation, the gating value average activation regularization term penalizes the global activation level of the gating matrix, causing the gating values ​​to migrate towards lower activation regions throughout the training process. This regularization term is typically based on the mean of all elements in the gating matrix or... The norm, as a penalty target, works by pushing the gating values ​​of a large number of background or noisy patches towards zero. This systematically suppresses the attention outputs of these patches during element-wise multiplication, effectively truncating their information propagation paths. Simultaneously, the gating values ​​of the target region patches maintain a high activation level due to the backpropagation requirements of task loss, thus creating a sparse contrast pattern of low background activation and high target activation globally. This global sparsity not only reduces the propagation interference of invalid information in subsequent network layers but also lowers the overall computational burden of the gating module, as the multiplication of gating values ​​approaching zero with the SDPA output can be optimized at the hardware level as conditional skipping or zero-value compression.

[0057] The gating entropy regularization term drives the gating output from fuzzy, moderately active states to definite extreme states (0 or 1) by minimizing the information entropy of the gating value distribution. The physical meaning of information entropy in the probability distribution is a measure of uncertainty: entropy is maximum when the gating value is uniformly distributed around 0.5, indicating high uncertainty in the gating decision; entropy is minimum when the gating value is polarized to 0 or 1, indicating high certainty in the gating decision. The technical effect of this regularization term is to accelerate the convergence of the gating module's decision. In the early stages of training, the gating values ​​are mostly in the exploratory intermediate range, and the entropy regularization term applies gradient pressure to shift them towards both ends; in the later stages of training, the gating values ​​gradually solidify into preserved or suppressed states, forming a clear binarization tendency. This increase in certainty has direct value for visual tasks: background and target regions in images usually have clear boundaries, and fuzzy gating decisions can lead to aliasing of feature representations in the boundary regions, while highly deterministic gating can create sharp preservation / suppression switching at the boundaries, enhancing the spatial clarity of the feature map.

[0058] The function edge contribution regularization term sparsifies the intrinsic structure of learnable univariate functions, penalizing the marginal contribution of each function edge to the gating output. Taking spline functions as an example, which consist of piecewise polynomial edges between multiple nodes, the contribution of each edge can be quantified by its activation frequency, output variance, or gradient magnitude. For piecewise linear functions, the slope and intercept parameters of each linear segment directly represent the function edges. The technical effect of this regularization term is to identify and weaken low-contribution components at the function structure level: when a function edge remains in a low-activation state for a long period during training, or when its parameter changes have a negligible impact on the gating output, the contribution regularization term applies contraction pressure to the corresponding parameters of that edge, causing it to gradually degenerate into an identity mapping or a zero mapping, ultimately allowing it to be safely pruned during the inference phase. This structural sparsity optimization differs from sparsifying the gating values ​​themselves; it directly acts on the internal components of the gating generation mechanism, enabling the gating module to achieve structural lightweighting while maintaining functional integrity, laying the foundation for model compression in the subsequent deployment phase.

[0059] The channel activation sparsity regularization term optimizes the long-term activation statistics of channels within the attention head, causing the gating values ​​of some channels within the same head to systematically approach zero. In the visual Transformer, different channels of the same attention head typically encode differentiated feature dimensions (such as color, texture, shape, and spatial frequency), but not all channels have equal value for the current task. The technical effect of this regularization term is to achieve automatic feature selection within the head: during training, the gating values ​​of channels with low task contribution are suppressed under the pressure of the regularization term, and the SDPA output of the corresponding channels is blocked from propagation; the gating values ​​of channels with high task contribution remain active with the support of the task loss gradient, and the features of the corresponding channels are preserved and enhanced. This in-head channel selection mechanism enables each attention head to adaptively focus on the feature subspace that is most discriminative to its specific visual pattern, filtering out redundant or interfering feature dimensions, and improving the purity and efficiency of in-head representation.

[0060] The four sparse constraint regularization terms mentioned above can be flexibly combined in at least one form, and their combination strategy can be customized according to task characteristics and deployment requirements. The gating value average activation regularization term and the gating entropy regularization term focus on sparsity at the gating output level, applying pressure from the two dimensions of global activation level and decision determinism, respectively; the function edge contribution regularization term and the channel activation sparse regularization term focus on sparsity at the gating structure level, performing pruning pre-training from the function component and channel dimensions, respectively. When multiple regularization terms are used in combination, their sparsity effect is synergistically amplified: the sparsity at the output level provides a clear optimization direction for the sparsity at the structure level (i.e., first determine which gating values ​​should be suppressed, and then trace back to the function components and channels that generate these gating values), while the sparsity at the structure level provides a mechanism guarantee for the sparsity at the output level (i.e., by removing redundant function edges and channels, fundamentally reducing the source of parameters that generate fuzzy gating decisions). This synergistic effect enables the gating module to not only possess high-quality feature selection capabilities after training, but also a highly sparse and compressible network structure, achieving a dual optimization of performance and efficiency.

[0061] Furthermore, according to one embodiment of the present invention, after training is completed, the contribution of each learnable univariate function edge in the gating module to the gating output is calculated based on the activation statistics of the gating matrix, and the function edges that meet the preset pruning conditions are pruned. The preset pruning conditions are determined based on at least one of the following indicators: the long-term mean activation of the function edge is lower than a first threshold, the activation variance of the function edge is lower than a second threshold, or the contribution of the function edge output to the gated logits is lower than a third threshold.

[0062] In this implementation, pruning decisions are based on the activation statistics of the gating matrix after training, rather than relying on human experience or heuristic rules, ensuring the objectivity and reproducibility of the pruning process. The gating matrix, as the actual output record of the gating module on a large number of samples, directly reflects the activation patterns of each function edge in real inference scenarios: the long-term activation mean reflects the frequency and intensity of function edge invocation, the activation variance reflects the dynamic range of function edge response to input changes, and the contribution of the output to the gating logits reflects the marginal influence of the function edge in the final gating decision. Calculating the function edge contribution based on these three types of statistics transforms the pruning problem into a quantifiable numerical comparison problem, turning low contribution from a qualitative description into a quantitative standard that can be accurately determined by a threshold. This data-driven pruning method differs from traditional pruning methods based on parameter magnitude or gradient sensitivity. Small parameter magnitudes do not necessarily mean functional redundancy (e.g., some key function edges may achieve precise mapping through parameter cancellation), and low gradient sensitivity does not necessarily mean stable output (e.g., the gradient of function edges in saturated regions is approximately zero, but the output contribution is significant). The activation statistics of the gating matrix are directly related to the actual functional performance of the function edges, making the pruning decision closer to the real working mechanism of the gating module.

[0063] The preset pruning conditions are determined based on at least one of three complementary indices. Each indices characterize the functional state of the function edge from different dimensions. When used together, they can form multidimensional cross-validation and reduce the risk of misjudgment. A long-term activation mean below the first threshold indicates that the function edge is in a near-zero activation state on the vast majority of samples, and its existence has a negligible impact on the overall gating output, belonging to a typical dormant component. An activation variance below the second threshold indicates that the function edge's response to input changes is extremely flat. Regardless of input fluctuations, its output always remains at a fixed level, losing the nonlinear modulation capability that a learnable univariate function should have, degenerating into an approximate identity mapping or constant shift. A function edge output contribution to gating logits below the third threshold indicates that the absolute value contribution of the function edge in the aggregation and summation of gating logits is extremely small. Even if the edge is removed and compensated by fine-tuning the parameters of other edges, the overall distribution of the gating output will not change significantly. While single indicators may have exceptions (e.g., a function edge with low mean but high variance might play a significant role on a few key samples), joint discrimination using multiple indicators can effectively eliminate such exceptions: when a function edge simultaneously satisfies low mean, low variance, and low contribution, its functional redundancy has a high degree of confidence, and the safety of the pruning operation is fully guaranteed. The numerical settings of the three thresholds can be adjusted according to deployment accuracy requirements and computational budget. Lowering the first, second, and third thresholds implies a more conservative pruning strategy (retaining more function edges to maintain accuracy), while raising the thresholds implies a more aggressive pruning strategy (removing more function edges to compress the model), giving the pruning process a flexible ability to balance accuracy and efficiency.

[0064] Removing function edges that meet preset pruning conditions directly reduces the number of parameters and computational complexity of the gating module. Taking spline functions as an example, each function edge corresponds to the polynomial coefficients or basis function weights between nodes; pruning the edge removes the corresponding parameter storage and forward computation overhead. For piecewise linear functions, each function edge corresponds to the slope and intercept of a linear interval; after pruning, the interval is merged into adjacent intervals or degenerates into a single global mapping. This lightweight structure is particularly effective during deployment: the pruned gating module maintains its core gating decision-making capabilities while significantly reducing inference latency and memory usage, making it adaptable to resource-constrained scenarios such as edge devices, mobile devices, or real-time vision systems. More importantly, since pruning is based on post-training activation statistics rather than dynamic gradients during training, the pruning operation does not interfere with the optimization process during training. Training and compression are decoupled, avoiding the complex trade-off between accuracy loss and compression rate in joint optimization.

[0065] The post-training pruning process naturally involves ranking and filtering the contributions of function edges, and this ranking result can be directly used for the visualization interpretation of the gating module. The contribution value of each function edge reflects the influence of a specific gating input component (such as a dimension of a normalized visual token, a visual attention aggregation score, or a dimension of a two-dimensional positional encoding) on ​​the gating decision within a specific value range. Comparing and visualizing high-contribution function edges with low-contribution function edges clearly reveals the decision logic of the gating module: for example, a high-contribution function edge may exhibit a steep decreasing curve in the high-value range of the visual attention aggregation score, intuitively indicating the gating channel's suppression strategy for high-aggregation patches; a low-contribution function edge may exhibit a nearly flat response across the entire input range, intuitively indicating that the function edge did not participate in effective gating decisions. The pruning operation removes the latter, allowing the visualization results to focus on the key components that truly affect the gating output, reducing analytical complexity and enhancing the understandability and auditability of model behavior. This interpretability is of great value for the application of visual Transformers in high-risk scenarios (such as medical image diagnosis and autonomous driving perception). When the model makes a decision to preserve or suppress the features of a certain image patch, the causal relationship between the decision and the degree of attention concentration, spatial location or visual semantics can be explained by tracing back to the response curve of the specific function edge.

[0066] In this embodiment, the enhanced image features are used as input to the downstream visual task module. The enhanced image features, after element-wise gating modulation and output projection, outperform the original multi-head attention output in three dimensions: semantic discriminativity, spatial clarity, and noise robustness. Regarding semantic discriminativity, the aberrant attention propagation of background regions, repetitive textures, and noise patches is systematically suppressed by the gating mechanism, significantly reducing their feature response amplitude. Meanwhile, effective features of target regions, key edges, and semantically significant patches are selectively preserved or enhanced, increasing the inter-class distance and intra-class compactness of different semantic categories in the feature space, providing a more easily separable feature distribution for downstream classification, detection, or segmentation tasks. Regarding spatial clarity, the joint input of visual attention aggregation score and two-dimensional positional encoding enables the gating decision to possess spatial perception capabilities, forming sharp feature transitions between image boundaries and centers, and between foreground and background. This reduces the spatial blurring effect caused by global averaging in traditional attention mechanisms, providing more accurate spatial positioning information for downstream tasks. In terms of noise robustness, the sigmoid constraint gating value suppresses abnormally large attention outputs to a limited range, avoiding interference from extreme activation values ​​on gradient propagation in subsequent layers. This allows downstream modules to receive more stable feature inputs during training, reducing their sensitivity to input perturbations.

[0067] The enhanced image features are output in the form of standardized tensors, with a data format completely consistent with the standard output of each layer of the visual Transformer. This allows for direct integration with various downstream modules in existing visual task systems, without requiring adaptation modifications to the network structure or input interface of the task head. This format compatibility enables plug-and-play deployment: in image classification tasks, enhanced features can be input into global average pooling layers and fully connected classification heads; reduced background redundancy after gating suppression makes classification decisions more focused on the target object. In object detection tasks, enhanced features can be input into region proposal networks or feature pyramid networks; improved spatial clarity makes candidate box localization more accurate. In image segmentation tasks, enhanced features can be input into upsampling decoders; patch-by-patch gating modulation ensures high consistency between segmentation boundaries and gated regions. In multimodal fusion tasks, enhanced features can be cross-modal aligned with text features; improved visual semantic purity makes image-text matching more accurate. The diversity of downstream task modules does not impose additional constraints on the output format of enhanced features, indicating that the gating augmentation mechanism serves a general level of visual representation. Its technical value does not depend on specific task assumptions and has cross-task universality.

[0068] The enhanced image features, as the intermediate layer output, follow a path identical to the original visual Transformer in their propagation to downstream task modules. Parameter optimization of the gating module can be jointly trained end-to-end with the downstream task loss through standard backpropagation. Specifically, the gradient of the downstream task loss on the enhanced features is backpropagated through the output projection layer to the gating modulation stage, then flows to the gating matrix and SDPA output via an element-wise multiplication gradient distribution mechanism. Finally, it is jointly updated through the learnable unary function parameters of the gating module and the query, key, and value projection parameters of the attention head. This coherent gradient propagation path ensures consistent optimization between the gating enhancement mechanism and the downstream task objective: the gating module is not pre-trained as an independent auxiliary task, but directly participates in the gradient backpropagation of the main task loss, and its parameter update direction always serves to reduce the downstream task error. This end-to-end optimization deeply couples gating decisions with task requirements. For classification tasks, the gating module tends to retain the patch features that are most informative for class discrimination; for detection tasks, the gating module tends to enhance local regions containing target bounding boxes; for segmentation tasks, the gating module tends to maintain consistency within regions of the same class and suppress cross-class confusion. Task-driven adaptive optimization automatically aligns the statistical properties of the enhanced features with the expected input distribution of downstream modules, reducing the need for manually designing feature selection rules.

[0069] According to the above-described solution of the present invention, the present invention can be applied to at least the following application scenarios: Basic Visual Classification: In the image classification model, the input image is segmented into fixed-size patches and embedded as visual tokens. The model computes multi-head attention in each visual Transformer block. For each attention head, this invention generates and applies a learnable unary function gating after the SDPA output. Attention outputs overly influenced by background or texture patches are suppressed, while patch features relevant to the target subject are preserved, thereby improving the discriminative power of the classification features.

[0070] Industrial Defect Detection: In defect detection on metal, glass, fabric, battery cell, or wafer surfaces, reflections, periodic textures, and background areas can generate strong attentional concentration, causing small defect areas such as scratches, cracks, dents, and stains to be obscured. This invention identifies anomalous clustered patches through visual attentional concentration scoring, and then uses a learnable unary function gating to fine-grainedly suppress the corresponding attention head output, making the effective features of the defect area easier to identify in subsequent detection heads.

[0071] Medical image recognition: In CT, MRI, ultrasound, fundus images, or pathological slide images, lesion areas are typically small, while normal tissue areas are large. This invention can utilize patch-level attention concentration scoring to identify redundant propagation in large areas of normal tissue or artifacts, and enhance the feature representation of suspected lesion areas through function gating.

[0072] Remote sensing small target recognition: In remote sensing images or UAV images, targets such as small boats, small vehicles, small buildings, and disaster areas may be obscured by background interference such as sea surface, roads, vegetation, and building textures. This invention can suppress abnormal attention propagation of background patches based on visual attention concentration scoring and two-dimensional location information, thereby improving the detection or segmentation effect of small targets.

[0073] Autonomous driving and multimodal vision models: In autonomous driving scenarios, distant pedestrians, traffic signs, cones, and low-light targets are easily interfered with by background areas such as roads, vehicle bodies, sky, and light reflections. This embodiment can serve as an attention modulation module for the visual backbone to enhance key target patches.

[0074] Furthermore, the present invention can also be applied to visual language models to modulate visual attention output before or after the fusion of image patch tokens and text tokens, thereby improving the reliability of image region features in multimodal understanding.

[0075] According to the above-described scheme of this invention, the present invention organically integrates visual attention aggregation scoring, learnable univariate function gating, and head-to-head modulation after SDPA output to achieve refined adaptive control of attention propagation and produce multiple technical effects: Visual attention aggregation scoring is calculated based on the attention weight matrix, explicitly quantifying the degree of abnormal attention to each image patch to identify background redundancy propagation or local over-dominance, thereby suppressing ineffective propagation of background and noise patches; a gating module composed of learnable univariate functions such as spline functions, piecewise linear functions, or polynomial functions is constructed, using the normalized visual token, aggregation score, and two-dimensional position code as input, breaking through the nonlinear expression bottleneck of linear gating, generating differentiated responses for different value ranges, and the response curve of the learnable univariate function can be intuitively plotted. The system analyzes how specific visual features, spatial locations, or clustering scores influence gating decisions, significantly improving interpretability. Element-wise gating modulation is applied independently after each attention head completes its SDPA output and before multi-head stitching, preserving multi-head functional differentiation and enabling differentiated control over different visual modes such as edges, textures, shapes, and long-distance dependencies. This enhances the representation of small targets and defect regions, reducing the overwhelming of key features in scenes such as industrial defects, medical lesions, and remote sensing small targets by large background areas. The gating value is restricted to the (0,1) interval by a Sigmoid function, suppressing abnormally large attention outputs and improving training and inference stability. After training, the contribution of function edges is evaluated based on the activation statistics of the gating matrix. Function edges with long-term low activation, low variance, or low contribution are pruned, achieving lightweight structure and low-overhead deployment. These core mechanisms work synergistically to significantly improve the feature discriminative power and task adaptability of the visual Transformer in complex visual scenes.

[0076] Furthermore, to achieve the above objectives, the present invention also provides an image feature enhancement system based on learnable univariate function gating, comprising: The image block embedding module divides the input image into multiple image blocks, maps each image block to a visual token to obtain a visual token sequence, and embeds the two-dimensional positional encoding of the image blocks into the visual token sequence; The normalization module normalizes the visual token sequence to obtain a normalized visual token. The multi-head attention calculation module calculates the query matrix, key matrix, and value matrix corresponding to each attention head in the multi-head attention layer based on the normalized visual token, and obtains the attention weight matrix and SDPA output of the attention head through scaled dot product attention calculation. The aggregation score calculation module calculates a visual attention aggregation score based on the attention weight matrix. The visual attention aggregation score is used to quantify the degree of aggregation of the corresponding image patch by other image patches, so as to identify whether the image patch belongs to a target region that is abnormally focused, a background region that generates redundant propagation, or a local region that over-dominates global features. The gating module receives the normalized visual token, the visual attention aggregation score, and the two-dimensional position code. The gating module is composed of learnable unary functions. Each channel of each attention head corresponds to a set of learnable unary functions. Each set of learnable unary functions contains the same number of learnable unary functions as the gating input components. Each learnable unary function is composed of at least one function edge, which is used to generate a gating matrix. The gated modulation module uses the gate matrix to perform element-wise gated modulation on the SDPA output of each attention head after it completes its SDPA output and before multiple attention heads are spliced ​​together, so as to obtain the gated attention head output. The output module stitches together the outputs of multiple gated attention heads and then projects the outputs to obtain enhanced image features.

[0077] The image feature enhancement system based on learnable univariate function gating according to the present invention can realize the image feature enhancement method based on learnable univariate function gating. The specific process steps are as described above and will not be repeated here.

[0078] Furthermore, to achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the image feature enhancement method based on learnable univariate function gating as described above.

[0079] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image feature enhancement method based on learnable univariate function gating as described above.

[0080] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely one preferred embodiment of the invention and are only used to explain the invention. They do not limit the scope of protection of the invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0081] Example 1

[0082] This embodiment provides an image feature enhancement method based on learnable unary function gating. The input image is segmented into patches to form visual tokens. In a multi-head attention layer, the SDPA output of each attention head is calculated first. Then, a visual attention aggregation score is calculated based on the attention matrix to characterize the degree to which a patch is abnormally focused, redundantly propagated in the background region, or excessively dominated by a local region in the global features. The visual token, visual attention aggregation score, and two-dimensional patch position information are then input into a gating module composed of learnable unary functions. Finally, after each attention head completes its SDPA output and before multiple attention heads are concatenated, element-wise gating modulation is performed on the output of each attention head.

[0083] Specifically, the image feature enhancement method based on learnable univariate function gating in this embodiment includes: (1) Image patch representation Obtain the input image I and divide it into multiple image patches. Each image patch is processed by linear mapping, convolutional mapping, or other patch embedding modules to obtain a visual token. All visual tokens form a sequence: ; in, This refers to the batch size. The number of patch tokens. This represents the hidden dimension of the visual token. Optionally, a two-dimensional positional encoding is added to the visual token to represent the row and column position of the patch in the original image.

[0084] (2) SDPA output of each attention head Visual Normalization is performed to obtain normalized vision. For the first Each attention point, calculate: ; ; ; ; ; in, This represents the attention relationship between each patch token in the h-th attention head. This represents the SDPA output of the h-th attention head.

[0085] (3) Visual attention concentration score To explicitly perceive abnormal visual attention clustering, this embodiment is based on an attention weight matrix. Computational visual attention concentration score This score is used to characterize the degree to which a patch receives excessive attention from other patches, the degree of attention focused on a certain type of background patch, or the degree to which a local region abnormally dominates global features.

[0086] In this embodiment, the first... The average intensity of attention a patch receives from other patches: ; in, Indicates the first The attention weights of each patch to the j-th patch. If Higher indicates the first A patch is the focus of attention in this attention head. This patch may be the target area, or it may be an abnormally concentrated area caused by background, reflection, texture or noise.

[0087] Furthermore, visual attention aggregation scores can be constructed by combining image region priors, saliency estimation, background estimation, patch location, or cross-head statistics. For example, scores can be generated for multiple attention heads. Calculate the average, maximum, or weighted sum to obtain a uniform cluster score. It is also possible to retain the independence of each attention point. Used for head-specific gating.

[0088] (4) Learnable univariate function gating module The gating module in this embodiment is not a typical linear layer, but rather consists of learnable unary functions. The inputs to the gating module include visual token representations, visual attention concentration scores, and two-dimensional patch location information. ; in, For normalized visual tokens, Scoring based on visual attention concentration The location features are two-dimensional. The gating module can be implemented based on spline functions, piecewise linear functions, polynomial functions, or other trainable univariate functions.

[0089] For the The first attention head and the first For each channel, the gated logits can be represented as: ; in, For learnable univariate functions, The first input for gating Each component. The gate value is obtained through the Sigmoid mapping: ; The overall gating matrix is: ; The gating matrix can be refined to each sample, each patch, each attention head, and each channel dimension.

[0090] (5) SDPA back-gating for each attention head The key gating position in this embodiment is: after each attention head completes SDPA aggregation, but before multiple attention heads are concatenated. For the first... Attention point, execute: ; in, This represents element-wise multiplication. When... When an element is close to 1, the corresponding attention output is basically preserved; when the element is close to 0, the corresponding attention output is suppressed; when the element is between 0 and 1, the corresponding attention output is partially preserved.

[0091] This position is different from the one mentioned above. Gating in this embodiment differs from uniform gating after concatenating multiple attention heads or after output projection. This embodiment first allows each attention head to complete its own attention aggregation, and then performs gating based on the contextual information related to the aggregation result. This preserves the functional differences between different attention heads and independently modulates the propagation of abnormal visual information within each attention head.

[0092] (6) Output stitching and image feature enhancement Concatenate all the attention head outputs after gating: ; The visual attention layer output is then obtained through output projection: ; The output can be fed into subsequent Transformer blocks, classification heads, detection heads, segmentation heads, or multimodal fusion modules for tasks such as image classification, object detection, image segmentation, industrial defect detection, medical image recognition, remote sensing object recognition, autonomous driving scene perception, or visual question answering.

[0093] During the training phase, sparse constraints can be added to the gating matrix or the learnable univariate function to further suppress the propagation of information corresponding to background, noise, or anomaly clustering patches. For example, regularization terms can be applied to the average activation of the gating value, the gating entropy, the function edge contribution, or long-term low activation channels.

[0094] After training, the contribution of each learnable univariate function can be statistically analyzed, and function edges that are consistently close to zero, have small fluctuations, or have a weak impact on the output can be pruned. Important univariate function curves can also be visualized to explain why certain visual features, location regions, or attentional focus scores trigger inhibition or enhancement.

[0095] Example 2 To verify the effectiveness of the proposed image feature enhancement method based on learnable univariate function gating in general image recognition tasks, this embodiment conducted a comparative experiment on the CIFAR-100 image classification dataset.

[0096] The CIFAR-100 dataset contains 100 image categories, with 50,000 images in the training set and 10,000 images in the test set. The images are 32×32 pixels in size. This dataset can be used to validate the feature representation capabilities of image classification models in multi-class visual recognition tasks.

[0097] This embodiment selects the ViT-small model with pre-trained weights as the base network. In the experiment, only the attention modules in the last two Transformer blocks of the ViT-small model were replaced, replacing the original attention modules with gated attention modules that include visual attention aggregation scoring, two-dimensional position encoding, and learnable unary function gating; the model's patch embedding, preceding Transformer blocks, MLP sublayers, normalization layers, and classification heads remained unchanged. Since the base model already has a high image recognition capability, this embodiment is mainly used to verify whether introducing the local gated attention modules of this invention on a strong base model can still bring further performance gains.

[0098] This embodiment uses Top-1 accuracy as the primary evaluation metric and Top-5 accuracy as a secondary evaluation metric. Three comparison methods are set up in the experiment: The first is the baseline method, which is the original ViT-small model without replacing the attention module; The second method is the ordinary linear gating method, which adds an ordinary linear sigmoid gating after the attention output of the last two Transformer blocks; The third method is the method of this invention, which introduces visual attention aggregation scoring, two-dimensional position encoding and learnable unary function gating into the attention modules of the last two Transformer blocks to modulate the attention output element by element.

[0099] In the method of this invention, for each attention head in the replaced Transformer block, after completing the scaling dot product attention calculation and before concatenating multiple attention heads, the output of the attention head is gated and modulated. The calculation can be expressed as follows: ; in, Indicates the first The original output of each attention head, Indicates the first The gating matrix corresponding to each attention head. This indicates element-wise multiplication. This represents the modulated attention output.

[0100] Gated matrix The input for generation includes visual token representations. Visual attention concentration score calculated from the attention matrix and two-dimensional patch position encoding The three components, when combined and used as input to the gating module, can be represented as: ; In this method, the visual token representation H provides local image semantic information, the visual attention aggregation score S represents the degree of aggregation of different visual tokens in attention interaction, and the two-dimensional patch position encoding P provides image spatial location information. Through the above combined inputs, the method of this invention can adaptively modulate the attention output based on image semantics, attention distribution, and spatial location.

[0101] This embodiment uses the AdamW optimizer for training and records the best Top-1 accuracy and corresponding Top-5 accuracy on the test set during training. The experimental results are shown in Table 1 below.

[0102] Table 1. Accuracy Experiment Results

[0103] The experimental results show that, on the ViT-small base model with pre-trained weights, the Top-1 accuracy of the baseline method is 90.90%, the Top-1 accuracy of the ordinary linear gating method is 91.33%, and the Top-1 accuracy of the method of this invention is 91.43%.

[0104] Compared to the baseline method, the Top-1 accuracy of the method of this invention is improved by 0.53 percentage points; compared to the ordinary linear gating method, the Top-1 accuracy of the method of this invention is improved by 0.10 percentage points. These results demonstrate that even when the base model already has a high recognition accuracy, the method of this invention can still achieve further positive gains by locally replacing the rear attention module.

[0105] From the perspective of the auxiliary metric Top-5 accuracy, the baseline method achieved a Top-5 accuracy of 98.71%, the ordinary linear gating method achieved a Top-5 accuracy of 98.88%, and the method of this invention achieved a Top-5 accuracy of 98.94%. Compared with the baseline method, the Top-5 accuracy of the method of this invention improved by 0.23 percentage points; compared with the ordinary linear gating method, the Top-5 accuracy of the method of this invention improved by 0.06 percentage points. These results further demonstrate that the method of this invention not only improves the accuracy of the model's preferred category identification but also enhances the overall identification capability within the candidate category range.

[0106] Furthermore, to verify the role of visual token representation, visual attention aggregation score, and 2D position encoding as gating inputs, this embodiment conducted a gating input ablation experiment. The experiment maintained consistency with the base model, dataset, and training strategy, only changing the input information of the learnable unary function gating module. The effects of using only visual token representation H, using H+P, using H+S, and using H+S+P simultaneously were tested. The experimental results are shown in Table 2 below.

[0107] Table 2 Accuracy Experiment Results

[0108] The ablation experiment results show that when only the visual token representation H is used as the gating input, the model's Top-1 accuracy is 91.17%; after adding the two-dimensional position code P, the Top-1 accuracy increases to 91.31%; after adding the visual attention concentration score S, the Top-1 accuracy increases to 91.37%; when H+S+P are used as gating inputs simultaneously, the model achieves the highest Top-1 accuracy, reaching 91.43%.

[0109] Compared to using only H, employing H+S+P simultaneously improved Top-1 accuracy by 0.26 percentage points; compared to H+P, it improved Top-1 accuracy by 0.12 percentage points; and compared to H+S, it improved Top-1 accuracy by 0.06 percentage points. These results indicate that visual token representation, visual attention concentration score, and two-dimensional positional encoding are complementary.

[0110] Among them, the visual token representation H provides semantic features of the image, the visual attention aggregation score S provides information on the degree of aggregation in attention interaction, and the two-dimensional positional encoding P provides spatial structure information. Using all three as inputs to a learnable unary function gating module enables the gating module to simultaneously perceive image content, attention distribution, and spatial location, thereby achieving more refined attention output modulation.

[0111] It should be noted that since this embodiment uses the ViT-small model with pre-trained weights, the baseline model already has a high classification accuracy, thus limiting the potential for further improvement. Under these conditions, the method of this invention still achieves positive gains in the Top-1 and Top-5 metrics, indicating that this method can serve as a local attention enhancement module, improving image feature representation capabilities by replacing the attention modules of several subsequent Transformer blocks.

[0112] In summary, the experimental results of this embodiment demonstrate that introducing a gated attention module based on visual attention aggregation scoring, 2D positional encoding, and a learnable unary function into the last two Transformer blocks of the ViT-small model can improve Top-1 accuracy in the CIFAR-100 image classification task. Compared to ungated baselines and ordinary linear gating methods, the method of this invention achieves superior classification results. Gated input ablation experiments further show that when visual token representation, visual attention aggregation scoring, and 2D positional encoding are all involved in gating generation, even better classification performance can be obtained.

[0113] Those skilled in the art will recognize that the modules and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0114] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and equipment can be referred to the corresponding process in the foregoing method implementation, and will not be repeated here.

[0115] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0116] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs.

[0117] In addition, the functional modules in the embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0118] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to conventional solutions, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the sending / receiving methods of various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0119] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

[0120] It should be understood that the sequence number of each step in the invention and its embodiments does not absolutely imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Claims

1. An image feature enhancement method based on learnable univariate function gating, characterized in that, include: The input image is divided into multiple image blocks, and each image block is mapped to a visual token to obtain a visual token sequence. The two-dimensional positional encoding of the image blocks is embedded in the visual token sequence. The visual token sequence is normalized to obtain a normalized visual token; For each attention head in the multi-head attention layer, the query matrix, key matrix, and value matrix corresponding to the attention head are calculated based on the normalized visual token, and the attention weight matrix and SDPA output of the attention head are obtained by scaling dot product attention calculation. The visual attention clustering score is calculated based on the attention weight matrix. The visual attention clustering score is used to quantify the degree of clustering of the corresponding image patch with attention from other image patches, so as to identify whether the image patch belongs to a target region that is abnormally focused, a background region that generates redundant propagation, or a local region that over-dominates global features. The normalized visual token, the visual attention aggregation score, and the two-dimensional position code are input into the gating module. The gating module is composed of learnable unary functions. Each channel of each attention head corresponds to a set of learnable unary functions. Each set of learnable unary functions contains the same number of learnable unary functions as the gating input components. Each learnable unary function is composed of at least one function edge, which is used to generate the gating matrix. After each attention head completes its SDPA output and before multiple attention heads are concatenated, the SDPA output of the attention head is modulated element-wise using the gating matrix to obtain the gated attention head output. The outputs of multiple gated attention heads are stitched together and then projected onto the output to obtain enhanced image features.

2. The image feature enhancement method based on learnable univariate function gating according to claim 1, characterized in that, The visual attention concentration score is calculated based on the attention weight matrix, including one or a combination of the following methods: For the h-th attention head, the average intensity of attention given to the j-th image patch by other image patches is calculated as the visual attention aggregation score of that image patch; A unified clustering score is obtained by averaging, taking the maximum value, or weighted summing the visual attention clustering scores of multiple attention heads. The visual attention concentration score is constructed by combining image region priors, saliency estimation, or background estimation.

3. The image feature enhancement method based on learnable univariate function gating according to claim 1, characterized in that, The learnable univariate function includes spline functions, piecewise linear functions, or polynomial functions; For the h-th attention head and the c-th channel, the gated logits are obtained by summing the components of the gated input after transforming each learnable unary function corresponding to the channel, and then obtaining the gated value through Sigmoid mapping, thus forming the gated matrix.

4. The image feature enhancement method based on learnable univariate function gating according to claim 1, characterized in that, The SDPA output of the attention head is modulated element-wise using the gating matrix, including: The gating matrix is ​​multiplied element-wise with the SDPA output of the corresponding attention head. When the gating value approaches 1, the corresponding attention output is retained. When the gating value approaches 0, the corresponding attention output is suppressed. When the gating value is between 0 and 1, the corresponding attention output is partially retained.

5. The image feature enhancement method based on learnable univariate function gating according to claim 1, characterized in that, During the training phase, a sparse constraint regularization term is applied to the gating matrix or the learnable univariate function. The sparse constraint regularization term includes at least one of the following: gating value average activation regularization term, gating entropy regularization term, function edge contribution regularization term, or channel activation sparse regularization term.

6. The image feature enhancement method based on learnable univariate function gating according to any one of claims 1-5, characterized in that, After training is completed, based on the activation statistics of the gating matrix, the contribution of each learnable univariate function edge in the gating module to the gating output is calculated, and the function edges that meet the preset pruning conditions are pruned. The preset pruning conditions are determined based on at least one of the following indicators: the long-term activation mean of the function edge is lower than a first threshold, the activation variance of the function edge is lower than a second threshold, or the contribution of the function edge output to the gated logits is lower than a third threshold.

7. An image feature enhancement system based on learnable univariate function gating, characterized in that, include: The image block embedding module divides the input image into multiple image blocks, maps each image block to a visual token to obtain a visual token sequence, and embeds the two-dimensional positional encoding of the image blocks into the visual token sequence; The normalization module normalizes the visual token sequence to obtain a normalized visual token. The multi-head attention calculation module calculates the query matrix, key matrix, and value matrix corresponding to each attention head in the multi-head attention layer based on the normalized visual token, and obtains the attention weight matrix and SDPA output of the attention head through scaled dot product attention calculation. The aggregation score calculation module calculates a visual attention aggregation score based on the attention weight matrix. The visual attention aggregation score is used to quantify the degree of aggregation of the corresponding image patch with attention from other image patches, so as to identify whether the image patch belongs to a target region with abnormal attention, a background region that generates redundant propagation, or a local region that over-dominates global features. The gating module receives the normalized visual token, the visual attention aggregation score, and the two-dimensional position code. The gating module is composed of learnable unary functions. Each channel of each attention head corresponds to a set of learnable unary functions. Each set of learnable unary functions contains the same number of learnable unary functions as the gating input components. Each learnable unary function is composed of at least one function edge, which is used to generate a gating matrix. The gated modulation module uses the gate matrix to perform element-wise gated modulation on the SDPA output of each attention head after it completes its SDPA output and before multiple attention heads are spliced ​​together, so as to obtain the gated attention head output. The output module stitches together the outputs of multiple gated attention heads and then projects the outputs to obtain enhanced image features.

8. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the image feature enhancement method based on learnable univariate function gating as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the image feature enhancement method based on learnable unary function gating as described in any one of claims 1-6.

Images