Salient map guided and entropy-regularized optimization method and system for small object detection

Abstract

The application provides a small target detection method and system based on saliency map guidance and entropy regularization optimization, and belongs to the field of small target detection. In order to solve the problem that the existing target detection method has insufficient feature expression ability when detecting small targets, is easily disturbed by noise and background, and leads to low detection stability and accuracy, the application combines saliency prior and Transformer to propose a Trans-Sal Det framework. The framework applies a Tiny-U-Net model in the saliency map generation stage to extract the fusion of low-level texture features and high-level semantic features, thereby enhancing the detection ability of the model for small targets in complex scenes. By introducing a cross-scale fusion layer and a dynamic saliency injection mechanism of a Query-Based Decoder, the target recall rate of Trans-Sal Det in the presence of occlusion and complex background is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of small target detection technology, and more specifically, to a small target detection method and system guided by saliency graph and optimized by entropy regularization. Background Technology

[0002] In recent years, object detection technology has made significant progress driven by deep learning, especially in the identification and localization of large and medium-sized targets. Classic frameworks such as the YOLO series, Faster R-CNN, and RetinaNet have achieved relatively mature performance. However, in fine-grained analysis scenarios such as remote sensing images, traffic monitoring, medical imaging, and industrial defect detection, small target detection still faces many challenges. These targets often have typical characteristics such as small area, blurred morphological features, sparse semantic information, and strong background interference, causing mainstream detection methods to exhibit insufficient accuracy, low recall, and weak generalization ability in this task. Therefore, the key question of "how to achieve accurate and robust detection of extremely small targets in complex backgrounds" has gradually become one of the current research focuses.

[0003] Current mainstream object detection frameworks, such as the YOLO series and RetinaNet, mostly rely on deep convolutional networks to extract semantic features from images. While these methods demonstrate excellent performance in detecting medium to large objects, they still face significant performance bottlenecks in detecting small objects. On one hand, due to the fixed receptive field and layer-by-layer downsampling mechanism of convolutional neural networks, small objects gradually disappear in high-level semantic feature maps, suppressing their feature representation capabilities. On the other hand, these models typically do not provide explicit guidance on salient regions, making the network susceptible to noise and background interference in the early stages of training, thus affecting detection stability and accuracy. Summary of the Invention

[0004] The technical problem to be solved by this invention is:

[0005] To address the problems of insufficient feature representation ability and susceptibility to noise and background interference in existing target detection methods when detecting small targets, resulting in low detection stability and accuracy.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] This invention provides a small target detection method guided by saliency graph and optimized by entropy regularization, comprising the following steps:

[0008] S100. In the small object detection model Trans-Saline Det, a high-resolution saliency map is generated through the lightweight operation mechanism Tiny U-Net; including,

[0009] S110. Extract attention weights, construct an attention score matrix, and define the original integral using a joint modeling approach of temperature regulation and local smoothing softmax. Then, use... Normalize the attention weights, and finally obtain the attention matrix after aggregating the attention map;

[0010] S120. Calculate the information entropy of the normalized attention probability distribution, including defining the standard information entropy of each row of probability vectors, reconstructing the information entropy after introducing the softmax temperature tensor, introducing a fusion method based on the significance weight of query tokens, weighted averaging the entropy values ​​of different queries to obtain the final regularization term, and constructing the total loss function together with the regularization term after optimizing the classification loss and boundary loss.

[0011] S130, the Transformer encoding module in the small object detection model Trans-Sal Det includes three resolution levels. For each resolution level, an independent attention weight matrix and corresponding information entropy term are constructed and added to the total loss function.

[0012] S200 introduces a pyramid-shaped Transformer structure to construct a bottom-up global feature fusion path, and enhances the focusing ability of each object query on the target region through a dynamic injection mechanism of saliency weights; including,

[0013] S210. Introduce a saliency-guided path to make the query-key interaction process regionally selective. In the decoding stage, each query is collaboratively driven by the saliency graph to drive attention interaction. That is, a saliency mask is injected into the cross-attention calculation to pre-modulate and adjust the bias of the response at the corresponding key-value position.

[0014] S220 introduces a saliency-guided bounding box geometric decoding mechanism, which applies saliency weights to the bounding box representation through a residual modulation structure, and designs a position embedding and scale-aware encoding method to improve the robustness of small target boundary localization and recall accuracy.

[0015] Further, in step S110, the following is included:

[0016] Let a token sequence of a certain scale be represented as ,in This indicates the number of tokens, which is the number of positions in the flattened feature map of this layer. Let H be the channel dimension, H be the height of the feature map, and W be the width of the feature map. In the h-th attention head, the calculation process for query(Q), key(K), and value(V) is as follows:

[0017]

[0018] in, It is a linear projection matrix. Dimensions for each attention head;

[0019] Attention score matrix The Line 1 Column elements represent the first The query for the first... The response strength of each token; to construct a more stable and regularizable attention graph, a joint modeling approach of temperature regulation and local smoothing softmax is adopted; the original score is defined. for:

[0020]

[0021] in, Let be the query volume of the i-th element under the h-th attention head; Let j be the key vector of the j-th element under the h-th attention head; This is the softening temperature coefficient; For Kronecker delta term, A constant offset term to suppress the effects of self-attention;

[0022] The final normalized attention weights are achieved using a smooth softmax variant. Introducing a power-normalization strategy to control curvature:

[0023]

[0024] Where k is the kth token; Let be the attention weight of the i-th element to the j-th element under the h-th attention head; This is the power exponent control parameter;

[0025] For each set of multi-head attention score matrices Preserving the spatial dimension, and averaging or max-pooling the score matrices of all attention heads, we obtain the aggregated attention map:

[0026]

[0027] in, This is the attention matrix after aggregating the attention map; Aggregate is the mean or maximum value learning weighted pooling; each row in this matrix... Considered the first Attention probability distribution of tokens for each query.

[0028] Further, in step S120, the following is included:

[0029] For each row of probability vector Define its standard information entropy for:

[0030]

[0031] in, A smoothing constant to prevent numerical overflow;

[0032] To improve the convergence speed, a softmax temperature tensor is introduced. And reconstruct the entropy expression as :

[0033]

[0034] in, Let i be the i-th softmax temperature tensor; Let be the entropy of the i-th element after reconstruction;

[0035] To improve the targeting of regularization terms to the target region, a fusion method based on query token saliency weights is introduced, which performs a weighted average of the entropy values ​​of different queries; let Represents the query weight, the final regular expression term. Represented as:

[0036]

[0037] in, This is the global entropy regulation coefficient; The weight of the i-th query;

[0038] Will Added to the total loss as a regularization term In the middle, with classification loss With bounding box loss Parallel optimization:

[0039]

[0040] in, and These are the weighting coefficients.

[0041] Further, in step S130, the following is included:

[0042] Let the normalized attention weights of each layer be... The corresponding information entropy regularization term is Then the overall entropy is regular and visible. aggregation;

[0043] Three optional fusion methods are defined: this method adds the entropy terms of the three scales with equal weights to construct a unified regularized path. The expression is as follows:

[0044]

[0045] Suitable for architectures with spatially balanced three-layer feature maps and approximate token density;

[0046] Considering that the saliency graph space structure and token fineness focus on the highest resolution layer, sparse regularization is applied only to the first layer:

[0047]

[0048] To achieve cross-scale adaptive control of information entropy constraints, learnable gating coefficients are introduced into learnable gating coefficients. ,satisfy Constructing fusion expression for:

[0049]

[0050] in, It is a trainable scalar vector; For the input vector conduct operate;

[0051] When incorporating the information entropy regularization term into the total loss function, let the independent coefficients be... Controlling its optimization intensity, the overall training objective is expressed as:

[0052]

[0053] in, This represents any one of the three strategies mentioned above. Including classification loss and bounding box loss, This represents the auxiliary loss guided by significance.

[0054] Further, in step S210, the following is included:

[0055] Let the input for the decoding stage be... A matrix composed of object queries The corresponding multi-scale output token of the encoder is represented as follows: Meanwhile, let the saliency map corresponding to the input image be... It was reshaped into a one-dimensional vector. And the attention bias term is obtained through linear mapping. :

[0056]

[0057] in, For a linear layer with significant weights, This represents the Sigmoid activation function;

[0058] In the cross-attention module, attention scoring is based on the standard attention scoring formula. for:

[0059]

[0060] in, Indicates the original degree of correlation; This represents the scaling factor associated with the j-th element;

[0061] The j-th significance bias Attention logits in the key direction are added to amplify the attractiveness of tokens in salient regions; finally, the attention matrix is ​​obtained by Softmax normalization. Used to generate the output for each query. :

[0062]

[0063] in, Perform the operation on the i-th row of matrix A The attention weights obtained after normalization;

[0064] A query gating mechanism is introduced to directly encode saliency priors in the query embedding; a gating vector is constructed by combining the input embedding of each query with the global average response of the saliency map. :

[0065]

[0066] in, This indicates a splicing operation. This indicates a feature concatenation operation, which combines a specific query vector... With global statistical vector By concatenating features along the feature dimension, new joint features are formed; This represents a multilayer perceptron; For the Sigmoid function;

[0067] The final query was modulated as follows:

[0068]

[0069] Further, in step S220, the following are included:

[0070] Suppose that the output of a certain object query is The initial bounding box regression representation is obtained through two layers of MLP. Including the coordinates of the center point ( ) and width ,high predict:

[0071]

[0072] in, It is a regression-type multilayer perceptron; The feature vector of the i-th target input;

[0073] To enhance the spatial selectivity of the prediction, a residual significance control module is introduced: the average response is extracted by matching the significance patch of the corresponding query. Define the adjustment factor:

[0074]

[0075] Where tanh is the hyperbolic tangent function. This represents the average significance of the top-k response positions in query-key attention. , To adjust the mapping parameters, this adjustment term is ultimately incorporated into the regression output as a residual weighting term. :

[0076]

[0077] in, For residual intensity factor, This represents element-wise multiplication;

[0078] To accommodate geometric scale awareness of different target sizes, the regression channel incorporates the input embedding of each query and the global average response of the saliency map to construct a gating vector. :

[0079]

[0080] in, Estimate the coordinates of the spatial center and receptive field pointed to by the query; ultimately... With query expression After splicing, the data is fed into the regressor:

[0081]

[0082] To enhance the ability to perceive edge structures, a significant gradient term at the boundary is introduced into the regression prediction. The Sobel gradient representation is constructed using the Gaussian fuzzy saliency map. Extract the directional gradient at the corresponding position of the query:

[0083]

[0084] This significant boundary direction is used to refine the correction of the center offset, and the final predicted center coordinates are:

[0085]

[0086] Where η is the gradient-guided weight constant.

[0087] Furthermore, the small target detection method is used for remote sensing detection of oil well targets.

[0088] A small target detection system guided by saliency graph and optimized by entropy regularization is provided. The system has program modules corresponding to the steps described above, and executes the steps in the small target detection method guided by saliency graph and optimized by entropy regularization during runtime.

[0089] A computer-readable storage medium storing a computer program configured to, when invoked by a processor, implement steps of a saliency graph-guided and entropy regularization-optimized small target detection method.

[0090] Compared with the prior art, the beneficial effects of the present invention are:

[0091] This invention proposes a small target detection model, Trans-Sal Det, which aims to streamline the process of "saliency modeling – global dependency – feature fusion" to improve the perception granularity and localization accuracy of the detection model for small targets. The Trans-Sal Det model employs an end-to-end architecture. First, Tiny U-Net generates a high-resolution saliency map to accurately capture potential target regions as prior information. This saliency map, through shallow detail extraction and deep semantic fusion, balances local edge accuracy with regional semantic consistency and exhibits good boundary preservation capabilities in spatial structure. The rationale behind this design is that the prior information generated by the self-supervised saliency mechanism can effectively alleviate the problem of insufficient representation ability of traditional backbone networks in small target regions, providing reliable spatial guidance for subsequent feature fusion and attention allocation.

[0092] In the core feature extraction section, a pyramid-shaped Transformer structure is introduced to construct a bottom-up global feature fusion path. A dynamic injection mechanism of saliency weights enhances the focusing ability of each object query on the target region. This structure meticulously considers the design of the number of Transformer layers, the selection of the window mechanism, and the inter-layer scale progression method. Unlike the full-size attention used in the Transformer detector structure, Trans-Sal Det employs a local window constraint mechanism in each level of attention modeling and maintains the semantic coherence of features between layers through cross-scale mapping. The Transformer decoder module, as a key component for target localization and classification, adopts an object query-driven interactive decoding structure. In Trans-Sal Det, object queries are initialized as a set of learnable vectors, and the detection results are output after several rounds of interaction.

[0093] To effectively suppress redundant background interference and improve attention focus, the model introduces an information entropy minimization regularization strategy. During training, this strategy adaptively optimizes the attention distribution, making it more focused on the real target region, thereby improving detection performance. The core idea of ​​this regularization term is to encourage a sparser and more concentrated distribution of attention weights in the Transformer, avoiding the problem of indiscriminate attention diffusion.

[0094] In summary, the network structure proposed by Trans-Sal Det in this invention not only effectively improves visual representation capabilities but also achieves structural closure through saliency map priors and attention-guided mechanisms. The model embodies a saliency-driven design philosophy at each stage, maintaining explicit enhancement of the target region and effective suppression of background interference from the initial feature extraction to the final target regression. Compared to existing convolutional detectors, it has stronger context modeling capabilities. Compared to existing full Transformer detectors, it avoids the problems of lengthy structures and difficult convergence. This structure balances performance and efficiency and lays a structural foundation for future expansion into multi-task detection, multi-modal fusion, and other directions. Attached Figure Description

[0095] Figure 1 This is a framework diagram of the small target detection method guided by saliency graph and optimized by entropy regularization in an embodiment of the present invention;

[0096] Figure 2 This is a comparison diagram of target detection models in embodiments of the present invention;

[0097] Figure 3 This is a training and validation loss curve diagram in an embodiment of the present invention;

[0098] Figure 4 This is a training and verification F1 score curve diagram in an embodiment of the present invention. Detailed Implementation

[0099] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0100] Specific Implementation Plan 1: Combining Figure 1 As shown, this invention provides a small target detection method guided by saliency graph and optimized by entropy regularization for remote sensing detection of oil well targets, comprising the following steps:

[0101] S100. Construct a high-resolution, structurally continuous, and semantically complete saliency map using Tiny-U-Net;

[0102] In the small target detection model Trans-Sal Det, the saliency map provides spatial guidance for the multi-scale Transformer encoding module, which can effectively improve the initial token expression strength of small target regions. However, in the internal mechanism of multi-head self-attention, especially after stacking deep structures, the attention distribution may still suffer from problems such as saliency signal diffusion and target focusing degradation. Especially when small targets are sparse or the background is complex, the model is prone to divergence of the attention region in the unconstrained attention flow, resulting in blurred target localization boundaries or even failure of attention weight averaging.

[0103] To further suppress this phenomenon, a lightweight operation mechanism, Tiny-U-Net, is introduced, which performs information entropy regularization on the attention weight distribution in each layer's attention graph. This operation does not modify the model structure but instead imposes entropy constraints on the query-key attention matrix, making the attention weights more sparse and prompting the model to focus its attention on a limited number of salient regions. This sparsity operation is accomplished by adding an additional regularization loss during the training phase, and the gradient can be directly applied to the Q-K (query-key) computation path inside the Transformer, achieving adaptive attention convergence control.

[0104] Specifically, it includes:

[0105] S110, Attention Weight Extraction and Normalization

[0106] To impose distributional constraints on attention behavior in the multi-scale Transformer module, we first define how to extract attention weights from each layer of the Transformer encoder and construct a regularizable explicit structure in the spatial dimension. This weight matrix will become the direct object of the information entropy regularization term, and its construction process must preserve the spatial correspondence between tokens and be compatible with the multi-head attention calculation mechanism.

[0107] Let a token sequence of a certain scale be represented as ,in , which represents the number of tokens (i.e., the number of positions after the feature map of this layer is flattened). Let H be the channel dimension, H be the height of the feature map, and W be the width of the feature map; in the h-th attention head, The calculation process is as follows:

[0108]

[0109] in, It is a linear projection matrix. Dimensions for each attention head;

[0110] Attention score matrix The Line 1 Column elements represent the first The query for the first... The response strength of each token; to construct a more stable and regularizable attention graph, a joint modeling approach of temperature regulation and local smoothing softmax is adopted; the original score is defined. for:

[0111]

[0112] in, Let be the query volume of the i-th element under the h-th attention head; Let j be the key vector of the j-th element under the h-th attention head; The softening temperature coefficient (e.g., 0.07). For Kronecker delta term, It is a constant offset term that suppresses the effects of self-attention (self-loop), and is usually set to a negative value to suppress self-attraction;

[0113] The final normalized attention weights are achieved using a smooth softmax variant. Introducing a power-normalization strategy to control curvature, written as:

[0114]

[0115] Where k represents the k-th token; This represents the attention weight of the i-th element to the j-th element under the h-th attention head; For the power exponent control parameter, when This can suppress excessively steep extreme value distributions; It can enhance the concentration of attention weight; and This is the default setting of "Do not adjust sharpness," which can be set during the early stages of training. Later, it was smoothly adjusted to 1.2, forming an attention weight aggregation control path;

[0116] For each set of multi-head attention score matrices Preserving the spatial dimension, and averaging or max-pooling the score matrices of all attention heads, we obtain the aggregated attention map:

[0117]

[0118] in,

[0119] This is the attention matrix after aggregating the attention map; Aggregate can be the mean, maximum, or a custom learnable weighted pooling; each row in this matrix... Considered the first The attention probability distribution of the tokens for each query will become the target of subsequent entropy regularization.

[0120] This attention weight construction method preserves the complete structure of the spatial token and also has the ability to fuse cross-head information, providing an accurate, controllable and differentiable numerical path for the subsequent definition of token-level information entropy;

[0121] S120, Information Entropy Calculation Path Design

[0122] After completing the attention matrix for each Transformer layer After normalization, we will normalize each row of the matrix. Considered the first The attention probability distribution of tokens for a query; this distribution characterizes the response strength of the query to all tokens. The goal of the information entropy regularization term is to minimize the entropy value of these distributions in order to guide attention to a small number of high-response tokens and reduce attention divergence.

[0123] For each row of probability vector Define its standard information entropy for:

[0124]

[0125] in, The smoothing constant, used to prevent numerical overflow, is typically set to 10. -6 Note that this expression measures the attention distribution of the token itself and does not involve mutual information or contextual relevance.

[0126] Considering that the attention distribution tends to be uniform in the early stages of training, we introduce a softmax temperature tensor to improve the convergence speed in the early stage. (Optional: learnable parameters or stage scheduling function), and the entropy expression is reconstructed as :

[0127]

[0128] in, Let i be the i-th softmax temperature tensor; It is the entropy of the i-th element after reconstruction;

[0129] This entropy expression is differentiable. It degenerates into the standard entropy function at time. Strengthening the weight of the head token effectively forms a two-stage mechanism of early-stage guidance and late-stage stable distribution focus; when Time: To do the scoring The exponentiation (equivalent to "shrinking" the score difference) makes the distribution smoother, the weights are distributed more evenly among multiple elements, and the attention is more dispersed;

[0130] To enhance the specificity of the regularization term in constraining the target region, we further introduce a fusion method based on the saliency weight of query tokens, performing a weighted average of the entropy values ​​of different queries; let The query weights (which can be derived from the saliency plot mean, token confidence, or learnable gate) are the final regularization term. Represented as:

[0131]

[0132] in, This is the global entropy regulation coefficient; Let be the weight of the i-th query. Through this weighting strategy, the model can automatically focus on the attention sparsity of significant queries during training, while weakening the regularization of queries in non-target regions, thus avoiding excessive shrinkage of background attention.

[0133] This regularization term maintains a sparse distribution within the token space; backpropagation is possible for all elements of the attention weight matrix; ultimately... It will be added to the total loss as a regularization term. In the middle, with classification loss With bounding box loss Parallel optimization, writing:

[0134]

[0135] in, The loss weight hyperparameter for the entropy regularization term typically ranges from 0.05 to 0.2. and These are weighting coefficients (hyperparameters).

[0136] S130, Fusion and Regularization Constraints of Multi-Scale Attention Entropy

[0137] The Transformer encoding module in the small object detection model Trans-Sal Det contains three resolution levels, each constructing an independent attention weight matrix and corresponding information entropy term. To uniformly model the attention sparsity constraints of the entire network, a reasonable multi-scale entropy fusion method needs to be designed, allowing attention structures at different levels to participate in optimization while preserving inter-scale differences and avoiding over-compression of low-level attention or failure of high-level guidance. Let the normalized attention weight of each layer be... The corresponding information entropy regularization term is Then the overall entropy is regular and visible. aggregation;

[0138] We define three optional fusion methods: this method adds the entropy terms of the three scales with equal weights to construct a unified regularized path. The expression is as follows:

[0139]

[0140] It is suitable for architectures with spatially balanced three-layer feature maps and similar token density, and has the advantages of simple operation and stable training.

[0141] Considering that the saliency graph space structure and token fineness mainly focus on the highest resolution layer (1 / 4), sparse regularization can be performed only on the first layer, while the other two layers do not participate in this calculation, written as:

[0142]

[0143] in, This can be understood as "focused entropy," and the corresponding loss is the focused entropy loss. This represents the first entropy loss. This scheme has a relatively fast convergence speed and is suitable for lightweight scenarios with a high proportion of small targets and limited computing resources. It can effectively tighten the low-level attention diffusion problem.

[0144] Learnable Gated Fusion (LGF) introduces learnable gating coefficients to achieve adaptive control of information entropy constraints across scales. ,satisfy Constructing fusion expression for:

[0145]

[0146] in, Given trainable scalar vectors, Softmax guarantees the interpretability of the scale. For the input vector conduct The operation transforms it into a probability distribution. SoftMax operation ensures , making To achieve a reasonable weight distribution, the weighted sum of different entropy losses should have a clear proportional meaning;

[0147] This mechanism can learn the sensitivity of different scales to sparsity control, and significantly improves stability when the token statistics of multiple scales differ greatly.

[0148] When incorporating the information entropy regularization term into the total loss function, independent coefficients can be set. Controlling its optimization intensity, the overall training objective is expressed as:

[0149]

[0150] in, This represents any one of the three strategies mentioned above. Including classification loss and bounding box loss, This represents the auxiliary loss guided by significance; it is generally set as follows: The progressive weighting strategy is superior; this multi-scale fusion regularization mechanism can coordinate the attention-focusing behavior at different spatial scales and avoid the information imbalance that may be caused by single-layer regularization, further improving the region discrimination ability and generalization stability of the Transformer encoder.

[0151] S200, Design of Target Decoding and Boundary Regression Mechanism Guided by Saliency

[0152] The Trans-Sal Det small target detection model effectively improves the attention focus and spatial recognition ability of small target regions during the encoding stage by introducing a high-resolution saliency map and an information entropy regularization mechanism. However, for an end-to-end detection framework, the accurate identification and spatial localization of targets still ultimately depend on the object query representation and bounding box regression mechanism in the decoding stage. Therefore, simply applying spatial focus constraints within the Transformer encoder is insufficient to fully address the problems of weak saliency, blurred boundaries, and target position offset of small targets. To address this, a multi-scale target decoding and regression structure design strategy based on saliency perception is proposed, further extending the guidance method of saliency information in the decoding path. This mechanism introduces a saliency guiding term in the query-key cross attention interaction, couples saliency feature responses in the bounding box regression channel, and introduces saliency posterior correction in the final target selection, achieving explicit control and progressive fusion of the saliency map over the entire detection process. Compared to the traditional anchor-free decoding method that only relies on the interaction between query and encoder features, the object in this structure... The query decoder has cross-scale fusion and saliency re-guidance capabilities, making it particularly suitable for semantic enhancement and geometric regression refinement of small targets in complex backgrounds.

[0153] By constructing a three-element linkage path of "saliency-query-location", the recall stability and boundary localization accuracy of the small object detection model Trans-Sal Det are significantly improved in high-density scenes. The following sections will elaborate on the saliency interaction modeling of object queries, the multi-scale encoding fusion method for saliency assistance, and the geometric control mechanism for fusing saliency maps in the bounding box regression path, further clarifying the core role of this saliency decoding path in the small object detection model Trans-Sal Det.

[0154] S210, object query saliency fusion mechanism,

[0155] To enhance the object query's ability to perceive spatially salient targets, a saliency-guided path is introduced, making the query-key interaction process regionally selective. In the decoding stage, each query will work with the saliency graph to drive attention interaction. The core idea is to inject a saliency mask into the cross-attention calculation, and pre-modulate and adjust the bias of the response at the corresponding position of the key-value pair.

[0156] Let the input for the decoding stage be... A matrix composed of object queries The corresponding multi-scale output token of the encoder is represented as follows: Meanwhile, let the saliency map corresponding to the input image be... It is reshaped into a one-dimensional vector. The attention bias term is obtained through linear mapping. :

[0157]

[0158] in, For a linear layer with significant weights, This represents the Sigmoid activation function; This represents the bias term, used to adjust the baseline of the linear transformation and increase the model's expressive power; subsequently, in the cross-attention module, we apply the standard attention scoring formula... :

[0159]

[0160] in, Indicates the original degree of correlation; This represents the scaling factor associated with the j-th element;

[0161] The j-th significance bias Attention logits (attention scores) are added in the key direction to amplify the attractiveness of tokens in salient regions; finally, the attention matrix is ​​obtained by Softmax normalization. Used to generate the output for each query. :

[0162]

[0163] Perform the following steps on the i-th row of matrix A (or the column related to the i-th query): Normalization yields the "attention weight," which takes the original correlation score as the weight. Transform it into a probability distribution between 0 and 1, with all weights summing to 1, representing the degree of attention the i-th query pays to the j-th element;

[0164] In addition to explicit bias control, a query gating mechanism is introduced to directly encode saliency priors in the query embedding; we construct a gating vector by combining the input embedding of each query with the global average response of the saliency map. :

[0165]

[0166] in, This indicates a splicing operation. This indicates a feature concatenation operation, which combines a specific query vector... With global statistical vector By concatenating features along the feature dimension, new joint features are formed; This represents a multilayer perceptron;

[0167] The query is ultimately modulated using the sigmoid function.

[0168]

[0169] This mechanism enables saliency-aware selectivity of the query itself, enhances the response capability of queries in salient regions, and suppresses the risk of cross-regional drift of redundant queries.

[0170] S220, the significance regulation mechanism of bounding box regression path,

[0171] A saliency-guided bounding box geometric decoding mechanism is introduced, which applies saliency weights to the bounding box representation through a residual modulation structure, and designs a position embedding and scale-aware encoding method to effectively improve the robustness and recall accuracy of small target boundary localization.

[0172] First, let the output of a certain object query be... We obtain the initial bounding box regression representation through a two-layer MLP. Including the coordinates of the center point ( ) and width ,high predict:

[0173]

[0174] in, The Multi-Layer Perceptron for Regression is a neural network module composed of fully connected layers, used to map the input to a continuous regression output (here, four continuous values, corresponding to the four attributes of the bounding box). The input feature vector (e.g., the feature representation of the i-th target obtained by the attention mechanism or feature extraction network);

[0175] To enhance the spatial selectivity of the prediction, we introduce a residual significance control module: the average response is extracted by the significance patch matched to the corresponding query. Define the adjustment factor:

[0176]

[0177] Where tanh represents the hyperbolic tangent function. This represents the average significance of the top-k response positions in query-key attention. To adjust the mapping parameters, this adjustment term is ultimately incorporated into the regression output as a residual weighting term. :

[0178]

[0179] in, For residual intensity factor, This indicates element-wise multiplication; this formula is essentially a saliency-guided nonlinear residual channel, which can enhance the saliency weight response of the positioning region without increasing the parameters.

[0180] To accommodate geometric scale awareness of different target sizes, the regression channel incorporates the input embedding of each query and the global average response of the saliency map to construct a gating vector. The structure is as follows:

[0181]

[0182] in, Estimate the coordinates of the spatial center and receptive field pointed to by the query; ultimately... With query expression After splicing, the data is fed into the regressor:

[0183]

[0184] To further enhance the edge structure perception capability, we introduce a boundary significant gradient term into the regression prediction. The Sobel gradient representation is constructed using the Gaussian fuzzy saliency map. Extract the directional gradient at the corresponding position of the query:

[0185]

[0186] This significant boundary direction is used to refine the correction of the center offset, and the final predicted center coordinates are:

[0187]

[0188] Where η is the gradient-guided weight constant; this compensation term is equivalent to fine-tuning at significant boundaries, making the bounding box more aligned in complex backgrounds.

[0189] In summary, this invention systematically proposes a hierarchical and structurally unified spatial fusion mechanism based on saliency map guidance strategies in the Trans-Sal Det decoding path. First, starting from the feature composition of the object query, a cross-scale token representation fusion method is introduced to fully utilize the multi-scale attention features output by the Transformer encoder. Then, spatial consistency alignment and weighted adjustment of tokens at each layer are performed using saliency maps, enabling the query-key cross-attention process to have target region-priority perception capabilities. Based on this, we designed a bounding box regression path under saliency control, introducing a residual saliency weight adjustment mechanism, which significantly improves the model's localization accuracy and robustness in complex backgrounds and dense small object scenes. The overall decoding path achieves a fully nested fusion from "saliency token residual adjustment" to "regression output spatial guidance."

[0190] Specific Implementation Scheme 2: The present invention provides a small target detection system guided by saliency graph and optimized by entropy regularization. The system has program modules corresponding to the above steps, and executes the steps in the above-mentioned small target detection method guided by saliency graph and optimized by entropy regularization when running.

[0191] The other combinations and connections in this implementation scheme are the same as in Specific Implementation Scheme 1.

[0192] Specific Implementation Scheme 2: The present invention provides a computer-readable storage medium storing a computer program configured to implement, when called by a processor, the steps of a small target detection method guided by saliency graph and optimized by entropy regularization.

[0193] The other combinations and connections in this implementation scheme are the same as in Specific Implementation Scheme 1.

[0194] experiment

[0195] Experimental configuration and parameter settings

[0196] The parameter design in the saliency plot-guided path includes: setting the saliency-token multiplication factor to... The attention bias weighting factor is Significant gated terms in the residuals of the bounding box regression channel This makes the guiding effect of the saliency map both controllable and possesses gradient propagation stability. This experiment was used for remote sensing detection of oil well targets.

[0197] The information entropy regularization term employs a gated fusion multi-scale fusion strategy, with initial gate coefficients of... Starting from the 10th epoch, the weights are linearly increased, ultimately resulting in the overall weights of the entropy-regularized loss. Set to 0.1. This regularization term only applies during the training phase and is completely disabled during the inference phase, introducing no computational redundancy.

[0198] The supervised loss for the saliency plot uses the binary cross-entropy (BCE) loss function, with loss weight coefficients of... The goal is to align the network input images and ensure spatial semantic consistency. The Tiny-U-Net module used to generate saliency maps accounts for only about 3% of the total parameters and is optimized synchronously during training, requiring no additional super vision process.

[0199] ablation experiment

[0200] In this invention, the following module abbreviations are used in various model variants: SM represents the Saliency Map module, and TW represents Token-Level Weighting, i.e., token weighting based on the saliency map.

[0201] AB stands for Attention Bias, which is attention bias injection guided by the saliency map; ER stands for Entropy Regularization, an information entropy regularization term used to shrink the attention distribution; MS stands for Multi Scale Token Branches, a multi-scale Transformer branch structure; S stands for Skip Connection, the encoder-decoder connection in the U-Net saliency map module; GF stands for Gated Fusion, a multi-scale entropy control module for gated fusion. Trans-Sal Det represents the complete Trans-Sal Det model.

[0202] Table 1: Comparative Experiments of Trans-Sal Det Structural Ablation (P / R / F1 Indices)

[0203]

[0204] As shown in Table 1, each component of Trans-Sal Det clearly improves performance in oil well target detection tasks, and the overall architecture design exhibits strong modular complementarity and synergistic enhancement. The basic version, Transformer-Only Baseline, relying solely on a minimized structure (no saliency guidance, no attention bias, and no entropy constraints), achieves a precision of 0.801, a recall of 0.867, and an F1 score of 0.832, significantly lower than the complete model.

[0205] Table 2: Ablation experimental results of each module of Trans-Sal Det (unit: %, on JLU-Oil Well test set)

[0206]

[0207] To further verify the contribution of each module in Trans-Sal Det to model performance, we conducted an ablation experiment on the AP (Average Precision) metric. In the experiment, we progressively removed key modules of the model and analyzed the impact of removing each module on the AP metric. The results are shown in Table 2. Through the ablation experiment on the AP metric, we can see the role of each module and its impact on object detection performance. In the experiment, we removed several key modules in Trans-Sal Det sequentially to evaluate their contribution to model performance.

[0208] As shown in Table 2, the key modules introduced by Trans-Sal Det have varying average accuracy (AP) and AP. 50 With AP 75 All three metrics showed significant improvements in model performance. The base model, consisting only of the backbone network and a shallow Transformer encoder, achieved an AP value of 55.783. 75 With an accuracy of only 57.145, this architecture has significant limitations in target boundary localization and small target detection. Introducing SM improves the overall AP by approximately 1.1 percentage points, indicating that the saliency map, as a spatial guide source, can effectively enhance feature focusing capabilities. Further overlaying TW and AB further improves the AP. 75 The improvement is even more significant (from 58.092 to 59.442), indicating that the saliency adjustment at the token level plays a significant role in improving the accuracy of boundary fitting. After further adding the ER information entropy regularization term, the AP reaches 58.269, verifying its optimization effect on sparse attention modeling, especially in complex backgrounds or dense regions with small targets, it can effectively suppress attention generalization.

[0209] Visualization Analysis

[0210] from Figure 2 As can be seen, different models exhibit varying performance in object detection tasks, particularly in terms of bounding box accuracy and confidence. The Trans-Sal Det model performs best across all test samples. Its detected bounding boxes almost perfectly overlap with the ground truth boxes, with minimal error and stable, high confidence. Regardless of complex backgrounds or significant overlap between objects, Trans-Sal Det maintains extremely high accuracy and demonstrates remarkably stable confidence, showcasing its exceptional capabilities in object detection. Other models show varying performance.

[0211] YOLOv8-SG achieves good detection results on some test images, but its bounding box accuracy is slightly lower than Trans-Sal Det, especially when dealing with small targets or complex scenes, where localization accuracy drops. Faster R-CNN (R50-FPN) and Cascade R-CNN (R50-FPN) can accurately detect targets in some simpler images, but in complex backgrounds, their bounding boxes deviate significantly from the ground truth boxes, and their confidence is low. Mask R-CNN (R50-FPN) and SSD (Darknet-53) perform relatively stably in some scenarios, but their bounding box localization accuracy and detail capture ability are clearly inferior to Trans-Sal Det. YOLOv3 (VGG16) and Retina Net models perform significantly worse, especially with small targets or complex backgrounds, where the bounding box localization accuracy is far lower than Trans-Sal Det. Overall, Trans-Sal Det demonstrates the strongest accuracy and robustness across all test samples. Whether in complex environments with overlapping targets or in situations with significant background interference, Trans-Sal Det maintains excellent detection performance. Other models perform poorly in these complex scenarios and cannot match Trans-Sal Det.

[0212] Performance in accuracy and stability. In summary, Trans-Sal Det demonstrates a clear advantage in this task, especially in accuracy and robustness, far surpassing other comparative models and proving its powerful capabilities in practical applications.

[0213] Combination Figure 3 As shown, the loss curves of the Trans-Sal Det model during training are illustrated. With increasing training epochs, the training loss continuously decreases, indicating that the model effectively learns data features and optimizes parameters. Although the validation loss fluctuates initially, it gradually stabilizes as training progresses, indicating that the model's performance on the validation set continuously improves. This trend suggests that the model gradually avoids overfitting or underfitting and reaches a balance during the learning process, ultimately improving its generalization ability to new data and achieving more stable performance.

[0214] Combination Figure 4 The diagram shows the changes in F1 scores on the training and validation sets. The F1 score on the training set gradually increases as training progresses, indicating that the model's performance on the training data is continuously improving. The F1 score on the validation set also gradually increases with relatively small fluctuations, indicating that the model's adaptability to the validation data is gradually increasing. Although the F1 score is close to 0.9, there are still slight fluctuations, reflecting the differences in the model's adaptability on different datasets.

[0215] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A small target detection method guided by saliency graph and entropy regularization optimization, characterized in that, Includes the following steps: S100. In the small object detection model Trans-Sal Det, a high-resolution saliency map is generated through the lightweight operation mechanism Tiny U-Net. include, S110. Extract attention weights, construct an attention score matrix, and define the original integral using a joint modeling approach of temperature regulation and local smoothing softmax. Then, use... Normalize the attention weights, and finally obtain the attention matrix after aggregating the attention map; S120. Calculate the information entropy of the normalized attention probability distribution, including defining the standard information entropy of each row of probability vectors, reconstructing the information entropy after introducing the softmax temperature tensor, introducing a fusion method based on the significance weight of query tokens, weighted averaging the entropy values ​​of different queries to obtain the final regularization term, and constructing the total loss function together with the regularization term after optimizing the classification loss and boundary loss. S130, the Transformer encoding module in the small object detection model Trans-Sal Det includes three resolution levels. For each resolution level, an independent attention weight matrix and corresponding information entropy term are constructed and added to the total loss function. S200 introduces a pyramid-shaped Transformer structure to construct a bottom-up global feature fusion path, and enhances the ability of each object query to focus on the target region through a dynamic injection mechanism of saliency weights. include, S210. Introduce a salient guidance path to make the query-key interaction process regionally selective; During the decoding phase, each query is collaboratively driven by the saliency graph to drive attention interaction. That is, a saliency mask is injected into the cross-attention calculation to pre-modulate and adjust the bias of the response at the corresponding key-value position. S220 introduces a saliency-guided bounding box geometric decoding mechanism, which applies saliency weights to the bounding box representation through a residual modulation structure, and designs a position embedding and scale-aware encoding method to improve the robustness of small target boundary localization and recall accuracy.

2. The small target detection method guided by saliency graph and optimized by entropy regularization according to claim 1, characterized in that: In step S110, the following are included: Let a token sequence of a certain scale be represented as ,in This indicates the number of tokens, which is the number of positions in the flattened feature map of this layer. Let H be the channel dimension, H be the height of the feature map, and W be the width of the feature map. In the h-th attention head, the calculation process for query(Q), key(K), and value(V) is as follows: ； in, It is a linear projection matrix. Dimensions for each attention head; Attention score matrix The Line 1 Column elements represent the first The query for the first... The response strength of each token; to construct a more stable and regularizable attention graph, a joint modeling approach of temperature regulation and local smoothing softmax is adopted; the original score is defined. for: ； in, Let be the query volume of the i-th element under the h-th attention head; Let j be the key vector of the j-th element under the h-th attention head; This is the softening temperature coefficient; For Kronecker delta term, A constant offset term to suppress the effects of self-attention; The final normalized attention weights use a smooth softmax variant. Introducing a power-normalization strategy to control curvature: ； Where k is the kth token; Let be the attention weight of the i-th element to the j-th element under the h-th attention head; This is the power exponent control parameter; For each set of multi-head attention score matrices Preserving the spatial dimension, and averaging or max-pooling the score matrices of all attention heads, we obtain the aggregated attention map: ； in, This is the attention matrix after aggregating the attention map; Aggregate is the mean or maximum value learning weighted pooling; each row in this matrix... Considered the first Attention probability distribution of tokens for each query.

3. The small target detection method guided by saliency graph and optimized by entropy regularization according to claim 2, characterized in that: In step S120, the following are included: For each row of probability vectors Define its standard information entropy for: ； in, A smoothing constant to prevent numerical overflow; To improve the convergence speed, a softmax temperature tensor is introduced. And reconstruct the entropy expression as : ； in, Let i be the i-th softmax temperature tensor; Let be the entropy of the i-th element after reconstruction; To improve the targeting of regularization terms to the target region, a fusion method based on query token saliency weights is introduced, which performs a weighted average of the entropy values ​​of different queries; let Represents the query weight, the final regular expression term. Represented as: ； in, This is the global entropy regulation coefficient; The weight of the i-th query; Will Added to the total loss as a regularization term In the middle, with classification loss With bounding box loss Parallel optimization: ； in, and These are the weighting coefficients.

4. The small target detection method guided by saliency graph and optimized by entropy regularization according to claim 3, characterized in that: In step S130, the following are included: Let the normalized attention weights of each layer be... The corresponding information entropy regularization term is Then the overall entropy is regular and visible. aggregation; Three optional fusion methods are defined: this method adds the entropy terms of the three scales with equal weights to construct a unified regularized path. The expression is as follows: ； Suitable for architectures with spatially balanced three-layer feature maps and approximate token density; Considering that the saliency graph space structure and token fineness focus on the highest resolution layer, sparse regularization is applied only to the first layer: ； To achieve cross-scale adaptive control of information entropy constraints, learnable gating coefficients are introduced into learnable gating coefficients. ,satisfy Constructing fusion expression for: ； in, It is a trainable scalar vector; For the input vector conduct operate; When incorporating the information entropy regularization term into the total loss function, let the independent coefficients be... Controlling its optimization intensity, the overall training objective is expressed as: ； in, This represents any one of the three strategies mentioned above. Including classification loss and bounding box loss, This represents the auxiliary loss guided by significance.

5. The small target detection method guided by saliency graph and optimized by entropy regularization according to claim 4, characterized in that: In step S210, the following are included: Let the input for the decoding stage be... A matrix composed of object queries The corresponding multi-scale output token of the encoder is represented as follows: Meanwhile, let the saliency map corresponding to the input image be... It was reshaped into a one-dimensional vector. And the attention bias term is obtained through linear mapping. : ； in, For a linear layer with significant weights, This represents the Sigmoid activation function; In the cross-attention module, attention scoring is based on the standard attention scoring formula. for: ； in, Indicates the original degree of correlation; This represents the scaling factor associated with the j-th element; The j-th significance bias Attention logits in the key direction are added to amplify the attractiveness of tokens in salient regions; finally, the attention matrix is ​​obtained through softmax normalization. Used to generate the output for each query. : ； in, Perform the operation on the i-th row of matrix A The attention weights obtained after normalization; A query gating mechanism is introduced to directly encode saliency priors in the query embedding; a gating vector is constructed by combining the input embedding of each query with the global average response of the saliency map. : ； in, This indicates a splicing operation. This indicates a feature concatenation operation, which combines a specific query vector... With global statistical vector By concatenating features along the feature dimension, new joint features are formed; Represents a multilayer perceptron; For the Sigmoid function; The final query was modulated as follows: 。 6. The small target detection method guided by saliency graph and optimized by entropy regularization according to claim 5, characterized in that: Step S220 includes, Suppose that the output of a certain object query is The initial bounding box regression representation is obtained through two layers of MLP. Including the coordinates of the center point ( ) and width ,high predict: ； in, It is a regression-type multilayer perceptron; The feature vector of the i-th target input; To enhance the spatial selectivity of the prediction, a residual significance control module is introduced: the average response is extracted by matching the significance patch of the corresponding query. Define the adjustment factor: ； Where tanh is the hyperbolic tangent function. This represents the average significance of the top-k response positions in query-key attention. , To adjust the mapping parameters, this adjustment term is ultimately incorporated into the regression output as a residual weighting term. : ； in, For residual intensity factor, This represents element-wise multiplication; To accommodate geometric scale awareness of different target sizes, the regression channel incorporates the input embedding of each query and the global average response of the saliency map to construct a gating vector. : ； in, Estimate the coordinates of the spatial center and receptive field pointed to by the query; ultimately... With query expression After splicing, the data is fed into the regressor: ； To enhance the ability to perceive edge structures, a significant gradient term at the boundary is introduced into the regression prediction. The Sobel gradient representation is constructed using the Gaussian fuzzy saliency map. Extract the directional gradient at the corresponding position of the query: ； This significant boundary direction is used to refine the correction of the center offset, and the final predicted center coordinates are: ； Where η is the gradient-guided weight constant.

7. The small target detection method guided by saliency graph and optimized by entropy regularization according to claim 6, characterized in that: The small target detection method is used for remote sensing detection of oil well targets.

8. A small target detection system guided by saliency graph and optimized by entropy regularization, characterized in that: The system has a program module corresponding to the steps described in any one of claims 1-7, and executes the steps in the saliency graph-guided and entropy regularization optimization small target detection method described above when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to, when invoked by a processor, implement the steps of the saliency graph-guided and entropy regularization-optimized small target detection method as described in any one of claims 1-7.

Images