A small target detection method based on a cross-modal hypergraph computing framework

Abstract

The present application belongs to the technical field of computer vision and object detection, and particularly relates to a small target detection method based on a cross-modal hypergraph computing framework, aiming to solve the problem of low identification of small targets caused by slice strategy redundancy and insufficient single-modal representation. The method is characterized in that: a target mask is generated by an FSOD module to suppress the background area and perform adaptive slicing; a CHCM module is used to introduce text prior, utilize bidirectional cross attention and cross-modal adaptive hypergraph convolution, and explicitly model the high-order association between visual features and semantic categories. By adopting the above technical solutions, the present application can effectively prune redundant calculations, introduce robust text semantics, and improve the detection accuracy and semantic recognition ability of small targets in complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and target detection technology, and in particular relates to a small target detection method based on a cross-modal hypergraph computation framework. Background Technology

[0002] Small object detection, a long-standing challenge in computer vision, holds immense research value and broad application prospects in practical scenarios such as video surveillance, autonomous driving, medical imaging, and remote sensing image analysis. Compared to objects of regular size, small objects occupy only a small pixel area in an image, resulting in scarce inherent visual cues and severe scale variations. This makes the detection process highly susceptible to adverse factors such as low resolution, occlusion, cluttered backgrounds, and class imbalance. To alleviate these difficulties, existing techniques typically focus on multi-scale feature aggregation, network architecture optimization, and data augmentation. Recently, clustering-based slicing strategies have significantly amplified small object regions by decomposing high-resolution images into multiple candidate regions, thereby improving detection recall to some extent. Furthermore, combining these strategies with feature-level acceleration mechanisms to suppress redundant background computation enhances the model's ability to perceive small objects.

[0003] However, current small target detection methods still face two fundamental technical bottlenecks in practical applications.

[0004] First, although cluster-based slicing strategies can improve recall, they inevitably introduce a large amount of redundant feature computation when processing overlapping regions or candidate blocks, resulting in a significant increase in computational load. This redundancy severely limits the model's efficiency and scalability, especially in large-scale or real-time scenarios where image resolution is constantly improving.

[0005] Secondly, existing detectors generally rely excessively on unimodal visual representations. When the visual features of small targets are inherently weak and blurry, the expressive power of a single modality is limited. Although some research has attempted to use hypergraph modeling to capture higher-order semantic relationships between visual features, mainstream frameworks are still mainly limited to intramodal semantic modeling, failing to explicitly establish cross-modal associations between visual features and heterogeneous features such as textual priors. This lack of utilization of cross-modal complementary information makes it difficult for existing methods to achieve accurate identification of small, blurry targets under complex visual conditions.

[0006] Therefore, how to design a small target detection framework that can effectively suppress redundant computation and enhance feature representation capabilities by modeling cross-modal high-order dependencies has become a key problem that urgently needs to be solved in the current field. Summary of the Invention

[0007] To address the technical problems of computational redundancy caused by clustering-based slicing strategies and low small target recognition due to the limitations of single-modal visual representation capabilities in existing technologies, this invention provides a small target detection method based on a cross-modal hypergraph computation framework.

[0008] This invention constructs a unified computational framework that integrates a Feature Screening Object Detector (FSOD) and a Cross-modal Hypergraph Computing Module (CHCM). This framework suppresses background-dominant regions at the feature level and explicitly models higher-order associations between visual features and text priors. As a result, it reduces computational overhead while improving the detection accuracy and semantic recognition capabilities of small targets in complex scenes.

[0009] This invention provides a small target detection method based on a cross-modal hypergraph computation framework, whose overall architecture is based on a deep neural network backbone-neck-detection head structure. The backbone network adopts a multi-stage feature extraction architecture to extract multi-scale visual features of the input image step by step.

[0010] A small object detection method based on a cross-modal hypergraph computation framework is proposed. The method is built upon a deep learning architecture integrating a Feature Filtering Object Detector (FSOD) and a Cross-Modal Hypergraph Computation Module (CHCM). The FSOD is deployed at the feature output of the backbone network to perform spatial domain filtering on the intermediate feature maps extracted by the backbone network, suppressing background-dominant regions by predicting class-independent object masks. The CHCM is integrated into a bidirectional cross-scale fusion path in the neck region, capturing high-order long-range dependencies between visual instances and semantic categories by introducing prior knowledge from external text.

[0011] The specific steps include: S1: Utilize a backbone network to extract multi-scale features from the input image to be detected, generating at least three sets of intermediate feature maps with different spatial resolutions and semantic depths. I ; S2: Transfer the intermediate feature map I The input is fed into the Feature Filtered Object Detector (FSOD), which generates a class-independent object mask through its lightweight prediction head. The target mask Used to characterize the probability distribution of the existence of potential targets in the spatial dimension; S3: Based on the target mask An adaptive slicing strategy is implemented to identify and extract target-related salient feature regions and suppress background-dominated redundant computation regions, thereby performing spatial domain feature pruning and recombination on the intermediate feature map I. S4: Input the recombined visual features into the cross-modal hypergraph computation module CHCM, and construct a high-order association between visual features and text prior by introducing the category text prior generated by the external text encoder; S5: In the cross-modal hypergraph computation module CHCM, semantic propagation is performed using a cross-modal adaptive hypergraph. By explicitly modeling the long-range dependency relationship between visual instances and semantic labels in non-Euclidean space, the semantically enhanced visual features are output. S6: The enhanced visual features are returned to the neck for bidirectional cross-scale feature fusion. The generated fused features P3, P4 and P5 are sent to the detection head to perform the final target classification and bounding box regression.

[0012] Preferably, the feature filtering target detector (FSOD) in step S2 generates a target mask. The specific implementation method is as follows: S21: Input the intermediate feature map I into a depthwise separable convolutional layer with a stride of 1 to extract local features in the spatial dimension, aiming to maintain the spatial resolution of the feature map while reducing the amount of parameter computation; S22: The output of the depth-separable convolutional layer is mapped to a single-channel energy response map through a 1×1 pointwise convolutional layer; S23: The energy response map is nonlinearly mapped using the Sigmoid activation function, constraining the values ​​to the interval 0,1, to obtain the target mask. The target mask The mathematical expression for calculation is defined as follows: ; in, For the Sigmoid function, For depthwise separable convolution operators, Conv 1×1 This is the pointwise convolution operator.

[0013] Preferably, during the training phase, the target mask The generation process of the hybrid pseudo-label M, controlled by a supervisory guide, is as follows: For each of ( x c , y c , w , hParametric bounding box annotations are used to calculate the Gaussian prior of the geometric center based on its center coordinates, and a Gaussian confidence map is generated. : ; in, τ Set it to 0.5. x c and y c These represent the x and y coordinates of the center of the bounding box, respectively. w and h These represent the width and height of the bounding box, respectively. Using a pre-trained segmentation model and ground truth bounding boxes as cues, shape-aware segmentation cues for the target are extracted to generate high-quality pseudomasks. : ; The Gaussian confidence plot With the high-quality pseudo-mask Perform element-wise Hadamard product operations to obtain the hybrid pseudo-labels. M The calculation formula is: ; Here, ⊙ represents the Hadamard product operation.

[0014] By minimizing the target mask The cross-entropy loss between the hybrid pseudo-labels M forces the feature-selected target detector (FSOD) to learn a discriminative response to the spatial distribution of small targets.

[0015] Preferably, the specific execution steps of the adaptive slicing strategy include: By the target mask Perform max pooling within the active region to extract the local response maximum point, targeting the center position of the target. Make an estimate: ; in, They represent the first i The x and y coordinates of the center of each target, where n represents the number of targets.

[0016] The target mask is obtained using average pooling. The global response statistic is used as a rough estimate of the target scale. s i : ; in, s i Indicates the first i The scale of a goaln Indicates the number of targets.

[0017] The center position of the target Generate an initial fixed size of The feature blocks, and the estimated scale. s i Constraint generation: ; in, These represent the x and y coordinates of the center of the i-th target, respectively. This represents the width and height of the initial feature block. and They represent the first i The x and y coordinates of the top left corner of the target bounding box. and The first i Each target represents the horizontal and vertical coordinates of the upper right corner of the bounding box.

[0018] Based on the spatial distribution of active pixels Adaptive adjustment of patch boundaries: ; in, and They represent the first i The x and y coordinates of the top left corner of the target bounding box. and The first i Each target represents the x and y coordinates of the upper right corner of the bounding box. and These represent the active regions. The Middle i The horizontal and vertical coordinates of each pixel.

[0019] Compact feature slices are generated based on the adjusted quadripolar boundaries, and a non-maximum suppression strategy is applied to remove redundant feature blocks with an overlap rate higher than a preset threshold, thereby completing the discretization extraction of salient feature regions.

[0020] Preferably, before constructing the hypergraph, the cross-modal hypergraph computation module CHCM first achieves preliminary interaction between visual features and text priors through a bidirectional cross-attention mechanism (BiCA). The specific process is as follows: Obtain the text embedding vector sequence generated by the pre-trained text encoder from the dataset category labels. , ,in N t Indicates the number of categories. C Indicates the feature dimension; In the visual branch, the visual features extracted by the adaptive slicing strategy are used as the query vector Query, and the text embedding vector sequence is used as the query vector Query. As key vectors and value vectors, a visual representation enhanced with textual context is obtained through a scaling dot product attention mechanism; In the text branch, the text embedding vector sequence As a query vector, the visual features are used as key vectors and value vectors to correct the semantics of the text based on the visual context. The visual features and text features after interaction through the Bi-directional cross-attention mechanism (BiCA) are concatenated to form the vertex set of the subsequent hypergraph structure.

[0021] Preferably, the cross-modal hypergraph computation module CHCM constructs the dynamic hyperedge prototype P in the following manner: A learnable parameterized matrix is ​​pre-defined in the memory library as the basic prototype. It is used to store resident semantic patterns among categories in the dataset; Global average pooling and global max pooling are performed on the current input visual features and text features respectively to capture the transient contextual information of the current scene; The pooled global vectors are concatenated and then projected through a multilayer perceptron to generate a global offset Δ. P ; The basic prototype With the global offset Δ P Element-wise addition yields a dynamic hyperedge prototype P that can dynamically adjust its topology according to the input image content.

[0022] Preferably, the cross-modal hypergraph computation module (CHCM) utilizes a multi-head similarity mechanism to establish the association between vertices and hyperedges. Specific steps include: Map the concatenated visual-text feature vertices to a vertex query vector Q: ; f v and f t These represent the visual and textual features after bidirectional cross-attention enhancement, respectively.

[0023] Calculate the vertex query vector Q With the aforementioned dynamic hyperedge prototype P Cross-correlation coefficients across multiple feature subspaces: ; in, h For the number of attention heads, dh Spatial dimension.

[0024] The cross-correlation coefficients are normalized based on a scaling factor to generate a vertex-hyperedge incidence matrix H. Each element in the vertex-hyperedge incidence matrix H represents the weight of a specific vertex to a specific hyperedge, and its mathematical expression is as follows: .

[0025] Preferably, the semantic propagation process of the cross-modal adaptive hypergraph is implemented through two-stage hypergraph convolution: In the first stage, vertex features are aggregated into the hyperedge space. The feature vectors of the vertex set are projected onto the hyperedge manifold using the vertex-hyperedge association matrix H, generating hyperedge space features E carrying higher-order combinatorial semantics. The calculation formula is as follows: ; In the second stage, the hyperedge features are backpropagated to the vertex space. The hyperedge space features E are redistributed to each visual and text vertex using the transpose of the vertex-hyperedge association matrix H, achieving deep collaborative enhancement of cross-modal information. The calculation formula is as follows: ; Where X represents the input vertex feature. D v and D e These are the vertex degree matrix and the hyperedge degree matrix, respectively. W 1 and W 2 is the learnable transformation weight matrix.

[0026] Preferably, the overall training objective function of the method is... L total Standard classification loss adopted by the YOLO series L cls Target confidence loss obj With bounding box regression loss reg Mask monitoring loss L pixel and cross-modal contrast loss L con It is composed of weighted combinations, and its expression is: ; Among them, the cross-modal contrast loss L con This is used to explicitly align visual representations with corresponding textual semantics within a shared embedding space, by calculating visual-text contrast terms. L v2t Text-visual contrast itemsL t2v The mean value was obtained to enhance the model's ability to identify small targets under conditions of sparse visual signals; ; ; in, Represents cosine similarity. For temperature parameters. This bidirectional modeling ensures symmetric contributions from both modalities during feature alignment, which is particularly important for modeling higher-order cross-modal relationships via hypergraphs, comparing loss weights. The value was kept constant at 0.3 in all experiments.

[0027] Preferably, the multi-scale feature maps B3, B4, and B5 output by the backbone network have downsampling rates of 8x, 16x, and 32x, respectively; the feature selection target detector (FSOD) operates in parallel on the feature streams corresponding to the multi-scale feature maps B3, B4, and B5; the cross-modal hypergraph computation module (CHCM) is deployed as a plug-in after the cross-scale splicing operator in the neck region, and uses the global semantic associations captured by hypergraph convolution to reconstruct the spliced ​​features, and outputs the reconstructed results as the fused features P3, P4, and P5 to the detection head.

[0028] In summary, this invention constructs a complete closed loop from low-level feature extraction to high-level semantic reasoning by combining the hard screening in the spatial domain using the FSOD module with the soft association enhancement in the semantic domain using the CHCM module. This framework not only suppresses the interference of invalid background noise on the detection results but also endows the model with the ability to perform reliable reasoning under extremely weak visual signals through cross-modal knowledge transfer. Its non-Euclidean modeling method breaks through the bottleneck of limited receptive field in traditional convolutional operations, providing a novel technical path for achieving high-efficiency and high-precision small target detection. Attached Figure Description

[0029] Figure 1 This is a comparative diagram of hypergraph computation paradigms, in which... Figure 1 In the middle (a), it is a schematic diagram of traditional single-modal hypergraph modeling that only captures semantic relationships within a modality. Figure 1 Figure (b) is a schematic diagram of the cross-modal hypergraph computation framework proposed in this invention.

[0030] Figure 2 This is a schematic diagram of the overall cross-modal hypergraph computation framework of the present invention.

[0031] Figure 3 This is a schematic diagram of the network architecture of the FSOD module of the present invention.

[0032] Figure 4 This is a schematic diagram of the network architecture of the CHCM module of the present invention. Detailed Implementation

[0033] The following is in conjunction with the appendix Figures 1-4 The present invention will be further described in detail below: To make the objectives, technical solutions, and advantages of this invention clearer, a small target detection method based on a cross-modal hypergraph computation framework will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention. The following description will cover multiple dimensions, including the overall system architecture, the micro-design of core modules, the construction of the mathematical model, the optimization of the training strategy, and the verification through quantization experiments.

[0034] See Figure 1 As shown, traditional single-modal hypergraph computation ( Figure 1 In (a) only semantic relationships within a single modality, thus failing to utilize complementary information from heterogeneous feature spaces. In contrast, the cross-modal adaptive hypergraph computing mechanism proposed in this paper ( Figure 1 (b) By jointly modeling visual features and textual priors within a unified hypergraph framework, non-Euclidean high-order dependencies are explicitly established. This design supports adaptive topology construction and cross-modal information transfer, thereby promoting effective semantic interaction and synergistic enhancement among cross-modal features. Furthermore, by introducing cross-modal contrastive loss to supervise the learning process, consistent semantic alignment between visual and textual representations is achieved within the shared embedding space.

[0035] This invention provides a small target detection method based on a cross-modal hypergraph computing framework. The implementation environment is built on a high-performance computing cluster, with hardware configured with a graphics processing unit possessing large-capacity video memory. The software environment employs a deep learning framework supporting operator depth optimization and a distributed communication protocol. The method's macro-architecture relies on a cascaded structure of a backbone network, a neck network, and a detection head. The backbone network extracts multi-scale features from the input image step-by-step through a preset downsampling rate.

[0036] Specifically, see Figure 2 As shown, the input image first enters the initial convolutional layer of the backbone network for downsampling and channel dimension mapping. Subsequently, it passes through a series of residual blocks or cross-stage local network structures to enhance semantic expressiveness while preserving spatial information. The multi-scale feature maps output by the backbone network are B3, B4, and B5, with corresponding downsampling ratios typically set to 8x, 16x, and 32x to ensure coverage of targets at different scales.

[0037] At the output of the backbone network, this invention implements spatial domain computational optimization by deploying a Feature-Selected Target Detector (FSOD). See also Figure 3 As shown, the Feature Filtering Object Detector (FSOD) does not simply crop features, but rather identifies and extracts salient target regions from the original feature map through a learnable mask generation path. Specifically, the intermediate feature map I is input into the FSOD, first passing through a depthwise separable convolutional layer (DWConv) with a stride of 1. This convolutional layer maintains the feature map resolution while reducing the number of computational parameters through intra-channel convolution. Subsequently, a 1×1 pointwise convolutional layer (PWConv) performs cross-channel feature fusion and dimensionality reduction, mapping the multi-channel visual features to a single-channel energy distribution map. A Sigmoid activation function is then applied to this energy distribution map. A nonlinear mapping is performed, and its value is constrained to the interval of 0 and 1 to generate the target mask. The target mask Each pixel value in the graph represents a probability estimate that the spatial location belongs to the foreground target region.

[0038] Regarding the target mask To address the accuracy issue, this invention designs a supervised learning mechanism based on hybrid pseudo-labels M. Since the geometric boundaries of small targets are easily blurred in coarse-grained feature maps, this invention introduces a Gaussian confidence map. With high-quality pseudomasks Composite constraints. In constructing the Gaussian confidence plot. In this invention, a two-dimensional Gaussian distribution is established based on the center coordinates of the truth bounding box, wherein the variance parameter... The Gaussian kernel is dynamically correlated with the short side length of the target to ensure that its coverage matches the actual target size. Simultaneously, a high-quality pseudomask is utilized. The high-quality pseudomask provides a more refined geometric profile. Generate by inputting truth boxes as cues into a pre-trained general segmentation model. This is achieved by using Gaussian confidence maps. With high-quality pseudomasks Element-wise Hadamard product operations are performed to obtain the final hybrid pseudo-label M. This hybrid labeling method retains the high weight of the target center region while suppressing the weight of background noise in the loss function through shape boundaries. During training, binary cross-entropy loss is used to apply the target mask. Gradient backpropagation forces the model to accurately lock onto potential target candidate regions during the inference phase.

[0039] Based on the generated target mask The present invention further implements an adaptive slicing strategy. Specifically, the system performs a max pooling operation with a step size of 1 in regions where the mask response is higher than a preset threshold, and extracts local maxima points as center position estimates. Simultaneously, the global statistical response of the mask is obtained using the average pooling operator, and combined with the receptive field range at the current scale, a coarse scale estimate of each candidate region is derived. s i Subsequently, based on the spatial distribution set A of activated pixels, the quadripolar boundary coordinates of candidate feature blocks are adaptively calculated. This process is accomplished by retrieving the minimum and maximum latitude and longitude coordinates in set A, thereby generating compact, target-centered feature blocks. To eliminate redundant calculations caused by overlapping regions, this invention applies a non-maximum suppression (NMS) strategy after slicing to merge feature blocks with a spatial overlap exceeding a certain proportion. This mask-guided dynamic slicing mechanism allows the backend fine-grained detection operator to process only high-value subsets of the image, thus significantly improving the system's throughput under high-resolution input.

[0040] Regarding cross-modal collaborative enhancement, this invention integrates a cross-modal hypergraph computation module (CHCM) in the neck region. See also Figure 4 As shown, the core logic of this module lies in using textual semantic priors to compensate for the lack of visual signal representation on extremely small targets. First, this invention pre-utilizes a text encoder from a vision-language model to transform all category labels in the dataset into high-dimensional embedding vectors. X t To achieve initial interaction of cross-modal information, a bidirectional cross-attention mechanism (BiCA) is deployed within the Cross-Modal Hypergraph Computation (CHCM) module. In the visual branch, the sliced ​​and reconstructed visual features are used as the query, while the categorical text embedding vectors are used as the key and value. A scaling dot product attention mechanism is used to calculate the affinity distribution of visual features for different semantic categories. A similar operation is performed in the symmetrical text branch, using visual context information to adjust the weights of the text embeddings. This bidirectional interaction ensures that the features input to subsequent hypergraph construction stages already possess preliminary cross-modal alignment properties.

[0041] The advanced solution of the Cross-Modal Hypergraph Computation Module (CHCM) lies in constructing a cross-modal adaptive hypergraph. Unlike traditional Euclidean convolution, the hypergraph structure can connect multiple vertices through a single hyperedge, thereby modeling high-order long-range dependencies between visual instances and semantic categories. In the implementation of this invention, the vertex set is composed of enhanced visual feature sequences and text embedding sequences. To construct the topology, this invention first generates a dynamic hyperedge prototype P. The dynamic hyperedge prototype P is derived from a statically stored base prototype. P 0 and dynamically generated global offset Δ PTogether they constitute the basic prototype. P 0 is a learnable parameter matrix that internally encodes the intrinsic co-occurrence probabilities and semantic topology of each category in the dataset. Global offset Δ P This is achieved by performing multidimensional global pooling on the visual and textual features of the current input frame, followed by projection through a multilayer perceptron (MLP). This design allows the topological logic of the hypergraph to evolve in real time as the input scene changes.

[0042] In establishing the association strength between vertices and hyperedges, this invention utilizes a multi-head similarity mechanism to calculate the cross-correlation between the vertex query vector Q and the dynamic hyperedge prototype P, thereby generating a vertex-hyperedge association matrix H. Each element in this matrix... H ij The first i The feature vertex pair of the first j The contribution of each hyperedge is calculated. Then, a hypergraph convolution operation is performed for deep semantic propagation. In the first stage, vertex features are aggregated into the hyperedge space via the association matrix H, generating hyperedge space features E. In the second stage, the hyperedge space features E are transformed by another set of transformation matrices and then backpropagated to the individual vertex spaces. This process employs the LeakyReLU activation function and a matrix derived from the vertex degree matrix. D v With hypermarginal matrix D e The diagonal normalization term ensures the stability of the gradient flow and prevents feature distribution shift. Visual features enhanced by hypergraph. They are remapped back to the original spatial arrangement and fused with the cross-scale feature paths of the neck to form fused features P3, P4, and P5.

[0043] To ensure modal convergence consistency during training, this invention introduces bidirectional contrastive loss, i.e., cross-modal contrastive loss, into the loss function. L con This loss function explicitly enhances the discriminative power of features by bringing visual representations closer to their corresponding ground truth class text vectors within a shared embedding manifold space, while simultaneously widening the distance to other irrelevant classes. Ultimately, the total loss function... L total It integrates classification and regression losses for detection tasks, mask supervision loss for FSOD, and contrastive loss for CHCM. In a preferred parameter configuration, the contrastive loss weights... Set to 0.3, mask loss weight Set to 1.0.

[0044] In terms of hardware, the study was conducted on a server equipped with four graphics accelerator cards, each with 24GB of dedicated video memory. On the software side, a deep learning framework was used to write the operators, and the simultaneous stochastic gradient descent (SGD) algorithm was used for weight updates. The initial learning rate was set to 0.01, with cosine annealing used for decay, a batch size of 16, and a total of 300 iterations (epochs) were executed.

[0045] Introducing the Cross-Modal Hypergraph Computation (CHCM) module effectively utilizes textual semantic priors to assist in visual feature modeling, providing crucial classification cues in scenarios with weak visual signals. Without the Feature Selection Object Detector (FSOD), the system suffers from high computational cost and a significant drop in FPS, severely impacting real-time performance. This invention, through adaptive slice selection in the FSOD module, reduces computational cost with only a small increase in parameter load, maintaining high-speed inference capabilities and achieving an optimal balance between accuracy and speed.

[0046] Furthermore, internal ablation experiments on the cross-modal hypergraph computation module CHCM demonstrate that the dynamic hyperedge prototype... P The design must be adaptable to complex environments. If the dynamic offset Δ P Remove, keep only the basic prototype P 0. The model's detection accuracy decreases in atypical scenarios such as fog and low light. This invention uses offsets generated by the global context to adjust the hypergraph topology in real time, giving the model stronger generalization robustness.

[0047] in, .

[0048] In practical engineering implementation, the mask threshold generated by the Feature Filtering Object Detector (FSOD) is set to 0.4. At this threshold, the mask can filter out approximately 60%-70% of invalid background regions. The edge expansion coefficient of the adaptive slicing is set to 1.2 to ensure that the target retains some local contextual background information after slicing. The number of hyperedges in the Cross-Modal Hypergraph Computation (CHCM) module is set to 64, the number of heads in the multi-head attention module is set to 8, and the subspace dimension of each head is [not specified]. d h The value is 64. These parameters were set after multiple rounds of hyperparameter search experiments, aiming to maximize the expressive power of topological associations.

[0049] In the first stage of performing hypergraph convolution, the weight matrix... W The initialization of 1 employs an orthogonal initialization strategy to maintain variance consistency across features. The negative slope coefficient of LeakyReLU is set to 0.01. For the vertex degree matrix... D vThe calculation of the degree matrix is ​​defined as the sum of the elements in each column of the incidence matrix H, representing the global connectivity strength of a single vertex to all hyperedges. Similarly, the hyperedge degree matrix... D e This is defined as the sum of the elements in each row, reflecting the information density carried by each hyperedge. Through and The premultiplication operation enables adaptive scaling of feature weights, avoiding the gradient explosion problem in densely populated vertex regions.

[0050] In this invention, the Bidirectional Cross-Attention (BiCA) mechanism employs a sequential reshaping operation for visual features. For a size of... H×W×C The feature map is flattened into a length of L=H×W The feature sequence is calculated using sinusoidal positional encoding to compensate for the loss of spatial location information caused by the flattening operation. The text feature branch employs local enhancement based on convolution operators, capturing local semantic combinations between words by applying one-dimensional convolution to the text embedding vectors.

[0051] In summary, this invention constructs a highly efficient, accurate, and robust closed-loop system for small target detection by employing hard spatial pruning at the data flow entry point of the Feature Filtered Target Detector (FSOD) and high-order semantic soft association implemented by the Cross-Modal Hypergraph Computation Module (CHCM) during the feature fusion stage. This system not only theoretically models non-Euclidean associations but also demonstrates its superior performance in resolving computational redundancy and semantic ambiguity through detailed data verification in engineering practice. This integrated technical concept provides solid technical support for precision target monitoring in high-resolution image environments, possessing significant industrial application value and technological foresight.

[0052] In practical software engineering deployment, each operator of the method has undergone computation graph-level fusion optimization. The depthwise separable convolution and subsequent slicing logic in FSOD are encapsulated into a unified CUDAKernel, reducing the synchronization overhead between the CPU and GPU. The CHCM module uses half-precision floating-point numbers for training and inference, leveraging the automatic mixed precision (AMP) technology of deep learning frameworks to further unleash the hardware's computational potential while ensuring numerical accuracy. Through this series of collaborative designs from low-level hardware instructions to high-level algorithm architecture, this invention ultimately realizes a production-grade detection system that can be directly applied to complex scenarios such as large-scale aerial imagery and medical pathology images.

[0053] Furthermore, this invention also considers the feasibility of deploying the model on edge devices. By lightweight compression of the mask generated by FSOD and employing knowledge distillation to transfer the knowledge of a large vision-language model to a lightweight cross-modal hypergraph computing module (CHCM), the model can maintain high detection consistency even in resource-constrained environments. This meticulous consideration of the entire engineering implementation process demonstrates the comprehensiveness and maturity of this invention's solution in practical applications.

[0054] By introducing category-level language priors and explicitly modeling higher-order cross-modal relationships, CHCM enhances semantic coherence and reduces category ambiguity when visual cues are insufficient. As shown in Figure 4, CHCM constructs a dynamic cross-modal hypergraph that can jointly reason about visual instances and category-level textual semantics, thereby achieving effective information dissemination beyond pairwise interactions.

[0055] Specifically, we first use a pre-trained CLIP model to encode the dataset category labels into text embeddings. ,in Indicates the number of categories. The feature dimensions are represented. These textual features provide a category-level semantic prior, which is then aligned with the visual representations extracted from the detector.

[0056] To effectively fuse complementary information across modalities, CHCM proposes a cross-modal adaptive hypergraph generation mechanism. Before constructing the hypergraph, visual and textual features are initially fused using a Bidirectional Cross-Attention (BiCA) module, as shown in the dashed area of ​​Figure 4. The two modalities are first projected using dot-matrix convolutions and decomposed into query, key, and value representations. In the text branch, the text query focuses on visual keys and aggregates visual values ​​to enhance the linguistic representation. In the visual branch, the visual query focuses on text keys to enrich visual features through semantic context. The outputs of both branches are linearly projected, then nonlinearly transformed and added to the original features via residual connections, ultimately generating enhanced visual features. f v and text features f t .

[0057] make and These represent the enhanced visual and textual vertex features, respectively. To generate an adaptive hyperedge prototype that reflects the global cross-modal context, we apply global average pooling and global max pooling to the two modalities respectively to generate context vectors. , , and These vectors are concatenated to form a unified global context representation: ; This value is then projected as a global offset via a learnable mapping transformation: .

[0058] This offset is related to the learnable base prototype. P Adding zeros generates a dynamic hyperedge prototype. P This allows the hypergraph structure to adapt to scene-specific semantic configurations. To compute the association between vertices and hyperedges, visual and textual features are concatenated and projected to obtain the vertex query vector: ; in, f v and f t These represent the visual and textual features after bidirectional cross-attention enhancement, respectively. For convolution operations, the aim is to perform linear projection on the concatenated visual-text joint features, thereby effectively capturing and fusing cross-modal information.

[0059] Then, a multi-head similarity mechanism is used to calculate the similarity between the vertex query and the hyperedge prototype. Q and P Divide equally into The overall similarity is defined as the average of the weighted dot product similarities across all subspaces: ; in, d h = C / After normalization, the resulting similarity scores form a continuous vertex-hyperedge participation matrix: ; This value represents the degree to which each visual or text vertex contributes to each hyperedge.

[0060] Based on the constructed hypergraph, CHCM performs hypergraph convolution to propagate and optimize semantic information. First, vertex features are aggregated into hyperedge representations, followed by a non-linear transformation: ; ; The updated hyperedge features are then fed back to the vertex domain to generate an enhanced vertex representation, which is split into visual and textual representations and residually connected back to the original textual or visual vertex features. ; ; Among them, W e and W n Let represent the hyperedge learning weight matrix and the vertex learning weight matrix, respectively, and H represent the continuous vertex-hyperedge participation matrix.

[0061] Through this process, CHCM can effectively capture high-order cross-modal dependencies and enhance discriminative semantic cues, which is particularly advantageous for recognizing small, ambiguous targets under complex visual conditions.

[0062] To enhance the semantic consistency between the visual and textual representations learned by the proposed CHCM, we introduce a bidirectional contrastive loss as an additional supervisory signal. This objective promotes the tight alignment of matched visual-text pairs in the embedding space while separating mismatched pairs, thereby achieving stable cross-modal representation learning within the hypergraph framework.

[0063] Overall training objective function L total Standard classification loss adopted by the YOLO series L cls Target confidence loss obj With bounding box regression loss reg Mask monitoring loss L pixel and cross-modal contrast loss L con It is composed of weighted combinations, and its expression is: ; Among them, the cross-modal contrast loss L con Cross-modal contrastive loss is used to explicitly align visual representations with corresponding textual semantics within a shared embedding space. L con Computational visual-text contrast items L v2t Text-visual contrast items L t2v The mean was obtained as follows: ; The aim is to enhance the model's ability to identify small targets under conditions of sparse visual signals; ; ; in, Represents cosine similarity. For temperature parameters. This bidirectional modeling ensures symmetric contributions from both modalities during feature alignment, which is particularly important for modeling higher-order cross-modal relationships via hypergraphs, comparing loss weights. The value was kept constant at 0.3 in all experiments.

[0064] Finally, robustness tests under different weather conditions show that the detection rate of this invention decreases significantly less than that of traditional single-modal detectors under extreme conditions such as rain and snow. This is attributed to the fact that the cross-modal hypergraph structure can forcibly anchor the essential features of the target through textual semantics, thus enabling it to make correct inference decisions based on prior knowledge of the category and the global semantic topology even when visual features are severely interfered with by weather noise. This characteristic makes this invention demonstrate significant application potential in fields with extremely high safety requirements, such as autonomous driving, smart city defense, and disaster search and rescue.

[0065] Those skilled in the art should understand that the above embodiments are merely one specific implementation of the present invention, and the parameters, number of network layers, and specific model components (such as backbone network type) used can be adjusted according to actual task requirements. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention. The present invention is not limited to the above specific embodiments, but aims to provide a general methodology combining cross-modal and hypergraph computation.

Claims

1. A small target detection method based on a cross-modal hypergraph computation framework, characterized in that, The method is built on a deep learning architecture that integrates a feature-selective object detector (FSOD) and a cross-modal hypergraph computation module (CHCM). Its specific steps include: S1: Utilize a backbone network to extract multi-scale features from the input image to be detected, generating at least three sets of intermediate feature maps with different spatial resolutions and semantic depths. I ; S2: Transfer the intermediate feature map I The input is fed into the Feature Filtered Object Detector (FSOD), which generates a class-independent object mask through its lightweight prediction head. The target mask Used to characterize the probability distribution of the existence of potential targets in the spatial dimension; S3: Based on the target mask An adaptive slicing strategy is implemented to identify and extract target-related salient feature regions and suppress background-dominated redundant computation regions, thereby performing spatial domain feature pruning and recombination on the intermediate feature map I. S4: Input the recombined visual features into the cross-modal hypergraph computation module CHCM, and construct a high-order association between visual features and text prior by introducing the category text prior generated by the external text encoder; S5: In the cross-modal hypergraph computation module CHCM, semantic propagation is performed using a cross-modal adaptive hypergraph. By explicitly modeling the long-range dependency relationship between visual instances and semantic labels in non-Euclidean space, the semantically enhanced visual features are output. S6: The enhanced visual features are returned to the neck for bidirectional cross-scale feature fusion. The generated fused features P3, P4 and P5 are sent to the detection head to perform the final target classification and bounding box regression.

2. The small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, The feature-selective target detector (FSOD) generates a target mask in step S2. The specific implementation method is as follows: S21: Input the intermediate feature map I into a depthwise separable convolutional layer with a stride of 1 to extract local features in the spatial dimension, aiming to maintain the spatial resolution of the feature map while reducing the amount of parameter computation; S22: The output of the depth-separable convolutional layer is mapped to a single-channel energy response map through a 1×1 pointwise convolutional layer; S23: The energy response map is nonlinearly mapped using the Sigmoid activation function, constraining the values ​​to the interval 0,1, to obtain the target mask. The target mask The mathematical expression for calculation is defined as follows: ； in, For the Sigmoid function, For depthwise separable convolution operators, Conv 1×1 This is the pointwise convolution operator.

3. The small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, During the training phase, the target mask Under the supervision and guidance of a hybrid pseudo-label M, the generation process of the hybrid pseudo-label M is as follows: For each of ( x c , y c , w , h Parametric bounding box annotations are used to calculate the Gaussian prior of the geometric center based on its center coordinates, and a Gaussian confidence map is generated. : ； in, τ Set it to 0.

5. x c and y c These represent the x and y coordinates of the center of the bounding box, respectively. w and h These represent the width and height of the bounding box, respectively. Using a pre-trained segmentation model and ground truth bounding boxes as cues, shape-aware segmentation cues for the target are extracted to generate high-quality pseudomasks. : ； The Gaussian confidence plot With the high-quality pseudo-mask Perform element-wise Hadamard product operations to obtain the hybrid pseudo-labels. M The calculation formula is: ； Where ⊙ represents the Hadamard product operation; By minimizing the target mask The cross-entropy loss between the hybrid pseudo-labels M forces the feature-selected target detector (FSOD) to learn a discriminative response to the spatial distribution of small targets.

4. The small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, The specific execution steps of the adaptive slicing strategy include: By the target mask Perform max pooling within the active region to extract the local response maximum point, targeting the center position of the target. Make an estimate: ； in, represents the x and y coordinates of the center of the i-th target, respectively, and n represents the number of targets; The target mask is obtained using average pooling. The global response statistic is used as a rough estimate of the target scale. s i : ； Among them, s i Let n represent the scale of the i-th target and n represent the number of targets. The center position of the target Generate an initial fixed size of The feature blocks, and the estimated scale. s i Constraint generation: ； in, These represent the x and y coordinates of the center of the i-th target, respectively. This represents the width and height of the initial feature block. and They represent the first i The x and y coordinates of the top left corner of the target bounding box. and The first i Each target represents the x and y coordinates of the upper right corner of the bounding box; Based on the spatial distribution of active pixels Adaptive adjustment of patch boundaries: ； in, and They represent the first i The x and y coordinates of the top left corner of the target bounding box. and Let x and y be the x and y coordinates of the upper right corner of the bounding box for the i-th target, respectively. and These represent the active regions. The horizontal and vertical coordinates of the i-th pixel; Compact feature slices are generated based on the adjusted quadripolar boundaries, and a non-maximum suppression strategy is applied to remove redundant feature blocks with an overlap rate higher than a preset threshold, thereby completing the discretization extraction of salient feature regions.

5. A small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, Before constructing the hypergraph, the cross-modal hypergraph computation module CHCM first achieves preliminary interaction between visual features and text priors through a bidirectional cross-attention mechanism (BiCA). The specific process is as follows: Obtain the text embedding vector sequence generated by the pre-trained text encoder from the dataset category labels. , ,in N t Indicates the number of categories. C Indicates the feature dimension; In the visual branch, the visual features extracted by the adaptive slicing strategy are used as the query vector Query, and the text embedding vector sequence is used as the query vector Query. As key vectors and value vectors, a visual representation enhanced with textual context is obtained through a scaling dot product attention mechanism; In the text branch, the text embedding vector sequence As a query vector, the visual features are used as key vectors and value vectors to correct the semantics of the text based on the visual context. The visual features and text features after interaction through the Bi-directional cross-attention mechanism (BiCA) are concatenated to form the vertex set of the subsequent hypergraph structure.

6. The small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, The cross-modal hypergraph computation module CHCM constructs the dynamic hyperedge prototype P in the following manner: A learnable parameterized matrix is ​​pre-defined in the memory library as the basic prototype. P 0, used to store persistent semantic patterns between categories in the dataset; Global average pooling and global max pooling are performed on the current input visual features and text features respectively to capture the transient contextual information of the current scene; The pooled global vectors are concatenated and then projected through a multilayer perceptron to generate a global offset ΔP. The basic prototype P 0 is added element-wise to the global offset ΔP to obtain the dynamic hyperedge prototype P that can dynamically adjust the topology according to the input image content.

7. A small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, The cross-modal hypergraph computation module CHCM establishes the association between vertices and hyperedges using a multi-head similarity mechanism. Specific steps include: Map the concatenated visual-text feature vertices to a vertex query vector Q: ； in, f v and f t These represent the visual and textual features after bidirectional cross-attention enhancement, respectively. Calculate the vertex query vector Q With the aforementioned dynamic hyperedge prototype P Cross-correlation coefficients across multiple feature subspaces: ； in, h For the number of attention heads, d h Spatial dimension; The cross-correlation coefficients are normalized based on a scaling factor to generate a vertex-hyperedge incidence matrix H. Each element in the vertex-hyperedge incidence matrix H represents the weight of a specific vertex to a specific hyperedge, and its mathematical expression is as follows: 。 8. A small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, The semantic propagation process of the cross-modal adaptive hypergraph is achieved through two-stage hypergraph convolution: In the first stage, vertex features are aggregated into the hyperedge space. The feature vectors of the vertex set are projected onto the hyperedge manifold using the vertex-hyperedge association matrix H, generating hyperedge space features E carrying higher-order combinatorial semantics. The calculation formula is as follows: ； In the second stage, the hyperedge features are backpropagated to the vertex space. The hyperedge space features E are redistributed to each visual and text vertex using the transpose of the vertex-hyperedge association matrix H, achieving deep collaborative enhancement of cross-modal information. The calculation formula is as follows: ； Where X represents the input vertex feature. D v and D e These are the vertex degree matrix and the hyperedge degree matrix, respectively. W 1 and W 2 is the learnable transformation weight matrix.

9. A small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, The overall training objective function of the method L total Standard classification loss adopted by the YOLO series L cls Target confidence loss obj With bounding box regression loss reg Mask monitoring loss L pixel and cross-modal contrast loss L con It is composed of weighted combinations, and its expression is: ； Among them, the cross-modal contrast loss L con This is used to explicitly align visual representations with corresponding textual semantics within a shared embedding space, by calculating visual-text contrast terms. L v2t Text-visual contrast items L t2v The mean value was obtained to enhance the model's ability to identify small targets under conditions of sparse visual signals; ； ； in, Represents cosine similarity. The temperature parameter is empirically set to 0.

07. This bidirectional modeling ensures symmetrical contributions from both modalities during feature alignment, which is particularly important for modeling higher-order cross-modal relationships via hypergraphs, comparing the loss weights. The value was kept constant at 0.3 in all experiments.

10. A small target detection method based on a cross-modal hypergraph computation framework according to claim 1, characterized in that, The multi-scale feature maps B3, B4, and B5 output by the backbone network have downsampling rates of 8x, 16x, and 32x, respectively. The Feature Filtering Target Detector (FSOD) operates in parallel on the feature streams corresponding to the multi-scale feature maps B3, B4, and B5. The Cross-Modal Hypergraph Computation Module (CHCM) is deployed as a plug-in after the cross-scale splicing operator in the neck region. It uses the global semantic associations captured by hypergraph convolution to reconstruct the spliced ​​features and outputs the reconstructed results as the fused features P3, P4, and P5 to the detection head.

Images