A weakly supervised counting method based on frequency domain guidance and hole scanning

Abstract

This invention relates to computer vision and target counting technology, and discloses a weakly supervised counting method based on frequency domain guidance and hole scanning. The counting method includes four stages: feature encoding, frequency domain guidance, hole scanning modeling, and density regression. First, a two-dimensional image to be counted is acquired, and its image features are extracted. These features are mapped to a one-dimensional sequential feature vector, and type embedding and position embedding are introduced. Then, Fourier transform is used to map the features to the frequency domain, filtering out background noise and restoring the guidance signal. Subsequently, the guidance signal is converted into a step-size bias and injected into the Mamba model. Multi-scale feature capture is achieved by combining multi-expansion-rate hole selective scanning. Finally, weakly supervised regression prediction is completed through density map integration. This invention combines frequency domain guidance and hole scanning to achieve complementary high and low frequency features. The model can locate target clusters in a large field of view and accurately distinguish individual small targets, ensuring counting accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer vision and target counting technology, specifically a weakly supervised counting method based on frequency domain guidance and hole scanning. Background Technology

[0002] Target counting is a core technology for calculating the number of specific target instances in an image or video region, and it has wide applications across various industries. Fully supervised target counting methods rely on bounding box or pixel-level mask annotations, enabling models to learn the precise contours and physical scale of targets and establish robust feature maps. However, the annotation cost of such methods is extremely high, making it difficult to apply them on a large scale in real-world scenarios.

[0003] To reduce annotation costs, weakly supervised counting methods that only provide the total number of objects in an image have become a research hotspot. These methods use only image-level quantity labels as supervision signals, without requiring detailed information such as the object's location coordinates, shape, and size. Existing weakly supervised counting methods mostly employ end-to-end global regression strategies, fitting a nonlinear mapping relationship between image features and the total number of objects through deep neural networks, and enhancing feature aggregation capabilities with attention mechanisms and multi-scale modules.

[0004] However, existing weakly supervised counting methods have the following drawbacks, making it difficult to meet the practical needs of dense target counting: 1. Ambiguity in Feature Localization: When establishing the mapping between image features and quantity, it is difficult to accurately distinguish foreground targets from high-frequency background noise in space. In order to fit the total quantity, the model often mistakenly activates background regions with similar textures as counting targets, or ignores low-contrast real targets. This false alarm in the background leads to highly unstable counting results.

[0005] 2. Scale perception is blind: In the absence of target size information, existing models mostly use a single scale receptive field to extract features, which cannot adapt to changes in the physical size of the target. When facing large-scale targets, they are prone to misjudging local texture details as multiple independent small targets, resulting in repeated counting. When facing small and dense targets, they are smoothed out and ignored because the receptive field is too large. Summary of the Invention

[0006] The purpose of this invention is to provide a weakly supervised counting method based on frequency domain guidance and hole scanning, so as to solve the problems of ambiguous feature localization and blind scale perception when using the current weakly supervised counting method to calculate the number of specific target instances in an image or video region.

[0007] The technical solution of this invention is: A weakly supervised counting method based on frequency domain guidance and hole scanning includes the following steps: The image features of the two-dimensional image to be counted are obtained, and the image features are flattened along the spatial dimension and mapped into a one-dimensional serialized feature vector. Type embedding and position embedding are introduced to form the input stream of the Mamba model. The image features are mapped to the frequency domain by a two-dimensional fast Fourier transform, and the background frequency components of the image features are filtered out by the periodic resonance features of the target object in the two-dimensional image to be counted, so as to obtain the processed frequency domain features. The processed frequency domain features are restored by a two-dimensional fast Fourier inverse transform to extract the texture distribution information of the target object and obtain the guidance signal in the spatial domain. The guidance signal is mapped to a stride bias through convolution and injected into the input stream of the Mamba model. After normalization, an adaptive feature sequence is obtained. The adaptive feature sequence is rearranged and split into multiple parallel sub-sequences according to different expansion rates. Each sub-sequence is captured by a multi-scale feature capture through an independent Mamba scanning unit, and then the sub-sequences are merged and projected to obtain the fused multi-scale features. The fused multi-scale features are globally integrated and mapped to a single-channel density map using a regression head. The number of predicted targets is obtained by integrating the entire single-channel density map.

[0008] Preferably, as a further improvement of the present invention, the process of obtaining the fused multi-scale features based on the adaptive feature sequence is as follows: For a given expansion rate Input adaptive feature sequence Reshaped into Parallel subsequences: , Each subsequence is independently modeled using the following selective state-space equation: , , In the formula, For the first Path sequence, It is in a hidden state. It is the discretized state transition matrix. It is a discretized input matrix. It is the output matrix. It is a moment t The output response; Output the sequence generated by each subsequence through the selective state-space equation. The original order is restored through reverse reshaping and splicing to form branch features corresponding to different expansion rates; The features from multiple branches are cascaded and fused to obtain global multi-scale features. .

[0009] Preferably, as a further improvement of the present invention, the multi-parallel hole selective scanning mechanism is set with different expansion rates, corresponding to feature capture at three scales: fine-grained, medium-grained, and coarse-grained.

[0010] Preferably, as a further improvement of the present invention, after filtering out the background frequency components of the image features, the periodic signal is enhanced by linear layer processing.

[0011] Preferably, as a further improvement of the present invention, during training, the image-level true total number of targets is used as a supervision signal, and an L1 loss function constraint is applied to the full-image integral result of the predicted single-channel density map to achieve weakly supervised regression of the number of targets.

[0012] Preferably, as a further improvement of the present invention, the applied L1 loss function is as follows: , In the formula, For L1 loss function, For predicted values, This is the actual value.

[0013] Preferably, as a further improvement of the present invention, the convolutional neural network is a VGG16 convolutional neural network.

[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. By introducing a frequency domain guidance mechanism, the unique periodic resonance characteristics of the foreground target in the spectrum are used as implicit position guidance to directly control the scanning step size of the state space model. This automatically suppresses non-periodic background interference and locks the target area without position labeling. Furthermore, frequency domain guidance can effectively identify high-frequency boundary signals between objects, resulting in finer segmentation / counting granularity and preventing a bunch of objects from being counted as one. This solves the problem of false detection and missed detection caused by the similarity between the target and the background texture.

[0015] 2. The frequency domain module captures the repetitive essential features of the target, without relying on the texture or shape information of a specific target category. The model can be directly applied to various scenarios such as agriculture, industry, biomedicine, and traffic monitoring without modifying the network structure, thus achieving general target counting.

[0016] 3. The multi-parallel cavity selective scanning mechanism constructs a multi-scale receptive field of micro, meso and macro, which can count large-scale targets as a whole, while accurately capturing small and dense targets, thus solving the problems of repeated counting and ignoring small targets caused by a single-scale receptive field.

[0017] 4. By combining frequency domain guidance with hole scanning, high and low frequency features are complemented, enabling the model to locate target clusters in a large field of view and accurately distinguish individual small targets, thus ensuring counting accuracy. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the steps of a weakly supervised counting method based on frequency domain guidance and hole scanning according to the present invention.

[0019] Figure 2 This is a schematic diagram of the network structure of the weakly supervised counting method based on frequency domain guidance and hole scanning of the present invention.

[0020] Figure 3 This is a schematic diagram illustrating an application example of the weakly supervised counting method based on frequency domain guidance and hole scanning according to the present invention. Detailed Implementation

[0021] The following is combined Figures 1-3 The specific embodiments of the present invention will be described in detail below. In the description of the invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0022] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; in the description of the invention, unless otherwise stated, "a plurality of" means two or more.

[0023] Example like Figures 1-2 As shown, this embodiment of the invention provides a weakly supervised counting method based on frequency domain guidance and hole scanning, including the following steps: Feature encoding stage: S11. Obtain the two-dimensional image to be counted, and extract the image features of the two-dimensional image to be counted through a convolutional neural network.

[0024] Specifically, the two-dimensional image to be counted is input into the front end of the convolutional neural network to extract the image features of the two-dimensional image to be counted. The convolutional neural network is a VGG16 convolutional neural network, and the image feature dimension is [H×W×C], where H and W are the height and width of the image, respectively, and C is the number of feature channels.

[0025] S12. Flatten the image features along the spatial dimension and map them into a one-dimensional serialized feature vector. Introduce type embedding and position embedding to form the input stream of the Mamba model.

[0026] Specifically, the Tokenization module flattens image features along the spatial dimension, flattening the [H×W×C]-dimensional spatial features along the spatial dimension (H×W dimension) and mapping them into a one-dimensional serialized feature vector. At the same time, type embedding and position embedding are introduced. Since the original topological structure and category attributes are lost after flattening two-dimensional image features into one-dimensional feature vectors, position embedding is introduced here to explicitly supplement the two-dimensional spatial coordinates of pixels for one-dimensional sequences. Type embedding is used to give the features basic semantic attributes. Both directly serve the serialization scan of Mamba and together with the one-dimensional serialized feature vector constitute the input stream of the Mamba model, providing the basic spatiotemporal and semantic context for subsequent state space modeling.

[0027] Frequency domain guidance stage: S2. The image features are mapped to the frequency domain space by two-dimensional fast Fourier transform, so as to filter out the background frequency components of the image features by utilizing the periodic resonance features of the target object in the two-dimensional image to be counted, and obtain the processed frequency domain features. The processed guidance signal is restored by two-dimensional fast Fourier inverse transform to extract the texture distribution information of the target object and obtain the guidance signal in the spatial domain.

[0028] Specifically, to distinguish foreground targets from background noise under weak supervision conditions lacking location annotations, this invention innovatively designs a frequency-domain guidance branch. This branch maps image features to the frequency domain using a two-dimensional fast Fourier transform (FFT). Utilizing the periodic resonance characteristics of the foreground target in the frequency spectrum, it suppresses frequency components in the two-dimensional image to be counted that do not match the target features in the frequency domain, thereby filtering out disordered background frequency components. The filtered frequency-domain features are then processed by a linear layer to amplify specific frequency components of the target characteristics to enhance the periodic signal. Finally, a two-dimensional inverse FFT restores the frequency-domain features to spatial domain features, which constitute the guidance signal. This guidance signal implicitly contains the target's texture distribution information, acting as a soft attention mask and providing precise guidance for the subsequent generation of the step size bias.

[0029] Densely distributed target objects constitute a quasi-periodic signal in the spatial domain. After a two-dimensional fast Fourier transform, significant energy enrichment occurs at specific frequency points in the spectrogram. In this embodiment, a learnable Spectral Attention Mask is constructed, and the spectrogram is input into a Process unit containing a 1x1 convolutional structure. Based on the distribution pattern of the amplitude spectrum, this unit automatically identifies and retains high-energy frequency components representing the periodic texture of the target, while suppressing the amplitude of background frequency components (such as imaging noise and non-target occlusion textures) that do not have periodic patterns and exhibit random distribution characteristics. Thus, when the inverse transform is used to restore the spatial domain signal, only the structural features of the target are retained as guidance signals to guide the Mamba scan.

[0030] The mathematical description of frequency domain feature extraction and guidance signal generation is as follows: Let the input features be Where ℝ represents the real number space, H represents the height of the image feature, W represents the width of the image feature, and C represents the number of feature channels; The frequency domain branch is first subjected to a two-dimensional real Fourier transform as follows: ; The periodic signal is enhanced by linear layer processing in the frequency domain, followed by an inverse transform as shown in the following equation to obtain the guidance signal: ; In the formula, For the guidance signal, the spatial dimension is preserved, but the foreground region with repeating texture is highlighted numerically.

[0031] Void scanning modeling stage: S31. The guidance signal in the spatial domain is mapped to a stride bias through convolution and injected into the input stream of the Mamba model. After normalization, an adaptive feature sequence is obtained.

[0032] Specifically, in this stage, a frequency-domain controlled hole scanning module was designed as the core processing unit. First, using the aforementioned frequency-domain guidance signal, a stride bias is generated through a 1×1 lightweight convolutional controller. This bias is then directly injected into the input features of the Mamba model, thereby dynamically intervening in the state transition process of the Mamba model, increasing the information throughput in the target region and suppressing information flow in the background region.

[0033] The generation of the step size bias and the adaptive feature sequence satisfy the following mathematical relationship: ; ; In the formula, For guidance signals, For step size offset, For the input stream of the Mamba model, It is an adaptive feature sequence.

[0034] S32. The spatial domain guidance signal is mapped to a stride bias through convolution and injected into the input stream of the Mamba model. After normalization, an adaptive feature sequence is obtained. A multi-parallel dilatation-selective scanning mechanism is introduced, and different dilation rates are set. The adaptive feature sequence is rearranged and split according to different dilation rates to form multiple parallel sub-sequences. Each sub-sequence is captured by an independent Mamba scanning unit for multi-scale feature capture. The features output by each Mamba scanning unit are merged and projected to obtain the fused multi-scale features. Specifically, to accommodate targets of varying sizes, this invention introduces a multi-path parallel hole-selective scanning mechanism. Long sequences are rearranged and split according to different expansion rates, forming multiple parallel short subsequences. These subsequences are then fed into independent Mamba scanning units, enabling the model to capture features at micro, meso, and macro scales. Finally, the multi-scale features are merged and projected.

[0035] For a given expansion rate Input sequence Reshaped into Parallel subsequences: , By increasing The model can obtain a larger receptive field through short sequence scans without increasing computational complexity; Each subsequence is independently modeled using the following selective state-space equation: , , In the formula, For the first Path sequence, It is in a hidden state. It is the discretized state transition matrix. It is a discretized input matrix. It is the output matrix. It is a moment t The output response; Output the sequence generated by each subsequence through the selective state-space equation. The original order is restored through reverse reshaping and splicing to form branch features corresponding to different expansion rates; By concatenating and fusing branch features at different scales, a global multi-scale feature is obtained. This enables leapfrog capture of long-distance dependent data.

[0036] Among them, the multi-path parallel hole selective scanning mechanism is set with expansion rates of 1, 2 and 4, which correspond to feature capture at three scales: fine-grained, medium-grained and coarse-grained, respectively.

[0037] Density regression phase: S4. The fused multi-scale features are integrated by the global mixing module and mapped into a single-channel density map through the regression head. The number of predicted targets is obtained by integrating the entire single-channel density map.

[0038] Specifically, the fused multi-scale features are further integrated with contextual information through a global mixing module, and then the high-dimensional features are mapped into a single-channel density map using a regression head.

[0039] During training, the model uses only the total number of image-level labels as supervision signals and applies an L1 loss function to constrain the integral results of the predicted density map, forcing the model to learn the distribution pattern of the target autonomously.

[0040] The applied L1 loss function is shown in the following equation: , In the formula, Let L1 loss function be used. For predicted values, This is the true value; the loss function only constrains the total number of targets and does not involve the target's location coordinates. The feature map output by the regression head is a single-channel density map. The integral over the entire graph is the predicted value: .

[0041] In specific implementation, such as Figure 3 As shown, this invention uses crowd counting as an example for demonstration. The method divides the image into 3*4 sub-images for visualization. It can be seen that the model can make accurate predictions for both background and foreground heads, and also shows good prediction results when the background is mixed with dense crowds. In addition, the method can be applied to various scenarios such as agriculture, industry, and medicine, for example, ventricular bubble counting.

[0042] The above-disclosed embodiments are merely preferred embodiments of the present invention. However, the embodiments of the present invention are not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A weakly supervised counting method based on frequency domain guidance and hole scanning, characterized in that, Includes the following steps: A two-dimensional image to be counted is acquired, and its image features are extracted using a convolutional neural network. The image features are flattened along the spatial dimension and mapped into a one-dimensional serialized feature vector. Type embedding and position embedding are introduced to form the input stream of the Mamba model. The image features are mapped to the frequency domain by a two-dimensional fast Fourier transform, and the background frequency components of the image features are filtered out by the periodic resonance features of the target object in the two-dimensional image to be counted, so as to obtain the processed frequency domain features. The processed frequency domain features are restored by a two-dimensional fast Fourier inverse transform to extract the texture distribution information of the target object and obtain the guidance signal in the spatial domain. The guidance signal is mapped to a stride bias through convolution and injected into the input stream of the Mamba model. After normalization, an adaptive feature sequence is obtained. The adaptive feature sequence is rearranged and split into multiple parallel sub-sequences according to different expansion rates. Each sub-sequence is captured by an independent Mamba scanning unit, and then the sub-sequences are merged and projected to obtain the fused multi-scale features. The fused multi-scale features are globally integrated and mapped to a single-channel density map using a regression head. The number of predicted targets is obtained by integrating the entire single-channel density map.

2. The weakly supervised counting method based on frequency domain guidance and hole scanning according to claim 1, characterized in that, The process of obtaining the fused multi-scale features based on adaptive feature sequences is as follows: For a given expansion rate Input adaptive feature sequence Reshaped into Parallel subsequences: ， Each subsequence is independently modeled using the following selective state-space equation: ， ， In the formula, For the first Path sequence, It is in a hidden state. It is the discretized state transition matrix. It is a discretized input matrix. It is the output matrix. It is a moment t The output response; Output the sequence generated by each subsequence through the selective state-space equation. The original order is restored through reverse reshaping and splicing to form branch features corresponding to different expansion rates; The features from multiple branches are cascaded and fused to obtain global multi-scale features. .

3. The weakly supervised counting method based on frequency domain guidance and hole scanning according to claim 2, characterized in that, The multi-path parallel hole selective scanning mechanism has different expansion rates, corresponding to feature capture at three scales: fine-grained, medium-grained, and coarse-grained.

4. The weakly supervised counting method based on frequency domain guidance and hole scanning according to claim 1, characterized in that, After filtering out the background frequency components of the image features, a linear layer is used to enhance the periodic signal.

5. The weakly supervised counting method based on frequency domain guidance and hole scanning according to claim 1, characterized in that, During training, the image-level true total number of targets is used as a supervision signal, and the L1 loss function is applied to the full-image integral result of the predicted single-channel density map to achieve weakly supervised regression of the number of targets.

6. The weakly supervised counting method based on frequency domain guidance and hole scanning according to claim 5, characterized in that, The applied L1 loss function is shown in the following equation: ， In the formula, For L1 loss function, For predicted values, This is the actual value.

7. The weakly supervised counting method based on frequency domain guidance and hole scanning according to claim 1, characterized in that, The convolutional neural network is a VGG16 convolutional neural network.

Images