A target counting method and system based on image shape embedding

Abstract

The application discloses a target counting method and system based on image shape embedding, and relates to the technical field of image processing.The method comprises the following steps: obtaining an image to be counted and an example sample image; preprocessing the image to be counted and the example sample image; performing image segmentation processing on the preprocessed image to be counted and the preprocessed example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image; inputting the first image block sequence and the second image block sequence into a counting model to perform target counting and obtain a target counting result; and the counting model adopts a visual transformer based on a class-independent counting framework, and the visual transformer comprises a self-similarity calculation module, a shape information embedding module and a density map regression module.The application realizes a class-independent counting task and significantly improves the accuracy and generalization ability of the counting task.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a target counting method and system based on image shape embedding. Background Technology

[0002] Counting objects in an image is a key task in the field of computer vision, with wide applications in many areas. However, when a scene contains densely packed objects, even human vision struggles to provide accurate estimates.

[0003] Traditional object counting methods typically employ supervised learning in specific domains. However, these methods require extensive manual annotation and are often only effective on the specific datasets used for training. Retraining is necessary when the scenario changes, making it a time-consuming and laborious process, lacking cross-scenario versatility and application flexibility. To address this issue, Tichu proposed a weakly supervised few-shot object counting method. It introduces a similarity comparison module to identify and count target objects using a small number of query image examples provided by the user. However, the similarity map obtained by this method is less information-rich than the feature map, and regression analysis relying solely on the similarity map struggles to distinguish the boundaries of densely packed objects.

[0004] To address these challenges, class-independent counting methods have been proposed in recent years. These methods require only a small number of image examples containing the target object during model training to accurately estimate the frequency of the target's appearance in an image. This type of model can generalize and identify unlabeled targets within the same category, even if these targets differ in shape, size, or color from the examples seen during training. The advantages of this approach are that it reduces the dependence on the amount of training data, makes it easier to generalize to other scenarios, and significantly reduces the additional work required to apply the model to different scenarios. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a target counting method and system based on image shape embedding, which realizes category-independent counting tasks and significantly improves the accuracy and generalization ability of counting tasks.

[0006] To address the aforementioned technical problems, this invention provides a target counting method based on image shape embedding, the method comprising:

[0007] Obtain the image to be counted and the example sample image;

[0008] The images to be counted and the example sample images are preprocessed to obtain preprocessed images to be counted and example sample images.

[0009] Image segmentation processing is performed on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image;

[0010] The first image patch sequence and the second image patch sequence are input into the counting model to perform target counting and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module, and a density map regression module.

[0011] Optionally, the step of performing image segmentation processing on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image includes:

[0012] The preprocessed images to be counted and the example images are subjected to size unification processing to obtain the size-unified images to be counted and the example images.

[0013] The image to be counted and the example sample image after size unification are divided into image blocks to obtain several first image blocks corresponding to the image to be counted after size unification and several second image blocks corresponding to the example sample image after size unification.

[0014] Linear mapping is performed on several first image blocks and several second image blocks respectively to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image.

[0015] Optionally, the self-similarity calculation module includes a multi-head self-attention module, which generates a high-dimensional feature representation with self-similarity information based on the first image patch sequence and the second image patch sequence.

[0016] Optionally, the expression for the high-dimensional feature representation is:

[0017]

[0018] Among them, D m For high-dimensional feature representation, Vit is the visual transformer, and CrossAtten is the joint-re-cross mechanism. The query vector is generated from the first image patch sequence. The key vector is generated from the second image patch sequence. D is the value vector generated from the second image patch sequence. x For the first image patch sequence, D y This is the second image block sequence.

[0019] Optionally, the shape information embedding module calculates the width and height of each annotation box and generates a two-dimensional matrix based on the width and height of the annotation boxes;

[0020] The two-dimensional matrix is ​​mapped to a shape feature with the same dimension as the high-dimensional feature representation based on a linear mapping.

[0021] The shape features are enhanced by shape information embedding to obtain the enhanced shape features.

[0022] Optionally, the expression for the shape feature is:

[0023] D s =f shape )linear([w i ,h i ])),

[0024] Among them, D s For shape features, f shape For shape extraction, `linear` is the linear mapping layer, and `w` is the shape extractor. i h is the width of the annotation box. i [w] is the height of the annotation box. i ,h i ] is a two-dimensional matrix.

[0025] Optionally, the expression for the enhanced shape feature embedded with the shape information is:

[0026]

[0027] Among them, D a The enhanced shape features are embedded into the shape information. Vit is a visual transformer, and CrossAtten is a joint-re-cross mechanism. The query input is generated from a high-dimensional feature representation. For key inputs generated from shape features, Input the value generated from the shape features.

[0028] Optionally, the density map regression module includes a density regression layer and several upsampling layers;

[0029] The upsampling layer performs convolution and bilinear interpolation on the enhanced shape features embedded with shape information to obtain an upsampling feature map.

[0030] The density map regression layer converts the upsampled feature map into a single-channel density heatmap, calculates the cumulative value of all pixel values ​​in the single-channel density heatmap, and determines the target counting result based on the cumulative value.

[0031] Optionally, the expression for the single-channel density heatmap is:

[0032] T = UpSample(Conv(D) a )) k k∈{1,2,3,4}

[0033] Where T is the single-channel density heatmap, UpSample is the upsampling stage, Conv is the convolution stage, and D is the density heatmap. a The enhanced shape features are embedded into the shape information, and k is the index of the upsampling stage and the convolution stage;

[0034] The expression for the accumulated value is:

[0035] TCount = ∑T,

[0036] Where TCount is the accumulated value and T is the single-channel density heatmap.

[0037] In addition, the present invention also provides a target counting system based on image shape embedding, the system comprising:

[0038] Image acquisition module: used to acquire the image to be counted and example sample images;

[0039] Image preprocessing module: used to preprocess the image to be counted and the example sample image to obtain the preprocessed image to be counted and the example sample image;

[0040] Image segmentation module: used to perform image segmentation processing on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image;

[0041] Target counting module: used to input the first image block sequence and the second image block sequence into the counting model to count targets and obtain target counting results. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module and a density map regression module.

[0042] In this embodiment of the invention, a first image patch sequence corresponding to the image to be counted and a second image patch sequence corresponding to the example sample image are input into a counting model to perform target counting and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module, and a density map regression module. The self-similarity calculation module enhances the model's ability to capture the internal structure of the image and strengthens the similarity features between the image to be counted and the example sample image. The shape information embedding module enhances the model's sensitivity to the shape and boundary information of the target object, thereby improving the recognition ability in scenes with dense objects or partial occlusion, enabling the model to not only effectively handle object categories encountered during training but also better generalize to new and unseen categories. The density map regression module improves the accuracy of the counting task. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the target counting method based on image shape embedding in an embodiment of the present invention;

[0045] Figure 2 This is a flowchart illustrating a target counting method based on image shape embedding in another embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the structural composition of the target counting system based on image shape embedding in an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Example 1

[0049] Please see Figure 1 , Figure 1 This is a flowchart illustrating a target counting method based on image shape embedding in an embodiment of the present invention. The method includes:

[0050] S11: Obtain the image to be counted and the example sample image;

[0051] In the specific implementation of this invention, the example sample image contains a bounding box, which can be set to one to three. At the same time, the number of example sample images is not too large; three images are sufficient to assist in completing the image counting process.

[0052] S12: Preprocess the image to be counted and the example sample image to obtain the preprocessed image to be counted and the example sample image;

[0053] In the specific implementation of this invention, the image to be counted and the example sample image are converted to grayscale. The pixel values ​​of the three primary colors in the image to be counted and the example sample image are calculated. The average value of the pixel values ​​of the three primary colors is calculated. A weighted average value is generated by combining the sensitivity of the human eye to the three primary colors with the average value. The image to be counted and the example sample image are then processed based on the weighted average value to obtain the grayscale image to be counted and the grayscale example sample image. The grayscale image to be counted and the grayscale example sample image are then denoised. The grayscale image to be counted and the grayscale example sample image are filtered according to the set filter values. The average grayscale value of adjacent pixels in the grayscale image to be counted and the grayscale example sample image are calculated. The grayscale image to be counted and the grayscale example sample image are then denoised based on the average grayscale value to obtain the denoised grayscale image to be counted and the grayscale example sample image. Image enhancement processing is performed on the denoised grayscale image to be counted and the grayscale example image. The histogram distribution of the denoised grayscale image to be counted and the grayscale example image is changed to make the image details clearer and improve the readability of the image. After the image enhancement processing is completed, the preprocessing is completed, and the preprocessed image to be counted and the example image are obtained.

[0054] S13: Perform image segmentation processing on the preprocessed image to be counted and the example sample image to obtain the first image block sequence corresponding to the preprocessed image to be counted and the second image block sequence corresponding to the preprocessed example sample image;

[0055] In a specific implementation of this invention, the step of performing image segmentation processing on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image includes: performing size unification processing on the preprocessed image to be counted and the example sample image to obtain size-unified image to be counted and example sample image; dividing the size-unified image to be counted and the example sample image into image blocks to obtain a plurality of first image blocks corresponding to the size-unified image to be counted and a plurality of second image blocks corresponding to the size-unified example sample image; and performing linear mapping on the plurality of first image blocks and the plurality of second image blocks respectively to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image.

[0056] Specifically, the preprocessed images to be counted and the example images are subjected to size unification processing, that is, the preprocessed images to be counted and the example images are adjusted to a uniform preset fixed size, resulting in size-unified images to be counted and example images. The size-unified images to be counted and the example images are then divided into image blocks, into several 16*16 pixel image blocks, resulting in several first image blocks corresponding to the size-unified images to be counted and several second image blocks corresponding to the size-unified images of the example images. Linear mapping is then performed on the first image blocks and the second image blocks respectively, that is, each image block is converted into a one-dimensional vector through linear mapping, and then these are combined into corresponding sequences, resulting in a first image block sequence corresponding to the preprocessed images to be counted and a second image block sequence corresponding to the preprocessed example images. Dividing the image into several image blocks and forming a sequence is to adapt to the input requirements of the visual transformer.

[0057] S14: Input the first image block sequence and the second image block sequence into the counting model to perform target counting and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module, and a density map regression module.

[0058] In a specific implementation of the present invention, the self-similarity calculation module includes a multi-head self-attention module, which generates a high-dimensional feature representation with self-similarity information based on the first image block sequence and the second image block sequence.

[0059] Furthermore, the expression for the high-dimensional feature representation is:

[0060]

[0061] Among them, D m For high-dimensional feature representation, Vit is the visual transformer, and CrossAtten is the joint-re-cross mechanism. The query vector is generated from the first image patch sequence. The key vector is generated from the second image patch sequence. D is the value vector generated from the second image patch sequence. x For the first image patch sequence, D y This is the second image block sequence.

[0062] Furthermore, the shape information embedding module calculates the width and height of each bounding box and generates a two-dimensional matrix based on the width and height of the bounding boxes; it maps the two-dimensional matrix to a shape feature with the same dimension as the high-dimensional feature representation based on a linear mapping; and it performs shape information embedding enhancement on the shape feature to obtain the shape feature after shape information embedding enhancement.

[0063] Furthermore, the expression for the shape feature is:

[0064] D s =f shape (linear([w i ,h i ])),

[0065] Among them, D s For shape features, f shape For shape extraction, `linear` is the linear mapping layer, and `w` is the shape extractor. i h is the width of the annotation box. i [w] is the height of the annotation box. i ,h i ] is a two-dimensional matrix.

[0066] Furthermore, the expression for the enhanced shape feature embedded with the shape information is as follows:

[0067]

[0068] Among them, D a The enhanced shape features are embedded into the shape information. Vit is a visual transformer, and CrossAtten is a joint-re-cross mechanism. The query input is generated from a high-dimensional feature representation. For key inputs generated from shape features, Input the value generated from the shape features.

[0069] Furthermore, the density map regression module includes a density regression layer and several upsampling layers; the upsampling layer performs convolution and bilinear interpolation on the enhanced shape features embedded with shape information to obtain an upsampled feature map; the density map regression layer converts the upsampled feature map into a single-channel density heatmap, calculates the cumulative value of all pixel values ​​in the single-channel density heatmap, and determines the target counting result based on the cumulative value.

[0070] Furthermore, the expression for the single-channel density heatmap is:

[0071] T = UpSample(Conv(D) a )) k k∈{1,2,3,4}

[0072] Where T is the single-channel density heatmap, UpSample is the upsampling stage, Conv is the convolution stage, and D is the density heatmap. a The enhanced shape features are embedded into the shape information, and k is the index of the upsampling stage and the convolution stage;

[0073] The expression for the accumulated value is:

[0074] TCount = ∑T,

[0075] Where TCount is the accumulated value and T is the single-channel density heatmap.

[0076] Specifically, the first and second image patch sequences are input into a counting model for target counting to obtain the target counting result. The counting model employs a visual transformer based on a class-independent counting framework, which includes a self-similarity calculation module, a shape information embedding module, and a density map regression module. The self-similarity calculation module includes a multi-head self-attention module with 12 layers of encoder stacked to fully extract and fuse the contextual information of the image. The multi-head self-attention module generates a high-dimensional feature representation with self-similarity information based on the first and second image patch sequences. The second image patch sequence generated from the example sample image is input into the self-similarity calculation module as the key vector and value vector, and the first image patch sequence generated from the image to be counted is input into the self-similarity calculation module as the query vector to form a high-dimensional feature map, which is the high-dimensional feature representation. The self-similarity calculation module not only enhances the model's ability to capture the internal structure of the image but also strengthens the similarity features between the image to be counted and the example sample image through cross-comparison of the self-attention mechanism. After multi-layer self-attention calculation, the resulting high-dimensional feature representation contains rich self-similarity information. By cross-processing the images to be counted and the example sample images, the model's sensitivity to object shape and boundary information can be enhanced, while capturing the correlation between the images and the samples more effectively.

[0077] The expression for the high-dimensional feature representation is:

[0078]

[0079] Among them, D m For high-dimensional feature representation, Vit is the visual transformer, and CrossAtten is the joint-re-cross mechanism. The query vector is generated from the first image patch sequence. The key vector is generated from the second image patch sequence. D is the value vector generated from the second image patch sequence. x For the first image patch sequence, D y This is the second image block sequence.

[0080] The shape information embedding module calculates the width and height of each bounding box. Each bounding box contains positional coordinate information, and its width and height can be calculated based on the positional coordinate system. A two-dimensional matrix is ​​then generated based on the width and height of the bounding boxes. This two-dimensional matrix is ​​mapped to shape features with the same dimensions as the high-dimensional feature representation using a linear mapping. A three-layer linear mapping process is employed, with the dimension initially changing from 2 to 64 dimensions. In the final layer, the spatial size is adjusted to the product of the number of bounding boxes and the embedded dimensions. Shape information embedding enhancement is performed on the shape features, using them as key and value inputs, and the high-dimensional feature representation as query input. The shape linear embedding module considers both global features and local shape information when calculating attention weights, resulting in shape features enhanced by shape information embedding. The shape linear embedding module extracts shape features by analyzing the coordinates of the bounding boxes and integrates these features into the model's feature representation. This integration strategy enhances the model's sensitivity to the shape and boundary information of the target object, thereby improving recognition capabilities in scenes with dense objects or partial occlusion. Meanwhile, the shape information embedding module, by introducing shape information, enables the model not only to effectively handle object categories encountered during training but also to better generalize to new and unseen categories. This is because shape features are highly discriminative between different object categories. By learning these features, the model can adapt to a wider range of counting scenarios, improving its robustness and generalization ability in practical applications.

[0081] The expression for the shape feature is:

[0082] D s =f shape (linear([w i ,h i ])),

[0083] Among them, D sFor shape features, f shape For shape extraction, `linear` is the linear mapping layer, and `w` is the shape extractor. i h is the width of the annotation box. i [w] is the height of the annotation box. i ,h i ] is a two-dimensional matrix, and the shape extractor is used to extract shape features from the width and height of the annotation box.

[0084] The expression for the enhanced shape feature embedded with the shape information is:

[0085]

[0086] Among them, D a The enhanced shape features are embedded into the shape information. Vit is a visual transformer, and CrossAtten is a joint-re-cross mechanism. The query input is generated from a high-dimensional feature representation. For key inputs generated from shape features, Input the value generated from the shape features.

[0087] The density map regression module includes a density regression layer and several upsampling layers. The density map regression module converts the enhanced shape features embedded with shape information into a feature representation that matches the resolution of the original input image. The upsampling layers perform convolution and bilinear interpolation on the enhanced shape features embedded with shape information. To gradually improve the resolution of the enhanced shape features until they reach the same scale as the original input image, a staged upsampling strategy is adopted, setting up four upsampling layers. Each upsampling layer contains a series of convolution and bilinear interpolation processes to achieve a two-fold increase in spatial resolution. The purpose of the convolution operation is to extract and refine key information in the feature map, while the purpose of the bilinear interpolation process is to expand the spatial dimension of the feature map while maintaining feature coherence, thereby obtaining an upsampled feature map. The density map regression layer converts the upsampled feature map into a single-channel density heatmap. The density regression layer includes a 1*1 convolution kernel and calculates the cumulative value of all pixel values ​​in the single-channel density heatmap. Each pixel value in the single-channel density heatmap represents the probability density of the object's occurrence. By accumulating these values, the total number of objects in the image can be accurately estimated, achieving efficient visual object counting, i.e., determining the target counting result based on the accumulated value.

[0088] The expression for the single-channel density heatmap is:

[0089] T = UpSample(Conv(D) a )) k k∈{1,2,3,4}

[0090] Where T is the single-channel density heatmap, UpSample is the upsampling stage, Conv is the convolution stage, and D is the density heatmap. a The enhanced shape features are embedded into the shape information, and k is the index of the upsampling stage and the convolution stage;

[0091] The expression for the accumulated value is:

[0092] TCount = ∑T,

[0093] Where TCount is the accumulated value and T is the single-channel density heatmap.

[0094] To verify the performance advantages of the counting model in this invention, the FSC*147 dataset was used to evaluate the performance of different models, examining their performance on a few-sample object counting task, especially their effectiveness and generalization ability. The mean absolute error (MAE) and root mean square error (RMSE) of the counting results obtained by each model in the counting task were used as the measure.

[0095]

[0096] p i Let g represent the prediction count for the i-th image. i denoted as the true count of the i-th image, and n represents the total number of images participating in the evaluation. By comparing the predicted counts and true counts of all images, the overall error index can be calculated.

[0097] The performance comparison table of each model is as follows:

[0098]

[0099] As can be seen from the table above, in the performance comparison of various models, the visual transformer based on the class-independent counting framework of this invention is used as the counting model, which shows a more outstanding performance advantage.

[0100] In this embodiment of the invention, a first image patch sequence corresponding to the image to be counted and a second image patch sequence corresponding to the example sample image are input into a counting model to perform target counting and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module, and a density map regression module. The self-similarity calculation module enhances the model's ability to capture the internal structure of the image and strengthens the similarity features between the image to be counted and the example sample image. The shape information embedding module enhances the model's sensitivity to the shape and boundary information of the target object, thereby improving the recognition ability in scenes with dense objects or partial occlusion, enabling the model to not only effectively handle object categories encountered during training but also better generalize to new and unseen categories. The density map regression module improves the accuracy of the counting task.

[0101] Example 2

[0102] Please see Figure 2 , Figure 2 This is a flowchart illustrating a target counting method based on image shape embedding according to another embodiment of the present invention, the method comprising:

[0103] S201: Obtain the image to be counted and the example sample image;

[0104] In the specific implementation of this invention, the example sample image contains a bounding box, which can be set to one to three. At the same time, the number of example sample images is not too large; three images are sufficient to assist in completing the image counting process.

[0105] S202: Preprocess the image to be counted and the example sample image to obtain the preprocessed image to be counted and the example sample image;

[0106] In the specific implementation of this invention, the image to be counted and the example sample image are converted to grayscale. The pixel values ​​of the three primary colors in the image to be counted and the example sample image are calculated. The average value of the pixel values ​​of the three primary colors is calculated. A weighted average value is generated by combining the sensitivity of the human eye to the three primary colors with the average value. The image to be counted and the example sample image are then processed based on the weighted average value to obtain the grayscale image to be counted and the grayscale example sample image. The grayscale image to be counted and the grayscale example sample image are then denoised. The grayscale image to be counted and the grayscale example sample image are filtered according to the set filter values. The average grayscale value of adjacent pixels in the grayscale image to be counted and the grayscale example sample image are calculated. The grayscale image to be counted and the grayscale example sample image are then denoised based on the average grayscale value to obtain the denoised grayscale image to be counted and the grayscale example sample image. Image enhancement processing is performed on the denoised grayscale image to be counted and the grayscale example image. The histogram distribution of the denoised grayscale image to be counted and the grayscale example image is changed to make the image details clearer and improve the readability of the image. After the image enhancement processing is completed, the preprocessing is completed, and the preprocessed image to be counted and the example image are obtained.

[0107] S203: Perform image segmentation processing on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image;

[0108] In the specific implementation of this invention, the preprocessed images to be counted and the example sample images are subjected to size unification processing, that is, the preprocessed images to be counted and the example sample images are adjusted to a uniform preset fixed size, resulting in size-unified images to be counted and example sample images. The size-unified images to be counted and the example sample images are then divided into image blocks, into several 16*16 pixel image blocks, resulting in several first image blocks corresponding to the size-unified images to be counted and several second image blocks corresponding to the size-unified images of the example sample images. Linear mapping is then performed on the several first image blocks and several second image blocks, that is, each image block is converted into a one-dimensional vector through linear mapping, and then they are combined into corresponding sequences, resulting in a first image block sequence corresponding to the preprocessed images to be counted and a second image block sequence corresponding to the preprocessed example sample images. Dividing the image into several image blocks and forming a sequence is to adapt to the input requirements of the visual transformer.

[0109] S204: Input the first image patch sequence and the second image patch sequence into the counting model, and the self-similarity calculation module in the counting model generates a high-dimensional feature representation based on the first image patch sequence and the second image patch sequence;

[0110] In the specific implementation of this invention, the self-similarity calculation module includes a multi-head self-attention module with 12 layers of encoder stacked to fully extract and fuse the contextual information of the image. The multi-head self-attention module generates a high-dimensional feature representation with self-similarity information based on the first image patch sequence and the second image patch sequence. The second image patch sequence generated from the example sample image is input as the key vector and value vector into the self-similarity calculation module, and the first image patch sequence generated from the image to be counted is input as the query vector into the self-similarity calculation module to form a high-dimensional feature map, which is the high-dimensional feature representation. The self-similarity calculation module not only enhances the model's ability to capture the internal structure of the image but also strengthens the similarity features between the image to be counted and the example sample image through cross-comparison of the self-attention mechanism. After multi-layer self-attention calculation, the resulting high-dimensional feature representation contains rich self-similarity information. Through cross-processing of the image to be counted and the example sample image, the model's sensitivity to object shape and boundary information can be enhanced, while more effectively capturing the correlation between the image and the sample.

[0111] The expression for the high-dimensional feature representation is:

[0112]

[0113] Among them, D m For high-dimensional feature representation, Vit is the visual transformer, and CrossAtten is the joint-re-cross mechanism. The query vector is generated from the first image patch sequence. The key vector is generated from the second image patch sequence. D is the value vector generated from the second image patch sequence. x For the first image patch sequence, D y This is the second image block sequence.

[0114] S205: The shape information embedding module in the counting model generates shape features based on high-dimensional feature representation, and performs shape information embedding enhancement on the shape features to obtain shape features after shape information embedding enhancement;

[0115] In the specific implementation of this invention, the shape information embedding module calculates the width and height of each bounding box. Each bounding box contains positional coordinate information, and its width and height can be calculated based on the positional coordinate system. A two-dimensional matrix is ​​then generated based on the width and height of the bounding boxes. The two-dimensional matrix is ​​mapped to shape features with the same dimensions as the high-dimensional feature representation using a linear mapping. A three-layer linear mapping process is employed, with the dimension changing from 2 to 64 dimensions initially. In the final layer, the spatial size is adjusted to the product of the number of bounding boxes and the embedded dimensions. Shape information embedding enhancement is performed on the shape features, using them as key and value inputs, and the high-dimensional feature representation as query input. The shape linear embedding module can simultaneously consider global features and local shape information when calculating attention weights, obtaining shape features enhanced by shape information embedding. The shape linear embedding module extracts shape features by analyzing the coordinates of the bounding boxes and integrates these features into the model's feature representation. This integration strategy enhances the model's sensitivity to the shape and boundary information of the target object, thereby improving recognition capabilities in scenes with dense objects or partial occlusion. Meanwhile, the shape information embedding module, by introducing shape information, enables the model not only to effectively handle object categories encountered during training but also to better generalize to new and unseen categories. This is because shape features are highly discriminative between different object categories. By learning these features, the model can adapt to a wider range of counting scenarios, improving its robustness and generalization ability in practical applications.

[0116] The expression for the shape feature is:

[0117] D s =f shape (linear([w i ,h i ])),

[0118] Among them, D s For shape features, f shape For shape extraction, `linear` is the linear mapping layer, and `w` is the shape extractor. i h is the width of the annotation box. i [w] is the height of the annotation box. i ,h i ] is a two-dimensional matrix, and the shape extractor is used to extract shape features from the width and height of the annotation box.

[0119] The expression for the enhanced shape feature embedded with the shape information is:

[0120]

[0121] Among them, D aThe enhanced shape features are embedded into the shape information. Vit is a visual transformer, and CrossAtten is a joint-re-cross mechanism. The query input is generated from a high-dimensional feature representation. For key inputs generated from shape features, Input the value generated from the shape features.

[0122] S206: The density map regression module in the counting model counts targets based on the enhanced shape features embedded with shape information, and obtains the target counting results.

[0123] In the specific implementation of this invention, the density map regression module includes a density regression layer and several upsampling layers. The density map regression module is used to convert the enhanced shape features embedded with shape information into a feature representation that matches the resolution of the original input image. The upsampling layers perform convolution and bilinear interpolation processing on the enhanced shape features embedded with shape information. To gradually improve the resolution of the enhanced shape features embedded with shape information until it reaches the same scale as the original input image, a staged upsampling strategy is adopted, setting up four upsampling layers. Each upsampling layer contains a series of convolution and bilinear interpolation processes to achieve a twofold increase in spatial resolution. The purpose of the convolution operation is to extract and refine the key information in the feature map, while the purpose of the bilinear interpolation process is to expand the spatial dimension of the feature map while maintaining feature coherence, thereby obtaining an upsampled feature map. The density map regression layer converts the upsampled feature map into a single-channel density heatmap. The density regression layer includes a 1*1 convolution kernel and calculates the cumulative value of all pixel values ​​in the single-channel density heatmap. Each pixel value in the single-channel density heatmap represents the probability density of the object's occurrence. By accumulating these values, the total number of objects in the image can be accurately estimated, achieving efficient visual object counting, i.e., determining the target counting result based on the accumulated value.

[0124] The expression for the single-channel density heatmap is:

[0125] T = UpSample(Conv(D) a )) k k∈{1,2,3,4}

[0126] Where T is the single-channel density heatmap, UpSample is the upsampling stage, Conv is the convolution stage, and D is the density heatmap. a The enhanced shape features are embedded into the shape information, and k is the index of the upsampling stage and the convolution stage;

[0127] The expression for the accumulated value is:

[0128] TCount = ∑T,

[0129] Where TCount is the accumulated value and T is the single-channel density heatmap.

[0130] In this embodiment of the invention, a first image patch sequence corresponding to the image to be counted and a second image patch sequence corresponding to the example sample image are input into a counting model to perform target counting and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module, and a density map regression module. The self-similarity calculation module enhances the model's ability to capture the internal structure of the image and strengthens the similarity features between the image to be counted and the example sample image. The shape information embedding module enhances the model's sensitivity to the shape and boundary information of the target object, thereby improving the recognition ability in scenes with dense objects or partial occlusion, enabling the model to not only effectively handle object categories encountered during training but also better generalize to new and unseen categories. The density map regression module improves the accuracy of the counting task.

[0131] Example 3

[0132] Please see Figure 3 , Figure 3 This is a schematic diagram of the structural composition of a target counting system based on image shape embedding in an embodiment of the present invention. The system includes:

[0133] Image acquisition module 31: used to acquire the image to be counted and the example sample image;

[0134] Image preprocessing module 32: used to preprocess the image to be counted and the example sample image to obtain the preprocessed image to be counted and the example sample image;

[0135] Image segmentation module 33: used to perform image segmentation processing on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image;

[0136] Target counting module 34: used to input the first image block sequence and the second image block sequence into the counting model to count the target and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module and a density map regression module.

[0137] In the specific implementation of this invention, the specific implementation methods of the system items can be referred to the above embodiments, and will not be repeated here.

[0138] In this embodiment of the invention, a first image patch sequence corresponding to the image to be counted and a second image patch sequence corresponding to the example sample image are input into a counting model to perform target counting and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module, and a density map regression module. The self-similarity calculation module enhances the model's ability to capture the internal structure of the image and strengthens the similarity features between the image to be counted and the example sample image. The shape information embedding module enhances the model's sensitivity to the shape and boundary information of the target object, thereby improving the recognition ability in scenes with dense objects or partial occlusion, enabling the model to not only effectively handle object categories encountered during training but also better generalize to new and unseen categories. The density map regression module improves the accuracy of the counting task.

[0139] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0140] Furthermore, the above provides a detailed description of the target counting method and system based on image shape embedding provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A target counting method based on image shape embedding, characterized in that, The method includes: Obtain the image to be counted and the example sample image; The images to be counted and the example sample images are preprocessed to obtain preprocessed images to be counted and example sample images. Image segmentation processing is performed on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image; The first image patch sequence and the second image patch sequence are input into the counting model to perform target counting and obtain the target counting result. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module and a density map regression module. The shape information embedding module calculates the width and height of each bounding box and generates a two-dimensional matrix based on the width and height of the bounding boxes. It then maps the two-dimensional matrix to a shape feature with the same dimensions as the high-dimensional feature representation using a linear mapping. The expression for the shape feature is as follows: D s =f shape (linear([w i ,h i ])), Among them, D s For shape features, f shape For shape extraction, `linear` is the linear mapping layer, and `w` is the shape extractor. i h is the width of the annotation box. i [w] is the height of the annotation box. i ,h i [A] is a two-dimensional matrix; shape information embedding enhancement is performed on the shape features to obtain the shape features after shape information embedding enhancement. The expression of the shape features after shape information embedding enhancement is: Among them, D a The enhanced shape features are embedded into the shape information. Vit is a visual transformer, and CrossAtten is a joint-re-cross mechanism. The query input is generated from a high-dimensional feature representation. For key inputs generated from shape features, Input the value generated from the shape features; The density map regression module includes a density regression layer and several upsampling layers. The upsampling layers perform convolution and bilinear interpolation on the enhanced shape features embedded with shape information to obtain an upsampled feature map. The density map regression layer converts the upsampled feature map into a single-channel density heatmap, the expression of which is: T=UpSample(Conv(D a )) k k∈{1,2,3,4}, Where T is the single-channel density heatmap, UpSample is the upsampling stage, Conv is the convolution stage, and D is the density heatmap. a The enhanced shape features are embedded to embed the shape information, k is the index of the upsampling stage and the convolution stage, and the accumulated value of all pixel values ​​in the single-channel density heatmap is calculated. The expression for the accumulated value is: TCount = ∑T, Where TCount is the accumulated value and T is the single-channel density heatmap; and the target counting result is determined based on the accumulated value.

2. The target counting method based on image shape embedding according to claim 1, characterized in that, The step of performing image segmentation processing on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image includes: The preprocessed images to be counted and the example images are subjected to size unification processing to obtain the size-unified images to be counted and the example images. The image to be counted and the example sample image after size unification are divided into image blocks to obtain several first image blocks corresponding to the image to be counted after size unification and several second image blocks corresponding to the example sample image after size unification. Linear mapping is performed on several first image blocks and several second image blocks respectively to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image.

3. The target counting method based on image shape embedding according to claim 1, characterized in that, The self-similarity calculation module includes a multi-head self-attention module, which generates a high-dimensional feature representation with self-similarity information based on the first image block sequence and the second image block sequence.

4. The target counting method based on image shape embedding according to claim 3, characterized in that, The expression for the high-dimensional feature representation is: Among them, D m For high-dimensional feature representation, Vit is the visual transformer, and CrossAtten is the joint-re-cross mechanism. The query vector is generated from the first image patch sequence. The key vector is generated from the second image patch sequence. D is the value vector generated from the second image patch sequence. x For the first image patch sequence, D y This is the second image block sequence.

5. A target counting system based on image shape embedding, characterized in that, The system includes: Image acquisition module: used to acquire the image to be counted and example sample images; Image preprocessing module: used to preprocess the image to be counted and the example sample image to obtain the preprocessed image to be counted and the example sample image; Image segmentation module: used to perform image segmentation processing on the preprocessed image to be counted and the example sample image to obtain a first image block sequence corresponding to the preprocessed image to be counted and a second image block sequence corresponding to the preprocessed example sample image; Target counting module: used to input the first image patch sequence and the second image patch sequence into the counting model to count targets and obtain target counting results. The counting model adopts a visual transformer based on a class-independent counting framework. The visual transformer includes a self-similarity calculation module, a shape information embedding module and a density map regression module. The shape information embedding module calculates the width and height of each bounding box and generates a two-dimensional matrix based on the width and height of the bounding boxes. It then maps the two-dimensional matrix to a shape feature with the same dimensions as the high-dimensional feature representation using a linear mapping. The expression for the shape feature is as follows: D s =f shape (linear([w i ,h i ])), Among them, D s For shape features, f shape For shape extraction, `linear` is the linear mapping layer, and `w` is the shape extractor. i h is the width of the annotation box. i [w] is the height of the annotation box. i ,h i [A] is a two-dimensional matrix; shape information embedding enhancement is performed on the shape features to obtain the shape features after shape information embedding enhancement. The expression of the shape features after shape information embedding enhancement is: Among them, D a The enhanced shape features are embedded into the shape information. Vit is a visual transformer, and CrossAtten is a joint-re-cross mechanism. The query input is generated from a high-dimensional feature representation. For key inputs generated from shape features, Input the value generated from the shape features; The density map regression module includes a density regression layer and several upsampling layers. The upsampling layers perform convolution and bilinear interpolation on the enhanced shape features embedded with shape information to obtain an upsampled feature map. The density map regression layer converts the upsampled feature map into a single-channel density heatmap, the expression of which is: T=UpSample(Conv(D a )) k k∈{1,2,3,4}, Where T is the single-channel density heatmap, UpSample is the upsampling stage, Conv is the convolution stage, and D is the density heatmap. a The enhanced shape features are embedded to embed the shape information, k is the index of the upsampling stage and the convolution stage, and the accumulated value of all pixel values ​​in the single-channel density heatmap is calculated. The expression for the accumulated value is: TCount = ∑T, Where TCount is the accumulated value and T is the single-channel density heatmap; and the target counting result is determined based on the accumulated value.

Images