Multi-scale counting method, apparatus, and computer-readable storage medium

Abstract

The application discloses a multi-scale counting method, device and computer readable storage medium, comprising the steps of: extracting a plurality of first feature maps of an input image at different scales, strengthening the first feature maps by key channel weight, and outputting second feature maps corresponding to the first feature maps; controlling the first feature maps and the second feature maps to be spliced in channels to obtain target features, wherein the target features include a query image feature and a plurality of sample image features; performing feature fusion on the sample image features to output fused sample image features, and performing feature enhancement on the query image feature to output an enhanced query image feature; constructing a similar graph consistent in size with the enhanced query image feature according to the fused sample image features corresponding to all the sample image features and the enhanced query image feature; and constructing a target density graph representing a counting value based on the similar graph, the fused sample image features and the enhanced query image feature. The application accurately realizes counting of small targets.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a multi-scale counting method, apparatus and computer-readable storage medium. Background Technology

[0002] Counting specific objects in images is a crucial task in computer vision, with wide applications in many fields, such as crowd counting, vehicle counting, and agricultural product counting. Thanks to the rapid development of deep learning, the accuracy of counting has continuously improved. Convolutional neural networks, due to their excellent ability to extract deep features from images, are widely used in object counting. However, traditional deep learning counting methods require a large amount of training data to learn a counting model for a specific object. The trained model often exhibits poor generalization performance, which greatly limits its application. In recent years, to address the generalization problem of counting models, researchers have proposed few-sample counting.

[0003] Few-sample counting refers to finding and counting target objects of the same category as one or more samples in an image. This model does not require the target objects to belong to the same category as those samples in the training data. Users can specify any number of samples in the image and input both the samples and the query image into the model. The model calculates the similarity between the samples and the objects, enhances the features of the target objects, and finally outputs the count value of the query image. In this way, a single model can be used to count objects of unspecified categories.

[0004] Existing small-sample counting methods outperform traditional deep learning counting methods in generalization, but they still have some shortcomings. 1) The features of the samples are not fully utilized. The model only uses a single sample to calculate the similarity map, and for multiple samples, it simply calculates the average similarity map. 2) The scale of the target objects in the sample and query images varies greatly. In images with dense objects, due to factors such as occlusion and lens distance, the scale of the sample and the target object is inconsistent, causing existing models to be unable to effectively count small objects. Summary of the Invention

[0005] The main objective of this application is to provide a multi-scale counting method, apparatus, and computer-readable storage medium, which aims to solve the problem of not being able to effectively count small targets in images of dense objects.

[0006] To achieve the above objectives, this application provides a multi-scale counting method, which includes the following steps:

[0007] Extract multiple first feature maps from the input image at different scales, perform key channel weight enhancement on the first feature maps, and output a second feature map corresponding to the first feature map.

[0008] The first feature map and the second feature map are concatenated by channels to obtain target features, wherein the target features include query image features and multiple sample image features;

[0009] The sample image features are fused to output fused sample image features, and the query image features are enhanced to output enhanced query image features.

[0010] A similarity graph with the same size as the enhanced query image features is constructed based on the fused sample image features and the enhanced query image features corresponding to all the sample image features;

[0011] A target density map representing the count values ​​is constructed based on the similarity map, the fused sample image features, and the enhanced query image features.

[0012] Optionally, the step of fusing the features of the sample image and outputting the fused sample image features includes:

[0013] Adjust the size of all the sample image features to be consistent to obtain multiple first sample image features;

[0014] The features of the first sample image are convolved separately, and the average value of the multiple convolved features of the first sample image is obtained;

[0015] The average value is combined with multiple first sample image features by channel concatenation and channel weight enhancement, and the common feature weights corresponding to the multiple first sample image features are output.

[0016] The common feature weights are multiplied by the multiple sample image features respectively to obtain multiple fused sample image features corresponding to the multiple sample image features.

[0017] Optionally, the step of enhancing the features of the query image and outputting enhanced query image features includes:

[0018] Based on the spatial self-attention mechanism, the target features in the query image features are enhanced, and the enhanced query image features are output.

[0019] Optionally, the step of constructing a similarity graph with the same size as the enhanced query image features based on the fused sample image features and the enhanced query image features corresponding to all the sample image features includes:

[0020] The fused sample image features are convolved with the enhanced query image features to obtain a similarity map with the same size as the enhanced query image features and corresponding to each of the sample image features.

[0021] Optionally, the step of constructing a target density map representing the count values ​​based on the similarity map, the fused sample image features, and the enhanced query image features includes:

[0022] Each of the similar images, the fused sample image features, and the enhanced query image features is concatenated by channel to obtain multiple concatenated image features;

[0023] Multiple stitched images are input into a preset regressor to obtain multiple target density maps with the same size as the query image in the input images;

[0024] After the step of obtaining multiple target density maps with the same size as the query image in the input image, the method further includes:

[0025] Count the multiple target density maps to obtain the count values.

[0026] Optionally, the input image includes a query image and a sample image; the first feature map includes a first query feature map and a first sample feature map; the second feature map includes a second query feature map and a second sample feature map;

[0027] The steps of extracting multiple first feature maps of the input image at different scales, applying key channel weighting to the first feature maps, and outputting a second feature map corresponding to the first feature maps include:

[0028] Extract multiple first query feature maps of the query image at different scales and multiple first sample feature maps of the sample image at different scales respectively;

[0029] The first query feature map and the first sample feature map are weighted by key channels, and the second query feature map and the second sample feature map are output at the same scale.

[0030] Optionally, the step of controlling the first feature map and the second feature map to perform channel concatenation to obtain the target feature includes:

[0031] The third feature map with the lowest scale in the first feature map is continuously upsampled to obtain a fourth feature map with the same scale as each of the first feature maps.

[0032] The target feature is obtained by channel-splitting the third feature map and the first feature map of the same scale as the third feature map, and by channel-splitting the fourth feature map and the second feature map of the same scale respectively.

[0033] Optionally, the multi-scale counting method further includes:

[0034] The target density map is verified according to a preset loss function, and the multi-scale computing network is corrected based on the verification results.

[0035] In addition, to achieve the above objectives, the present invention also provides an apparatus, characterized in that the apparatus includes: a memory, a processor, and a multi-scale counting program stored in the memory and executable on the processor, wherein the multi-scale counting program, when executed by the processor, implements the steps of the multi-scale counting method as described above.

[0036] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a multi-scale counting program, which, when executed by a processor, implements the steps of the multi-scale counting method as described above.

[0037] This invention provides a multi-scale counting method, apparatus, and computer-readable storage medium. The multi-scale counting method includes the following steps: extracting multiple first feature maps of an input image at different scales; performing channel weight enhancement on the first feature maps; outputting a second feature map corresponding to the first feature maps; controlling the first feature maps and the second feature maps to perform channel concatenation to obtain target features, wherein the target features include query image features and multiple sample image features; performing feature fusion on the sample image features to output fused sample image features, and performing feature enhancement on the query image features to output enhanced query image features; obtaining multiple similarity maps with the same size as the enhanced query image features based on the multiple fused sample image features and the enhanced query image features; and obtaining a density map representing the count value based on the similarity maps and the enhanced query image features. Through the above method, this invention can fuse the features of all samples, integrate the common features of multiple samples into the features of a single sample, improve the commonality between samples, and optimize the method of simply averaging similarity maps when counting multiple sample images; furthermore, by obtaining query image features and sample image features at different scales, it can solve the problem of inconsistent scales between samples and target objects. This allows for the counting of small targets in images with dense objects, avoiding the problem of inconsistent sample and target object scales caused by factors such as occlusion and lens distance. It enables more accurate counting of small targets. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the hardware operating environment involved in the embodiment of the multi-scale counting method of the present invention;

[0039] Figure 2 This is a schematic diagram of the data flow in one embodiment of the multi-scale counting method of the present invention;

[0040] Figure 3 This is a schematic diagram of the data flow direction across the self-attention layer in the multi-scale counting method of this invention;

[0041] Figure 4 This is a flowchart illustrating the operation of the counting module in the multi-scale counting method of the present invention;

[0042] Figure 5 This is a schematic diagram of the sample image feature fusion process in the multi-scale counting method of the present invention;

[0043] Figure 6 This is a flowchart illustrating an embodiment of the multi-scale counting method of the present invention.

[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0046] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention.

[0047] In this embodiment of the invention, the terminal is a device.

[0048] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a DVI interface 1004; a USB interface 1005; and a memory 1006. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The DVI interface 1004 may optionally include a standard wired interface for connecting to other external devices via a DVI cable. The USB interface 1005 may optionally include a standard wired interface for connecting to other external devices via a USB cable. The memory 1006 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1006 may also be a storage device independent of the aforementioned processor 1001.

[0049] Optionally, the terminal may also include audio circuitry, etc., which will not be elaborated here.

[0050] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0051] like Figure 1 As shown, the memory 1006, which serves as a computer storage medium, may include an operating system, a DVI interface module, a USB interface module, a user interface module, and a multi-scale counting program.

[0052] exist Figure 1 In the terminal shown, the DVI interface 1004 is mainly used to connect to external devices and communicate with them; the USB interface 1005 is mainly used to connect to external devices and communicate with them; the user interface 1003 is mainly used to connect to clients and communicate with them; and the processor 1001 can be used to call the multi-scale counting program stored in the memory 1006 and perform the following operations:

[0053] Extract multiple first feature maps from the input image at different scales, perform key channel weight enhancement on the first feature maps, and output a second feature map corresponding to the first feature map.

[0054] The first feature map and the second feature map are concatenated by channels to obtain target features, wherein the target features include query image features and multiple sample image features;

[0055] The sample image features are fused to output fused sample image features, and the query image features are enhanced to output enhanced query image features.

[0056] A similarity graph with the same size as the enhanced query image features is constructed based on the fused sample image features and the enhanced query image features corresponding to all the sample image features;

[0057] A target density map representing the count values ​​is constructed based on the similarity map, the fused sample image features, and the enhanced query image features.

[0058] The specific embodiments of the device of the present invention are basically the same as the embodiments of the multi-scale counting program described below, and will not be repeated here.

[0059] Please see Figure 6 , Figure 6 This is a flowchart illustrating the first embodiment of the multi-scale counting method of the present invention. The multi-scale counting method provided in this embodiment includes the following steps:

[0060] Step S10: Extract multiple first feature maps of the input image at different scales, perform key channel weight enhancement on the first feature maps, and output a second feature map corresponding to the first feature map.

[0061] Please see Figure 2 , Figure 2 This invention provides a schematic diagram of the data flow in a multi-scale counting system. The system comprises a feature extraction layer, a cross-channel self-attention layer, a channel stitching module, and a counting module. Specifically, the feature extraction layer extracts a first feature map. In this embodiment, the input image includes a sample image and a query image. Both the sample image and the query image have features extracted using a ResNet convolutional neural network. The outputs of different layers of the ResNet are extracted separately to obtain first feature maps at different scales. The first feature map includes a first query feature map and a first sample feature map; the second feature map includes a second query feature map and a second sample feature map.

[0062] Specifically, in one embodiment, step S10 further includes:

[0063] Step A11: Extract multiple first query feature maps of the query image at different scales and multiple first sample feature maps of the sample image at different scales, respectively;

[0064] In this embodiment, the query image is the object to be counted, and the sample image is the basis for the counting. The problem to be solved in this invention is to find and count target objects of the same category as the sample objects in the query image using the sample images. Feature maps at different scales can be obtained by extracting the outputs of different layers of a ResNet convolutional neural network. For example, inputting a sample image of (3, H, W) into a ResNet convolutional neural network will extract first sample feature maps at four scales: (256, H / 4, W / 4), (512, H / 8, W / 8), (1024, H / 16, W / 16), and (2048, H / 32, W / 32), respectively, at different output layers. Here, 3, 256, 512, 1024, and 2048 are the number of channels in the image, H is the length of the image, and W is the width of the image.

[0065] Step A12: The first query feature map and the first sample feature map are weighted by key channels, and the second query feature map and the second sample feature map are output at the same scale.

[0066] Please see Figure 3In this embodiment, the weights of key channels in the first query feature map and the first sample feature map are strengthened, which enhances the features of the first query feature map and the first sample feature map, making them more prominent. Specifically, the first query feature map and the first sample feature map can be input into a cross-channel self-attention layer, which includes a multilayer perceptron (MLP) and channel self-attention. The LN layer refers to layer normalization. The cross-channel self-attention combines four channels of different depths. Through upsampling and downsampling, the length and width of the first query feature maps and the first sample feature maps of different scales are unified. Then, they are concatenated along the channel dimension, and the number of channels is reduced by convolution to match the number of channels in each layer of the first query feature map and the first sample feature map. This is used as K (Key) and V (Value). Q (Query) is multiplied by K, divided by the sum of the corresponding channels, and then multiplied by V. After layer normalization, the output is added to the previous Q. Through the LN layer and the MLP layer, the output of the cross-channel self-attention, namely the second query feature map and the second sample feature map, is obtained. Figure 2 The left half of the output of the cross-channel self-attention layer in the middle.

[0067] Step S20: Control the first feature map and the second feature map to perform channel splicing to obtain target features, wherein the target features include query image features and multiple sample image features;

[0068] Channel concatenation expands the channel depth; for example, two 512-channel feature maps are combined into a 1024-channel feature map, increasing the number of channels. This step is implemented in the cross-channel self-attention layer. Specifically...

[0069] In one embodiment, step S20 further includes:

[0070] Step A21: The third feature map with the lowest scale in the first feature map is continuously upsampled to obtain a fourth feature map with the same scale as each of the first feature maps.

[0071] The third feature map is the feature map with the lowest scale among the multiple first feature maps. Upsampling is the process of enlarging the size of the feature map. Specifically, upsampling can be performed through deconvolution. In this embodiment, upsampling is performed sequentially starting from the third feature map with the lowest scale. Before each upsampling, it is concatenated with the second feature map of the corresponding scale. Then, the number of channels is reduced through a convolutional layer, and then upsampling is performed through deconvolution to obtain a fourth feature map with the same scale as each first feature map. For example, ... Figure 2As shown, the lowest scale of the third feature map is (2048, H / 32, W / 32). After channel concatenation with the first feature map with the same lowest scale, and then upsampling, one of the fourth feature maps obtained is (1024, H / 16, W / 16). After channel concatenation with the second feature map with the same scale, and then upsampling, one of the fourth feature maps obtained is (512, H / 8, W / 8). After channel concatenation with the second feature map with the same scale, and then upsampling, one of the fourth feature maps obtained is (256, H / 4, W / 4).

[0072] Step A22: Channel stitching is performed on the third feature map and the first feature map of the same scale as the third feature map, and channel stitching is performed on the fourth feature map and the second feature map of the same scale respectively to obtain the target feature.

[0073] In this embodiment, in the channel stitching module, the third feature map with the lowest scale is channel stitched with the first feature map with the same lowest scale, and the fourth feature map and the second feature map with the same scale are channel stitched. For example... Figure 2 As shown, the third feature map with 2048 channels is concatenated with the first feature map with 2048 channels, resulting in 2048 × 2 channels. Similarly, the fourth feature map with 1024 channels is concatenated with the second feature map with 1024 channels, resulting in 1024 × 2 channels. The third feature map includes a third query feature map and a third sample feature map, and the fourth feature map includes a fourth query feature map and a fourth sample feature map. The corresponding target features are the query image features and multiple sample image features, respectively. After obtaining the target features, they are input into a counting module for feature fusion and enhancement, and the module outputs a target density map. For details, please refer to the following steps.

[0074] Step S30: Perform feature fusion on the sample image features and output fused sample image features; and perform feature enhancement on the query image features and output enhanced query image features.

[0075] In one embodiment, the step S30 of performing feature fusion on the sample image features and outputting the fused sample image features further includes:

[0076] Step A31: Adjust the size of all the sample image features to be consistent to obtain multiple first sample image features;

[0077] Please see Figure 4 and Figure 5 , Figure 4 This is a flowchart of the counting module. Figure 5This is a schematic diagram of sample image feature fusion. "Consistent dimensions" means that the length and width of the sample image features are consistent. In the counting module of this embodiment, the length and width of all sample image features need to be adjusted to be consistent through ROI Pooling. ， This facilitates input model training, resulting in first sample image features that correspond one-to-one with the features of the sample images, but with consistent dimensions.

[0078] Step A32: Convolve the features of the first sample image respectively, and obtain the average value of the multiple first sample image features after convolution;

[0079] Step A33: Perform channel concatenation and channel weight enhancement on the average value and multiple first sample image features respectively, and output the common feature weights corresponding to the multiple first sample image features;

[0080] Step A34: Multiply the common feature weights with the multiple sample image features respectively to obtain multiple fused sample image features corresponding to the multiple sample image features.

[0081] In this embodiment, each first sample feature needs to be convolved individually to obtain the average value of all first sample image features after convolution. This average value is then concatenated with the features of a single sample and fed into the convolutional layer. After passing through an MLP layer, the channels are multiplied by corresponding weights. The MLP layer outputs the common feature weights of the corresponding samples, with the output dimension being the same as the dimension of the single sample feature. This weight is then multiplied with the original sample image features to obtain multiple sample features with enhanced commonality. In this embodiment, by fusing multiple sample image features, their commonality is enhanced, thereby enabling more accurate counting.

[0082] In one embodiment, step S30, which involves enhancing the features of the query image and outputting the enhanced query image features, includes:

[0083] Step A35: Enhance the target features in the query image features and output the enhanced query image features.

[0084] The query image is the image to be counted, while the target features are the specific features present in the query image that need to be counted. Enhanced query image features are those that have been strengthened with the target features. The query image features are processed by a spatial self-attention module, which enhances the features of the target object to distinguish it from the background features of the query image. The formula for the spatial self-attention module is as follows:

[0085]

[0086] Among them, X iX is the feature at a certain point after image downsampling. i It is X i The feature space centered at X is an R×R space, where R can be any number, such as 2 or 3. For example, if the image is 5×5 in size, X... i The characteristics of the point at the exact center, and X′ i That is, the 2×2 or 3×3 characteristic space surrounding the point at the exact center.

[0087] In this formula, X needs to be... i With X′ i Calculate the cosine similarity of each point on the graph, then perform Softmax normalization to obtain X. i A similarity map with surrounding feature points is used as a weight for X′. i Multiplication reduces the dimensionality of the R×R weighted feature map into feature points through summation. These feature points are then passed through a convolutional layer and finally multiplied by X. i Add them together.

[0088] Step S40: Construct a similarity map with the same size as the enhanced query image features based on the fused sample image features and the enhanced query image features corresponding to all the sample image features;

[0089] In one embodiment, step S40 further includes:

[0090] Step A41: Convolve the fused sample image features with the enhanced query image features to obtain similarity maps that have the same size as the enhanced query image features and correspond to the features of the multiple sample images respectively.

[0091] After obtaining the enhanced query image features through the spatial self-attention module, the MLP layer then convolves with the fused sample image features to obtain a similarity map corresponding to multiple sample image features. The size of the similarity map is consistent with the size of the query image features.

[0092] Step S50: Construct a target density map representing the count values ​​based on the similarity map, the fused sample image features, and the enhanced query image features.

[0093] In one embodiment, step S50 further includes:

[0094] Step A51: For each of the similar images, the fused sample image features and the enhanced query image features are respectively concatenated by channel to obtain multiple concatenated image features;

[0095] Step A52: Input the multiple stitched images into a preset regressor to obtain multiple target density maps with the same size as the query image in the input images;

[0096] The fused sample features are spatially copied and expanded to the same size as the query image features. Then, they are combined with similar images to enhance the query image features in the channel dimension. The result is fed into a regressor consisting of convolutional layers and bilinear upsampling layers, so that the target density map output by this module is consistent with the size of the original query image.

[0097] Following step A52, the method further includes:

[0098] Step A53: Count the multiple target density maps to obtain count values.

[0099] The average density map is obtained by averaging the target density maps output by each counting module. Finally, the total count value is obtained by summing the average density maps. Alternatively, the total count value can be obtained by summing all target density maps directly.

[0100] This invention provides a multi-scale counting method, comprising the steps of: extracting multiple first feature maps of an input image at different scales; weighting the first feature maps by focusing on key channels; outputting a second feature map corresponding to the first feature maps; controlling the first feature maps and the second feature maps to perform channel concatenation to obtain target features, wherein the target features include query image features and multiple sample image features; performing feature fusion on the sample image features to output fused sample image features, and performing feature enhancement on the query image features to output enhanced query image features; obtaining multiple similarity maps with the same size as the enhanced query image features based on the multiple fused sample image features and the enhanced query image features; and obtaining a density map representing the count value based on the similarity maps and the enhanced query image features. Through the above method, this invention can fuse the features of all samples, integrate the common features of multiple samples into the features of a single sample, improve the commonality between samples, and optimize the method of simply averaging similarity maps when using multiple sample images for counting; furthermore, by obtaining query image features and sample image features at different scales, it can solve the problem of inconsistent scales between samples and target objects. This allows for the counting of small targets in images with dense objects, avoiding the problem of inconsistent sample and target object scales caused by factors such as occlusion and lens distance. It enables more accurate counting of small targets.

[0101] Furthermore, based on the first embodiment of the multi-scale counting method of the present invention, a second embodiment of the present invention is proposed, wherein the multi-scale counting method further includes:

[0102] Step A60: Verify the target density map according to the preset loss function, and correct the multi-scale computing network according to the verification results.

[0103] Specifically, the verification result is the error result. When the error is large, the multi-scale computing network constructed by the multi-scale computing method is corrected to achieve a more accurate count value.

[0104] In this embodiment, the loss function is composed of the following functions:

[0105]

[0106] L sim =SSIM(D g D p )+β·L NCE (D g D p )

[0107] In the above formula, D g D represents the true density map. p Let represent the target density map, where α and β are coefficients. The formula SSIM represents the image density structure similarity loss. L NCE This is a similarity-based noise contrast estimation method. It compares a similarity map with the true density map, classifying the locations in the similarity map where the corresponding element value in the density map is greater than zero as positive signal regions i, and locations where the value is equal to zero as noise regions j. Its function is as follows:

[0108]

[0109] In this embodiment, the loss function can be used to modify and adjust the network constructed by the multi-scale counting method, so that the multi-scale counting network can more accurately count small samples.

[0110] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a multi-scale counting program, which, when executed by a processor, performs the following operations:

[0111] Extract multiple first feature maps from the input image at different scales, perform key channel weight enhancement on the first feature maps, and output a second feature map corresponding to the first feature map.

[0112] The first feature map and the second feature map are concatenated by channels to obtain target features, wherein the target features include query image features and multiple sample image features;

[0113] The sample image features are fused to output fused sample image features, and the query image features are enhanced to output enhanced query image features.

[0114] A similarity graph with the same size as the enhanced query image features is constructed based on the fused sample image features and the enhanced query image features corresponding to all the sample image features;

[0115] A target density map representing the count values ​​is constructed based on the similarity map, the fused sample image features, and the enhanced query image features.

[0116] Furthermore, when the multi-scale counting program is executed by the processor, it also performs the following operations:

[0117] Adjust the size of all the sample image features to be consistent to obtain multiple first sample image features;

[0118] The features of the first sample image are convolved separately, and the average value of the multiple convolved features of the first sample image is obtained;

[0119] The average value is combined with multiple first sample image features by channel concatenation and channel weight enhancement, and the common feature weights corresponding to the multiple first sample image features are output.

[0120] The common feature weights are multiplied by the multiple sample image features respectively to obtain multiple fused sample image features corresponding to the multiple sample image features.

[0121] Furthermore, when the multi-scale counting program is executed by the processor, it also performs the following operations:

[0122] Based on the spatial self-attention mechanism, the target features in the query image features are enhanced, and the enhanced query image features are output.

[0123] Furthermore, when the multi-scale counting program is executed by the processor, it also performs the following operations:

[0124] The fused sample image features are convolved with the enhanced query image features to obtain a similarity map with the same size as the enhanced query image features and corresponding to each of the sample image features.

[0125] Furthermore, when the multi-scale counting program is executed by the processor, it also performs the following operations:

[0126] For each of the similar images, the fused sample image features and the enhanced query image features are respectively concatenated by channel to obtain multiple concatenated image features;

[0127] Multiple stitched images are input into a preset regressor to obtain multiple target density maps with the same size as the query image in the input images;

[0128] After the step of obtaining multiple target density maps with the same size as the query image in the input image, the method further includes:

[0129] Count the multiple target density maps to obtain the count values.

[0130] Furthermore, when the multi-scale counting program is executed by the processor, it also performs the following operations:

[0131] Extract multiple first query feature maps of the query image at different scales and multiple first sample feature maps of the sample image at different scales respectively;

[0132] The first query feature map and the first sample feature map are weighted by key channels, and the second query feature map and the second sample feature map are output at the same scale.

[0133] Furthermore, when the multi-scale counting program is executed by the processor, it also performs the following operations:

[0134] The third feature map with the lowest scale in the first feature map is continuously upsampled to obtain a fourth feature map with the same scale as each of the first feature maps.

[0135] The target feature is obtained by channel-splitting the third feature map and the first feature map of the same scale as the third feature map, and by channel-splitting the fourth feature map and the second feature map of the same scale respectively.

[0136] Furthermore, when the multi-scale counting program is executed by the processor, it also performs the following operations:

[0137] The target density map is verified according to a preset loss function, and the multi-scale computing network is corrected based on the verification results.

[0138] The present invention also proposes a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium may be... Figure 1 The memory 02 in the device may also be at least one of ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc., and the computer-readable storage medium includes several information for causing the device to perform the methods described in various embodiments of the present invention.

[0139] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0140] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0142] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A multi-scale counting method, characterized in that, The multi-scale counting method is applied to a multi-scale counting system, and the multi-scale counting method includes the following steps: Extract multiple first feature maps from the input image at different scales, perform key channel weight enhancement on the first feature maps, and output a second feature map corresponding to the first feature map. The first feature map and the second feature map are concatenated by channels to obtain target features, wherein the target features include query image features and multiple sample image features; The sample image features are fused to output fused sample image features, and the query image features are enhanced to output enhanced query image features. A similarity graph with the same size as the enhanced query image features is constructed based on the fused sample image features and the enhanced query image features corresponding to all the sample image features; A target density map representing the count values ​​is constructed based on the similarity map, the fused sample image features, and the enhanced query image features. The step of fusing the features of the sample image and outputting the fused sample image features includes: Adjust the size of all the sample image features to be consistent to obtain multiple first sample image features; The features of the first sample image are convolved separately, and the average value of the multiple convolved features of the first sample image is obtained; The average value is combined with multiple first sample image features by channel concatenation and channel weight enhancement, and the common feature weights corresponding to the multiple first sample image features are output. The common feature weights are multiplied by the multiple sample image features respectively to obtain multiple fused sample image features corresponding to the multiple sample image features; The input image includes a query image and a sample image; the first feature map includes a first query feature map and a first sample feature map; the second feature map includes a second query feature map and a second sample feature map. The steps of extracting multiple first feature maps of the input image at different scales, applying key channel weighting to the first feature maps, and outputting a second feature map corresponding to the first feature maps include: Extract multiple first query feature maps of the query image at different scales and multiple first sample feature maps of the sample image at different scales respectively; The first query feature map and the first sample feature map are weighted by key channels, and the second query feature map and the second sample feature map at the same scale are output respectively. The step of controlling the first feature map and the second feature map to perform channel concatenation to obtain the target feature includes: The third feature map with the lowest scale in the first feature map is continuously upsampled to obtain a fourth feature map with the same scale as each of the first feature maps. The target feature is obtained by channel-splitting the third feature map and the second feature map of the same scale, and by channel-splitting the fourth feature map and the second feature map of the same scale respectively.

2. The multi-scale counting method as described in claim 1, characterized in that, The step of enhancing the features of the query image and outputting the enhanced query image features includes: Based on the spatial self-attention mechanism, the target features in the query image features are enhanced, and the enhanced query image features are output.

3. The multi-scale counting method as described in claim 2, characterized in that, The step of constructing a similarity map with the same size as the enhanced query image features based on the fused sample image features and the enhanced query image features corresponding to all the sample image features includes: The fused sample image features are convolved with the enhanced query image features to obtain a similarity map with the same size as the enhanced query image features and corresponding to each of the sample image features.

4. The multi-scale counting method as described in claim 3, characterized in that, The step of constructing a target density map representing count values ​​based on the similarity map, the fused sample image features, and the enhanced query image features includes: Each of the similar images, the fused sample image features, and the enhanced query image features is concatenated by channel to obtain multiple concatenated image features; Multiple stitched images are input into a preset regressor to obtain multiple target density maps with the same size as the query image in the input images; After the step of obtaining multiple target density maps with the same size as the query image in the input image, the method further includes: Count the multiple target density maps to obtain the count values.

5. The multi-scale counting method as described in claim 1, characterized in that, The multi-scale counting method also includes: The target density map is verified according to a preset loss function, and the multi-scale computing network is corrected based on the verification results.

6. An apparatus, characterized in that, The apparatus includes: a memory, a processor, and a multi-scale counting program stored in the memory and executable on the processor, wherein the multi-scale counting program, when executed by the processor, implements the steps of the multi-scale counting method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a multi-scale counting program, which, when executed by a processor, implements the steps of the multi-scale counting method as described in any one of claims 1 to 5.

Images