Target counting method and device, electronic equipment and storage medium
By acquiring the features of the input image in the target counting task, acquiring the foreground target feature weight data and image feature compression processing, the impact of complex background on the dense scene target counting task is solved, and more efficient and accurate target number recognition is achieved.
Patent Information
- Application Number
- CN202411805009.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-09
- Publication Date
- 2025-04-11
AI Technical Summary
In dense scenarios, complex backgrounds have a significant impact on the inference speed and accuracy of the target counting task, and existing algorithms are difficult to effectively suppress the interference of background data.
By acquiring the input image, feature extraction and acquisition of foreground target feature weight data, and combined with image feature compression processing, the impact of background data on the counting task is suppressed, thereby improving the accuracy of the number of targets.
This method effectively suppresses the impact of background data on the target counting task, improves the system's inference prediction efficiency and accuracy in dense scenarios, and reduces execution costs.
Smart Images
Figure CN119810734A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a target counting method and device, electronic equipment, and storage medium. Background Art
[0002] In recent years, with the development of deep learning, various tasks in the field of computer vision have achieved remarkable results. Object counting tasks in dense scenes have also made great progress. For example, crowd counting has important application potential in the field of security.
[0003] At present, most algorithms use the density map estimation algorithm, which first converts the labels of dense targets (gt) into density maps. Then, the powerful representation ability of convolutional neural networks (CNN) is used to extract features from the input data, and the model is trained through objective functions such as MSE (mean squared error). In the inference and prediction stage, the density map predicted by the model is integrated, that is, the value of the density map is accumulated to obtain the predicted number of dense targets.
[0004] However, in dense scene counting tasks, complex background has always been a large distractor, seriously affecting the inference speed and accuracy of inference prediction. Summary of the invention
[0005] In view of the above problems, a target counting method and device, electronic device, and storage medium are proposed to overcome the above problems or at least partially solve the above problems, including:
[0006] A target counting method, the method comprising:
[0007] Acquire an input image, and perform feature extraction on the input image to obtain image feature data of the input image;
[0008] Performing foreground extraction processing on the image feature data to obtain foreground target feature weight data, and performing image feature compression processing on the image feature data to obtain compressed feature data;
[0009] Foreground target feature data is obtained according to the foreground target feature weight data and the compressed feature data, and the number of targets in the input image is determined according to the foreground target feature data.
[0010] Optionally, performing foreground extraction processing on the image feature data to obtain foreground target feature weight data includes:
[0011] The target category information of the input image and the channel number information of the image feature data are determined, and the image feature data is subjected to foreground extraction processing according to the target category information and the channel number information to obtain the foreground target feature weight data.
[0012] Optionally, performing image feature compression processing on the image feature data to obtain compressed feature data includes:
[0013] The channel number information of the image feature data is determined, and image feature compression processing is performed on the image feature data according to the channel number information by a preset operator to obtain the compressed feature data.
[0014] Optionally, obtaining the foreground target feature data according to the foreground target feature weight data and the compressed feature data includes:
[0015] According to a preset objective function and the foreground target feature weight data, target template data is obtained, and the target template data and the compressed feature data are multiplied to obtain the foreground target feature data.
[0016] Optionally, determining the number of objects in the input image according to the foreground object feature data includes:
[0017] The foreground target feature data is subjected to target regression processing to obtain a target density map, and the number of targets in the input image is determined based on the target density map.
[0018] Optionally, the input image includes at least original density map information, and determining the number of targets in the input image according to the target density map includes:
[0019] The target density map is interpolated according to the original density map information, and the target density map after the interpolation is integrated and accumulated to obtain the number of targets in the input image.
[0020] Optionally, the image feature data is a feature map including quantity information of the input image, feature data channel number information, feature data height information and feature data width information.
[0021] A target counting device, comprising:
[0022] An image feature data acquisition module is used to acquire an input image and perform feature extraction on the input image to obtain image feature data of the input image;
[0023] a foreground target feature weight data acquisition module, used to perform foreground extraction processing on the image feature data to obtain foreground target feature weight data, and perform image feature compression processing on the image feature data to obtain compressed feature data;
[0024] The target number determination module is used to obtain foreground target feature data according to the foreground target feature weight data and the compressed feature data, and determine the number of targets in the input image according to the foreground target feature data.
[0025] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the target counting method as described above when executed by the processor.
[0026] A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the target counting method as described above is implemented.
[0027] A computer program product comprises a computer program, and when the computer program is executed by a processor, the target counting method as described above is implemented.
[0028] The embodiments of the present invention have the following advantages:
[0029] The present invention provides a target counting method, which obtains image feature data of the input image by acquiring an input image and performing feature extraction on the input image; then performs foreground extraction processing on the image feature data to obtain foreground target feature weight data, and performs image feature compression processing on the image feature data to obtain compressed feature data; then obtains foreground target feature data according to the foreground target feature weight data and the compressed feature data, and determines the number of targets in the input image according to the foreground target feature data. The method realizes the suppression of the influence of background data on the counting task by acquiring foreground target feature weight data and compressed feature data and combining and applying them, so that the system can more effectively distinguish between actual counting targets (foreground targets) and background targets as interference items, reduces the execution cost of the target counting task, and improves the reasoning prediction efficiency and accuracy of the target counting task. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0031] Figure 1is a flowchart of the steps of a target counting method provided by some embodiments of the present invention;
[0032] Figure 2 is an example diagram of a system architecture provided by some embodiments of the present invention;
[0033] Figure 3 It is a schematic diagram of the structure of a target counting device provided by some embodiments of the present invention. DETAILED DESCRIPTION
[0034] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] In the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. This is only for the purpose of clearly describing the technical solutions of the embodiments of the present application, and shall not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.
[0036] In the embodiments of the present application, the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0037] For the sake of convenience, the following is a glossary of terms used in the embodiments of the present application:
[0038] gt: ground truth (real label), the target information marked in the image. Depending on the algorithm used, the label content will be different. It can be the center point coordinates of the target or the density map obtained by transforming the center point coordinates.
[0039] CNN: convolutional neural network, convolutional neural network.
[0040] Conv2d: Convolution operator, which is a two-dimensional convolution operation function. It processes input data (usually one or more images) by sliding a learnable convolution kernel (or filter), performing element-level multiplication and summing each time it slides (sometimes including a bias term), and finally generates a feature map. The parameters it contains are:
[0041] --in_channels: the number of channels of the input signal;
[0042] --out_channels: the number of output channels produced by convolution;
[0043] --kernel_size: The size of the convolution kernel, which can be a single integer or a tuple of integers, indicating the height and width of the convolution kernel;
[0044] --stride: The stride of the convolution, which can be a single integer or a tuple of integers, indicating the horizontal and vertical strides of the convolution kernel sliding on the input data;
[0045] --padding: If True, padding is performed in the height and width directions so that the output feature map size is consistent with the input; if False, no padding is performed.
[0046] Softmax: It is a very commonly used activation function, especially when dealing with multi-classification problems. It can convert an unnormalized real vector into a probability distribution, that is, each element in the vector is compressed between 0 and 1, and the sum of all elements is 1. It contains the parameter "dim" to specify which dimension of the input data to operate on.
[0047] Max: The maximum value operation function. Its parameter "dim" is used to specify the dimension of the input data on which the operation is performed.
[0048] In recent years, with the development of deep learning, various tasks in the field of computer vision have achieved remarkable results. Object counting tasks in dense scenes have also made great progress. For example, crowd counting has important application potential in the field of security.
[0049] At present, most algorithms use the density map estimation algorithm, which first converts the labels of dense targets (gt) into density maps. Then, the powerful representation ability of convolutional neural networks (CNN) is used to extract features from the input data, and the model is trained through objective functions such as MSE. In the inference and prediction stage, the density map predicted by the model is integrated, that is, the value of the density map is accumulated to obtain the predicted number of dense targets.
[0050] The point regression-based method proposed in recent years directly borrows the approach of detection algorithms. It first generates some preset candidate points (point proposals) based on the feature map, designs some matching rules to match them with gt, and divides these candidate points into positive sample points and negative sample points. Then, the classification head and regression head are used to predict the categories and coordinates of these candidate points.
[0051] However, no matter what algorithm is used, complex background has always been a very large interference factor in dense scene counting tasks. In order to solve this problem, many algorithms have designed complex feature extraction structures or added complex attention mechanisms, but the effect is not obvious, and the reasoning speed and accuracy of inference prediction will still be seriously affected.
[0052] In order to solve the above problems, the present invention is based on the core technical concept of suppressing the influence of background data on the counting task by obtaining foreground target feature weight data and compressed feature data and combining them for application, and improves the target counting method in the related art. The present invention will be described in detail in conjunction with the accompanying drawings as follows:
[0053] Reference Figure 1 , shows a flowchart of a target counting method provided by some embodiments of the present invention, which may specifically include the following steps:
[0054] Step 101, obtaining an input image, and performing feature extraction on the input image to obtain image feature data of the input image;
[0055] In the specific implementation, Figure 2 As shown, an image data input module, an image feature extraction module, a background filtering module and a target regression module can be set in the system architecture to implement the corresponding functions in the steps of the embodiment of the present application, wherein the background filtering module can be composed of a foreground extraction module and an image feature compression module.
[0056] Among them, the input image data can be received through the image data input module. In the training stage, the input image can also include label information (gt), where the label information can be the center point coordinate information of all targets in the input image, or it can be the original density map obtained by filtering with a Gaussian kernel function based on the center point of the target. After the original density map is integrated and accumulated, the number of targets can be obtained.
[0057] In one example, the target counting system can perform model training on an algorithm based on density map regression. The image data of the training data set can carry label information through the image data input module and be passed into the training system as an input image, where the label information can include the original density map information of the input image. If the height of the input image is image_h and the width is image_w, the size of the original density map is also a matrix with a height of image_h and a width of image_w, and its value range is between 0 and 1.
[0058] Furthermore, the feature information of the input image can be extracted through the image feature extraction module. The feature extraction network of this module can be any CNN structure network, such as VGG, ResNet, MobileNet, DarkNet, etc. The image data processed by this module is the image feature data, and its shape can be in the format of [N, C, H, W] (N represents the number of input images, C is the number of channels of the feature data, H is the height of the feature data, and W is the width of the feature data).
[0059] In one example, the image feature extraction module can be used to extract features from the input image data to obtain a feature map with a width of feature_width, a height of feature_height, and a number of channels of K. The features extracted by this module can be the output of networks such as VGG, ResNet, MobileNet, DarkNet, etc., or can be features processed by modules such as ASPP (Atrous Spatial Pyramid Pooling).
[0060] In some embodiments of the present invention, the image feature data is a feature map including quantity information of the input image, feature data channel number information, feature data height information, and feature data width information.
[0061] In practical applications, the feature information of the input image can be extracted through the image feature extraction module. The feature extraction network of this module can be any CNN structure network, such as VGG, ResNet, MobileNet, DarkNet, etc. The image data processed by this module is the image feature data, and its shape can be in the format of [N, C, H, W] (N represents the number of input images, C is the number of channels of feature data, H is the height of feature data, and W is the width of feature data).
[0062] Step 102, performing foreground extraction processing on the image feature data to obtain foreground target feature weight data, and performing image feature compression processing on the image feature data to obtain compressed feature data;
[0063] In a specific implementation, the foreground extraction processing and the image feature compression processing can be respectively implemented through the foreground extraction module and the image feature compression module in the background filtering module, that is, the background filtering module can be designed with two branches, one branch is the foreground extraction module, which is used to extract the target foreground; the other branch is the image feature compression module, which is used to perform high-level extraction and compression of the input image feature information.
[0064] Further, for the convenience of explanation, the relevant formulas involved in the embodiments of the present application are given below and corresponding explanations are given:
[0065] conv_1=Conv2d(K,M,kernel_size,stride=1,padding
[0066] =True) (1-1)
[0067] Among them, K is the number of channels of the input feature map; M is the number of channels of the output feature map, which usually represents the number of targets to be learned; kernel_size is the size of the convolution kernel; stride is the step size of the convolution operation; padding is True, which means that the convolution operation will fill zeros at the edges of the input to maintain the size of the feature map.
[0068] feature_filter=conv_1(X) (1-2)
[0069] It means that the input data X (input feature map, i.e. image feature data) is processed through the convolution layer conv_1 to obtain the feature map feature_filter after preliminary convolution processing.
[0070] feature_weight=Max(Softmax(feature_filter,dim=1),dim
[0071] =1) (1-3)
[0072] Among them, the Softmax function can be applied to each channel of feature_filter to convert the feature value into a probability distribution. The Max function can take the maximum value of these probabilities to produce the weight of the channel that is most likely to represent the foreground at each position. This weight map (feature_weight) can be used to highlight important features in subsequent processing.
[0073] conv_2=Conv2d(K,Q,kernel_size,stride=1,padding
[0074] =True) (1-4)
[0075] It is the definition of the second convolutional layer conv_2, which can be used to further process the input data X and reduce the number of channels of the feature map to extract higher-level semantic information.
[0076] feature_map = conv_2(X) (1-5)
[0077] It can process the input X through the convolution layer conv_2 and output the compressed feature map feature_map, whose number of channels is reduced from K to Q, where Q <K。
[0078] feature_out=feature_map*feature_weight (1-6)
[0079] It can perform element-wise multiplication of the weight map (feature_weight) obtained by the foreground extraction module and the compressed feature map (feature_map) to generate a feature map (feature_out) that finally contains only the foreground target.
[0080]
[0081] Where N is the number of image data input to the system, Z(X i ,θ) is the output of the system target regression module, X i is the i-th picture data input, is the label data corresponding to the input image. It is the mean square error loss function used to evaluate the model output Z(X i ,θ) (predicted density map or point location) and true label The error between .
[0082]
[0083] Where N is the number of image data input to the system, Z(X i ,θ) is the output of the system target regression module, X i is the i-th picture data input, is the number of targets corresponding to the input image data, and abs() is a function that takes an absolute value. It can evaluate the mean absolute error between the predicted number of targets and the actual number of targets. is the actual number of targets, Z(X i ,θ) is the number of predicted targets.
[0084] L total (θ) = L(θ) + α*L num (θ) (1-9)
[0085] This formula means that the total loss function is the weighted sum of the mean square error loss and the target number error loss, where α is a hyperparameter that adjusts the relative importance of the two losses.
[0086] On this basis, in the foreground extraction module, the target category information of the input image can be determined first. Taking crowd counting as an example, the target category of people is the target category actually targeted when the target counting task is performed in this scene. Therefore, the target of the target category of people can be determined as the foreground target, and the targets of other target categories such as trees and vehicles in the target category information can be determined as background targets. Here, it is assumed that the number of complex background target categories is M-1, plus the target category to which the foreground target belongs, a total of M target categories for learning. As shown in formulas 1-1 and 1-2, assuming that the number of channels of the input image feature data is K, after being processed by the roll operator (Conv2d), the results of M activation units can be obtained, so that the feature data of M channels are obtained, which can also be understood as the score data of M targets. The score data of these M targets can be normalized to between 0 and 1 after being processed by the softmax function in the channel dimension, and then the maximum value of the results of the M channels can be taken in the channel dimension, so that a feature map weight data with a shape of [N, 1, H, W] is obtained (as shown in formula 1-3, which can also be called foreground target feature weight data). During the training phase, through the preset objective function, this feature map weight data will learn a template (mask) data of an extracted target (i.e., target template data).
[0087] In the image feature compression module, the number of channels of the input image feature data can be compressed to extract high-level target semantic information. As shown in equations 1-4 and 1-5, assuming that the number of channels of the input feature data is K, after processing by the convolution operator, feature data of Q channels can be obtained (Q <K)。
[0088] In some embodiments of the present invention, performing foreground extraction processing on the image feature data to obtain foreground target feature weight data includes:
[0089] The target category information of the input image and the channel number information of the image feature data are determined, and the image feature data is subjected to foreground extraction processing according to the target category information and the channel number information to obtain the foreground target feature weight data.
[0090] In practical applications, in the foreground extraction module, the target category information of the input image can be determined first. Taking crowd counting as an example, the target category of people is the target category actually targeted when performing target counting tasks in this scene. Therefore, the target of the target category of people can be determined as the foreground target, and the targets of other target categories such as trees and vehicles in the target category information can be determined as background targets. Here, it is assumed that the number of complex background target categories is M-1, plus the target category to which the foreground target belongs, a total of M target categories for learning. As shown in equations 1-1 and 1-2, assuming that the number of channels of the input image feature data is K, after processing by the roll operator (Conv2d), the results of M activation units can be obtained, so that the feature data of M channels are obtained, which can also be understood as the score data of M targets. The score data of these M targets can be normalized to between 0 and 1 after being processed by the softmax function in the channel dimension. Then, the maximum value of the results of the M channels can be taken in the channel dimension, thus obtaining a feature map weight data of the shape [N, 1, H, W] (as shown in Formula 1-3, also called foreground target feature weight data). In the training stage, through the pre-set objective function, this feature map weight data will learn a template (mask) data of the extracted target (i.e., target template data).
[0091] In some embodiments of the present invention, the step of performing image feature compression processing on the image feature data to obtain compressed feature data includes:
[0092] The channel number information of the image feature data is determined, and image feature compression processing is performed on the image feature data according to the channel number information by a preset operator to obtain the compressed feature data.
[0093] In practical applications, the image feature compression module can compress the number of channels of the input image feature data to extract high-level target semantic information. As shown in equations 1-4 and 1-5, assuming that the number of channels of the input feature data is K, after processing by the convolution operator, feature data of Q channels can be obtained (Q <K)。
[0094] Step 103, obtaining foreground target feature data according to the foreground target feature weight data and the compressed feature data, and determining the number of targets in the input image according to the foreground target feature data.
[0095] In the specific implementation, in the foreground extraction module, the target category information of the input image can be determined first. Taking the crowd counting as an example, the target category of people is the target category actually targeted when the target counting task is performed in this scene. Therefore, the target of the target category of people can be determined as the foreground target, and the targets of other target categories such as trees and vehicles in the target category information can be determined as background targets. Here, it is assumed that the number of complex background target categories is M-1, plus the target category to which the foreground target belongs, a total of M target categories for learning. As shown in equations 1-1 and 1-2, assuming that the number of channels of the input image feature data is K, after being processed by the roll operator (Conv2d), the results of M activation units can be obtained, so that the feature data of M channels are obtained, which can also be understood as the score data of M targets. The score data of these M targets can be normalized to between 0 and 1 after being processed by the softmax function in the channel dimension. Then, the maximum value of the results of the M channels can be taken in the channel dimension, thus obtaining a feature map weight data of the shape [N, 1, H, W] (as shown in Formula 1-3, also called foreground target feature weight data). In the training stage, through the pre-set objective function, this feature map weight data will learn a template (mask) data of the extracted target (i.e., target template data).
[0096] The image feature compression module can compress the number of channels of the input image feature data to extract high-level target semantic information. As shown in equations 1-4 and 1-5, assuming that the number of channels of the input feature data is K, after processing by the convolution operator, feature data of Q channels can be obtained (Q <K)。
[0097] Furthermore, as shown in Formula 1-6, the compressed feature data and the target template data can be multiplied element-wise to obtain feature data of a shape of [N, Q, H, W]. In this way, feature data containing only foreground targets (foreground target feature data) is obtained, thereby suppressing background target data to a considerable extent.
[0098] Furthermore, the predicted number of targets can be obtained through the target regression module. Specifically, the target regression module can adopt an algorithm based on density map regression or an algorithm based on point regression. For example, a density map prediction algorithm such as CSRNet can also be a point regression algorithm such as P2PNet. Here, CSRNet is taken as an example. The target regression module can be a Conv2d convolution operator, which can process the foreground target feature data obtained by formula 1-6 and obtain an output target density map of size [1,1,H,W]. Finally, the target density map can be post-processed. The target density map can be interpolated to the same width and height as the input image data based on the original density map information. In the prediction and inference stage, the target density map can be integrated and accumulated to obtain the predicted number of targets in the input image.
[0099] On this basis, the target counting system can perform model training on point regression-based algorithms (such as the P2PNet algorithm). The image data and label information of the training data set can be passed into the training system through the image data input module, where the label information can include the coordinate information of the target (target point coordinates cx, cy).
[0100] The data processing procedures of the image feature extraction module and the background filtering module may be the same as the corresponding procedures in the aforementioned embodiments.
[0101] The data processed by the background filtering model can be input to the target regression module. The processing of the input features by the target regression module depends on the specific algorithm. For example, it can be in the form of a P2PNet algorithm. The target regression module can have two branches, a classification head and a regression head, which are used for classification and coordinate regression of the algorithm candidate points respectively.
[0102] It should be emphasized that in formula 1-1, the number of input channels of the convolution operator is K, the number of output channels is M, the convolution kernel size is kernel_size (usually 3 or 1), the convolution step is 1, and the edge padding is performed. In this way, if the input data size is [1, K, H, W], after being processed by this convolution operator (as shown in formula 1-2), the result (feature_filter) size is [1, M, H, W].
[0103] In formula 1-4, the number of input channels of the convolution operator is K, the number of output channels is Q, the convolution kernel size is kernel_size (usually 3 or 1), the convolution step is 1, and the edge padding is performed. In this way, if the input data size is [1, K, H, W], after being processed by this convolution operator (as shown in formula 1-5), the result (feature_map) size is [1, Q, H, W]. In order to achieve the purpose of channel compression, Q is smaller than K here.
[0104] Formula 1-7 is a target optimization function (Euclidean loss) commonly used in regression branch of regression algorithm. Here N is the number of image data input to the system, Z(X i ,θ) is the output of the system target regression module, X i is the i-th picture data input, It is the label data corresponding to the input image.
[0105] Formula 1-8 is a loss function for calculating the mean absolute error, where N is the number of image data input to the system, and Z(X i ,θ) is the output of the system target regression module, X i is the i-th picture data input, It is the target number corresponding to the input image data, and abs() is a function that takes the absolute value.
[0106] In formula 1-9, α is a hyperparameter between 0 and 1.
[0107] In some embodiments of the present invention, obtaining the foreground target feature data according to the foreground target feature weight data and the compressed feature data includes:
[0108] According to a preset objective function and the foreground target feature weight data, target template data is obtained, and the target template data and the compressed feature data are multiplied to obtain the foreground target feature data.
[0109] In the specific implementation, in the foreground extraction module, the target category information of the input image can be determined first. Taking the crowd counting as an example, the target category of people is the target category actually targeted when the target counting task is performed in this scene. Therefore, the target of the target category of people can be determined as the foreground target, and the targets of other target categories such as trees and vehicles in the target category information can be determined as background targets. Here, it is assumed that the number of complex background target categories is M-1, plus the target category to which the foreground target belongs, a total of M target categories for learning. As shown in equations 1-1 and 1-2, assuming that the number of channels of the input image feature data is K, after being processed by the roll operator (Conv2d), the results of M activation units can be obtained, so that the feature data of M channels are obtained, which can also be understood as the score data of M targets. The score data of these M targets can be normalized to between 0 and 1 after being processed by the softmax function in the channel dimension. Then, the maximum value of the results of the M channels can be taken in the channel dimension, thus obtaining a feature map weight data of the shape [N, 1, H, W] (as shown in Formula 1-3, also called foreground target feature weight data). In the training stage, through the pre-set objective function, this feature map weight data will learn a template (mask) data of the extracted target (i.e., target template data).
[0110] The image feature compression module can compress the number of channels of the input image feature data to extract high-level target semantic information. As shown in equations 1-4 and 1-5, assuming that the number of channels of the input feature data is K, after processing by the convolution operator, feature data of Q channels can be obtained (Q <K)。
[0111] Furthermore, as shown in Formula 1-6, the compressed feature data and the target template data can be multiplied element-wise to obtain feature data of a shape of [N, Q, H, W]. In this way, feature data containing only foreground targets (foreground target feature data) is obtained, thereby suppressing background target data to a considerable extent.
[0112] In some embodiments of the present invention, determining the number of objects in the input image according to the foreground object feature data includes:
[0113] The foreground target feature data is subjected to target regression processing to obtain a target density map, and the number of targets in the input image is determined based on the target density map.
[0114] In practical applications, the target regression module can be used to obtain the predicted number of targets. Specifically, the target regression module can use an algorithm based on density map regression or a point regression algorithm. For example, a density map prediction algorithm such as CSRNet can also be a point regression algorithm such as P2PNet. Here, CSRNet is used as an example. The target regression module can be a Conv2d convolution operator, which can process the foreground target feature data obtained by formula 1-6 and obtain an output target density map of size [1,1,H,W]. Finally, the target density map can be post-processed to obtain the predicted number of targets in the input image.
[0115] In some embodiments of the present invention, the input image includes at least original density map information, and determining the number of targets in the input image according to the target density map includes:
[0116] The target density map is interpolated according to the original density map information, and the target density map after the interpolation is integrated and accumulated to obtain the number of targets in the input image.
[0117] In practical applications, the target regression module can be used to obtain the predicted number of targets. Specifically, the target regression module can use an algorithm based on density map regression or a point regression algorithm. For example, a density map prediction algorithm such as CSRNet can also be a point regression algorithm such as P2PNet. Here, CSRNet is used as an example. The target regression module can be a Conv2d convolution operator, which can process the foreground target feature data obtained by formula 1-6 and obtain an output target density map of size [1,1,H,W]. Finally, the target density map can be post-processed. The target density map can be interpolated to the same width and height as the input image data based on the original density map information. In the prediction and inference stage, the target density map can be integrated and accumulated to obtain the predicted number of targets in the input image.
[0118] In general, the target counting method provided by the above embodiment of the present invention obtains the image feature data of the input image by acquiring the input image and performing feature extraction on the input image; then performs foreground extraction processing on the image feature data to obtain foreground target feature weight data, and performs image feature compression processing on the image feature data to obtain compressed feature data; then obtains foreground target feature data according to the foreground target feature weight data and the compressed feature data, and determines the number of targets in the input image according to the foreground target feature data. The method realizes the suppression of the influence of background data on the counting task by acquiring foreground target feature weight data and compressed feature data and combining and applying them, so that the system can more effectively distinguish the actual counting target (foreground target) from the background target as an interference item, reduces the execution cost of the target counting task, and improves the reasoning prediction efficiency and accuracy of the target counting task.
[0119] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0120] Reference Figure 3 , shows a schematic diagram of the structure of a target counting device provided by some embodiments of the present invention; specifically, it may include the following modules:
[0121] The image feature data acquisition module 301 is used to acquire an input image and perform feature extraction on the input image to obtain image feature data of the input image;
[0122] The foreground target feature weight data acquisition module 302 is used to perform foreground extraction processing on the image feature data to obtain foreground target feature weight data, and perform image feature compression processing on the image feature data to obtain compressed feature data;
[0123] The target number determination module 303 is used to obtain foreground target feature data according to the foreground target feature weight data and the compressed feature data, and determine the number of targets in the input image according to the foreground target feature data.
[0124] In some embodiments of the present invention, the foreground object feature weight data acquisition module 302 includes:
[0125] The foreground target feature weight data acquisition submodule is used to determine the target category information of the input image and the channel number information of the image feature data, and perform foreground extraction processing on the image feature data according to the target category information and the channel number information to obtain the foreground target feature weight data.
[0126] In some embodiments of the present invention, the foreground object feature weight data acquisition module 302 includes:
[0127] The compressed feature data acquisition submodule is used to determine the channel number information of the image feature data, and perform image feature compression processing on the image feature data according to the channel number information through a preset operator to obtain the compressed feature data.
[0128] In some embodiments of the present invention, the target number determination module 303 includes:
[0129] The foreground target feature data acquisition submodule is used to obtain target template data according to a preset target function and the foreground target feature weight data, and to multiply the target template data with the compressed feature data to obtain the foreground target feature data.
[0130] In some embodiments of the present invention, the target number determination module 303 includes:
[0131] The target number determination submodule is used to perform target regression processing on the foreground target feature data to obtain a target density map, and determine the number of targets in the input image based on the target density map.
[0132] In some embodiments of the present invention, the input image includes at least original density map information, and the target number determination submodule includes:
[0133] The target number determination unit is used to perform interpolation processing on the target density map according to the original density map information, and integrate and accumulate the target density map after the interpolation processing to obtain the number of targets in the input image.
[0134] Some embodiments of the present invention further provide an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the target counting method as described above when executed by the processor.
[0135] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the target counting method as described above is implemented.
[0136] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above target counting method when executed by a processor.
[0137] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0138] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0139] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0140] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0143] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0144] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.
[0145] The target counting method and device, electronic device, and storage medium provided are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A target counting method, characterized in that: The method comprises: Acquire an input image, and perform feature extraction on the input image to obtain image feature data of the input image; Performing foreground extraction processing on the image feature data to obtain foreground target feature weight data, and performing image feature compression processing on the image feature data to obtain compressed feature data; Foreground target feature data is obtained according to the foreground target feature weight data and the compressed feature data, and the number of targets in the input image is determined according to the foreground target feature data.
2. The method according to claim 1, characterized in that The performing foreground extraction processing on the image feature data to obtain foreground target feature weight data includes: The target category information of the input image and the channel number information of the image feature data are determined, and the image feature data is subjected to foreground extraction processing according to the target category information and the channel number information to obtain the foreground target feature weight data.
3. The method according to claim 1, characterized in that The performing image feature compression processing on the image feature data to obtain compressed feature data includes: The channel number information of the image feature data is determined, and image feature compression processing is performed on the image feature data according to the channel number information by a preset operator to obtain the compressed feature data.
4. The method according to claim 1, characterized in that: The step of obtaining the foreground target feature data according to the foreground target feature weight data and the compressed feature data includes: According to a preset objective function and the foreground target feature weight data, target template data is obtained, and the target template data and the compressed feature data are multiplied to obtain the foreground target feature data.
5. The method according to claim 1, characterized in that The step of determining the number of objects in the input image according to the foreground object feature data comprises: The foreground target feature data is subjected to target regression processing to obtain a target density map, and the number of targets in the input image is determined based on the target density map.
6. The method according to claim 5, characterized in that The input image at least includes original density map information, and determining the number of targets in the input image according to the target density map includes: The target density map is interpolated according to the original density map information, and the target density map after the interpolation is integrated and accumulated to obtain the number of targets in the input image.
7. The method according to claim 1, characterized in that The image feature data is a feature map including quantity information of the input image, feature data channel number information, feature data height information and feature data width information.
8. A target counting device, characterized in that: The device comprises: An image feature data acquisition module is used to acquire an input image and perform feature extraction on the input image to obtain image feature data of the input image; a foreground target feature weight data acquisition module, used to perform foreground extraction processing on the image feature data to obtain foreground target feature weight data, and perform image feature compression processing on the image feature data to obtain compressed feature data; The target number determination module is used to obtain foreground target feature data according to the foreground target feature weight data and the compressed feature data, and determine the number of targets in the input image according to the foreground target feature data.
9. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the target counting method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the target counting method according to any one of claims 1 to 7 is implemented.