Target counting method and device, storage medium, computer program product and equipment

Abstract

The invention provides a target counting method and device, a storage medium, a computer program product and equipment, and the method comprises the steps: obtaining an input image, wherein the input image comprises a plurality of targets and a position mark for at least one target; determining at least one detection frame according to the position label of the at least one target; extracting an appearance query from the at least one detection frame through RoI pooling, and constructing a shape query based on the width and height of the at least one detection frame; at least fusing the appearance query and the shape query to obtain a target prototype; generating a density map according to the similarity between the target prototype and each pixel position in the input image, wherein each pixel value in the density map represents the probability density of the target at the position; and determining a target total number according to the pixel value of each pixel in the density map. The invention provides a scheme capable of improving the target counting accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a target counting method and device, a storage medium, a computer program product, and equipment. BACKGROUND

[0002] In the quantity statistics of production goods, the traditional target counting technology usually relies on an object detector that needs a large amount of labeled data. In small work orders, which are products for various factories, training each product of each customer separately will generate a huge workload.

[0003] In recent years, with the development of few-shot learning, how to use only a few labeled examples (few-shot) or even no labeling (zero-shot) to complete the task of target counting is a new research direction. Among them, the few-shot counter counts all targets of a certain class through a small number of labeled boxes (samples), and the zero-shot counter counts the most frequent class without labeling.

[0004] However, the counting accuracy of the traditional target counting technology needs to be improved. SUMMARY

[0005] The present application provides a scheme capable of improving the accuracy of target counting.

[0006] In order to achieve the above purpose, the present application provides the following technical scheme:

[0007] In a first aspect, a target counting method is provided, which is applied to a computer device, or a chip in the computer device, or a chip module. The target counting method comprises: obtaining an input image, the input image comprising a plurality of targets and a position label of at least one target; determining at least one detection box according to the position label of the at least one target; extracting an appearance query from the at least one detection box through RoI pooling, and constructing a shape query based on the width and height of the at least one detection box; at least fusing the appearance query and the shape query to obtain a target prototype; generating a density map according to the similarity between the target prototype and each pixel position in the input image, each pixel value in the density map representing the probability density of the existence of a target at the position; and determining a total number of targets according to the pixel value of each pixel in the density map.

[0008] Optionally, the extracting an appearance query from the at least one detection box through RoI pooling comprises: cropping an image feature region corresponding to each detection box and pooling it into a fixed-size tensor as the appearance query.

[0009] Optionally, the fusing the appearance query and the shape query comprises fusing the appearance query and the shape query by multi-head attention.

[0010] Optionally, the target counting method further comprises: fusing the appearance query and the shape query by multi-head attention to obtain an updated query; fusing the updated query and the global image feature of the input image by multi-head attention to obtain a fusion result; and transforming the fusion result by a feedforward network to obtain the target prototype.

[0011] Optionally, the generating a density map according to the similarity between the target prototype and each pixel position in the input image comprises: performing correlation calculation on the target prototype and the global image feature of the input image to obtain a similarity tensor, the similarity tensor representing the matching degree of each position of the input image and the target prototype; and generating the density map according to the similarity tensor.

[0012] Optionally, the determining the target total number according to the pixel value of each pixel in the density map comprises: summing all pixel values in the density map to obtain the target total number.

[0013] Optionally, the determining the target total number according to the pixel value of each pixel in the density map comprises: performing Gaussian smoothing on the density map; performing statistics on the pixel values of the smoothed density map and selecting a value at a first percentile as a threshold; determining a first pixel point higher than the threshold in the smoothed density map; clustering the first pixel point to obtain a plurality of categories and determining the number of categories as the target total number.

[0014] Optionally, the target total number is determined by a counting model, the counting model being obtained by training data, and the training data being constructed in the following manner: obtaining a target data set, the target data set including a plurality of detection boxes, each detection box including a complete target; randomly cropping the detection boxes to obtain cropped detection boxes including partial textures of the target; and adding the target data set and the cropped detection boxes to the training data.

[0015] In a second aspect, the application further discloses a target counting device, comprising: an acquisition module, configured to acquire an input image, wherein the input image comprises a plurality of targets and a position label of at least one target; a bounding box determination module, configured to determine at least one bounding box according to the position label of the at least one target; a query construction module, configured to extract an appearance query from the at least one bounding box through RoI pooling, and construct a shape query based on the width and height of the at least one bounding box; a target prototype determination module, configured to fuse at least the appearance query and the shape query to obtain a target prototype; a density map generation module, configured to generate a density map according to the similarity between the target prototype and each pixel position in the input image, wherein each pixel value in the density map represents the probability density of the target existing at the position; and a counting module, configured to determine the total number of targets according to the pixel value of each pixel in the density map.

[0016] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, wherein the computer program is run by a processing module to execute the steps of the method provided in the first aspect.

[0017] In a fourth aspect, a computer program product is provided, and the computer program product stores a computer program, wherein the computer program is run by a processing module to execute the steps of the method provided in the first aspect.

[0018] In a fifth aspect, an embodiment of the application further provides a chip, and the chip stores a computer program, wherein when the computer program is executed by the chip, the steps of the method provided in the first aspect are implemented.

[0019] In a sixth aspect, an embodiment of the application further provides a system chip applied to a terminal, and the system chip comprises at least one processing module and an interface circuit, wherein the interface circuit and the at least one processing module are interconnected through a circuit, and the at least one processing module is configured to execute instructions to execute the steps of the method provided in the first aspect.

[0020] Compared with the prior art, the technical scheme of the application has the following beneficial effects:

[0021] In the technical scheme of the application, when counting the targets in the input image, the target prototype is constructed by combining the shape information and the appearance information of the target, and when subsequently counting the number of targets based on the target prototype, the target positioning accuracy and the counting accuracy in the low sample scene can be significantly improved. In addition, by separating the shape query and the appearance query, the modeling capability of the geometric characteristics of the target can also be improved.

[0022] Further, the embodiment combines global image features to construct a target prototype, and by introducing global semantic information into the target prototype, the target prototype can dynamically adapt to the global context in the image, thus more accurately representing the common features of the target and suppressing noise interference.

[0023] Further, the embodiment uses a multi-round attention mechanism to dynamically balance shape information and appearance information in multiple rounds of iteration, avoiding dominance by one side. At the same time, the target prototype gradually adapts to the global features of the image, enhancing the generalization to unlabeled targets. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flowchart of a target counting method provided by an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of an input image provided by an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of a density map provided by an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a counting model provided by an embodiment of the present application;

[0028] Figure 5 is a flowchart of another target counting method provided by an embodiment of the present application;

[0029] Figure 6 is a structural schematic diagram of a target counting device provided by an embodiment of the present application;

[0030] Figure 7 is a hardware structural schematic diagram of a target counting device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to facilitate understanding of the technical solutions of the present application, first, the related technologies involved in the present application are briefly introduced.

[0032] Pooling is an operation in convolutional neural networks, mainly used to reduce the spatial size of feature maps, reduce the amount of calculation and the number of parameters, while preserving important features. Common pooling methods include max pooling and average pooling.

[0033] Tensor is a mathematical concept of multi-dimensional array, used to represent data and intermediate results in networks. In the image pooling operation, tensor usually refers to the multi-dimensional data organization form of the input feature map.

[0034] In a convolutional neural network (CNN), an image or a feature map is usually represented as a 4-dimensional tensor with shape: (batch size, height, width, channel number) or (batch size, channel number, height, width). For example, the input tensor of a CNN is (32, 224, 224, 3), which means that there are 32 RGB images in a batch, each image has a resolution of 224x224, and there are 3 color channels. The feature map before pooling is (32, 112, 112, 64), which means that after the convolutional layer, the height and width are reduced to 112x112, and the channel number is expanded to 64. The pooling layer (such as max pooling and average pooling) of the convolutional neural network reduces the spatial dimensions (Height, Width) of the tensor, while the channel dimension (Channels) remains unchanged.

[0035] In the technical solution of the present application, when counting the target in the input image, the target prototype is constructed by combining the shape information and the appearance information of the target. In subsequent target quantity statistics based on the target prototype, the target positioning accuracy and counting accuracy in the low sample scene can be significantly improved. In addition, by separating the shape query and the appearance query, the modeling ability of the target geometric characteristics can also be improved.

[0036] In the embodiment of the present application, the appearance query (Appearance Queries) is used to represent the appearance features of the target, such as color, texture, local pattern, etc. Specifically, the appearance query can be represented as: where sxs represents the spatial dimension, which means that the target region is divided into sxs sub-regions for feature aggregation. d represents the channel number (for example, d=256), which corresponds to the channel dimension of the encoded feature. i identifies the ith detection box.

[0037] In the embodiment of the present application, the shape query is used to represent the geometric characteristics of the target, such as size and proportion. Specifically, it can be generated by nonlinear mapping of the width and height of the detection box, and the shape query can be represented as: .

[0038] In the embodiment of the present application, the global image feature represents the encoded image feature, which contains the semantic information of the whole input image.

[0039] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0040] The embodiment of the present application provides a target counting method, which is described in detail with reference to Figure 1 .

[0041] It can be understood that, in specific implementation, the target counting method can be implemented in the form of a software program running in a processing module integrated in a chip or a chip module. The method can also be implemented in the form of software combined with hardware, and the present application does not make any limitation. The following is introduced taking a computer device as an execution subject as an example.

[0042] In step 101, an input image is obtained, the input image including a plurality of targets, and a position annotation for at least one target.

[0043] In specific implementation, the input image includes a plurality of targets to be counted. Meanwhile, a user can annotate the position of at least one target to be counted in the input image, for example, frame the target. The position annotation can be pixel coordinate information.

[0044] For details, please refer to Figure 2 , Figure 2 An input image is shown. The input image has a plurality of targets, and a user can select one of the targets to frame, as shown in box 201 in Figure 2 . The box 201 includes a complete target. At this time, the position annotation is the pixel coordinate information of the four corners of the box 201.

[0045] It should be noted that Figure 2 only one box 201 is shown in the figure, and in actual application, there can be a plurality of boxes to increase the accuracy of target technology.

[0046] In actual application scenarios, the target can be any implementable article, for example, a process manufactured product. Moreover, the targets to be counted in the input image are usually the same kind of article.

[0047] In step 102, at least one detection box is determined according to the position annotation of at least one target.

[0048] In specific implementation, the pixel coordinate information of at least one target can be used to determine the corresponding detection box, as shown in the box 201 in Figure 2 . The detection box includes a complete target.

[0049] In step 103, an appearance query is extracted from the at least one detection box through RoI pooling, and a shape query is constructed based on the width and height of the at least one detection box.

[0050] In one non-limiting embodiment, the image feature region corresponding to each detection box can be cropped and pooled into a fixed-size tensor as an appearance query. Specifically, a fixed-size feature tensor is extracted from the detection box region annotated by a user through region of interest pooling.

[0051] Embodiments of the present application separate shape and appearance, supplement shape information through shape query, and focus on appearance features through appearance query.

[0052] In one embodiment, referring to Figure 4 , an input image can be input into a counting model 40. The counting model 40 includes an encoding layer 401 and a pooling layer 402.

[0053] The encoding layer 401 converts the input image into a low-dimensional, compact, and semantically rich latent representation. This process provides efficient feature representation for subsequent tasks by progressively extracting and abstracting key features of the input data while filtering out redundant or noisy information. The encoding layer 401 outputs the encoded image features , where h represents the height of the detection box, w represents the width of the detection box, and d represents the number of channels.

[0054] The pooling layer 402 inputs the encoded image features and the detection box provided by the user . The pooling layer 402 crops and pools the image feature region corresponding to the detection box into a fixed-size s x s tensor. Each detection box generates an appearance query , which represents the local appearance features of the target.

[0055] The pooling operation (such as bilinear interpolation or max pooling) unifies detection boxes of different sizes into features of the same spatial dimension, ensuring the shape independence of the appearance query.

[0056] This embodiment extracts visual features such as color and texture of the target region through the encoding layer 401 and the pooling layer 402, which are used for subsequent similarity matching with image features. For example, in a counting scenario, the appearance query can distinguish the target from the background (such as a colored target from a transparent background).

[0057] This embodiment makes the appearance query ignore the absolute position and shape of the target through RoI pooling, focusing on local features, so that it can be generalized to different instances of the same target (such as the same object under different angles or occlusion conditions).

[0058] In one embodiment, the counting model 40 includes the encoding layer 401, the pooling layer 402, and a feedforward network 403. The width w and height h of the detection box are input into the feedforward network 403 to generate a shape query .

[0059] Specifically, the feedforward network 403 can be a three-layer feedforward network, including a hidden layer, an intermediate layer, and an output layer. The shape query is a tensor of sxsxd.

[0060] Exemplarily, if 3s=3, the shape query Each spatial position (3x3) corresponds to a shape feature of different scale.

[0061] In this embodiment, the feedforward network 403 encodes the geometric information (width and height) of the detection box into high-dimensional features, making up for the loss of spatial information caused by RoI pooling and preserving the geometric characteristics of the target. For example, a detection box with a width and height of [50, 30] generates a shape query after mapping, which can represent a "long strip-shaped object" or a "local feature with a specific ratio".

[0062] Continuing to refer to Figure 1 In step 104, at least the appearance query and the shape query are fused to obtain a target prototype.

[0063] In this embodiment, the appearance query and the shape query are respectively reshaped into matrix form ).

[0064] In one non-limiting embodiment, the appearance query and the shape query are fused by multi-head attention to obtain an updated query; the updated query and the global image features of the input image are fused by multi-head attention to obtain a fusion result, further fusing the image context information; the fusion result is transformed by a feedforward network to obtain a target prototype . The target prototype can represent the common appearance and shape characteristics of the same type of target.

[0065] Specifically, the multi-head attention can include multiple (e.g., 8) attention heads, each of which focuses on a different feature subspace (such as color, texture, edge, etc.) to enhance the counting model's ability to capture different types of features.

[0066] The appearance query is a compressed representation of the local visual features of the target, and the shape query is achieved by RoI pooling to achieve shape independence, which is complementary to the shape query. In the iterative adaptation process, the global context is gradually fused, and finally a robust prototype is generated that can accurately match the same type of target in the image. This design significantly improves the counting accuracy and generalization ability in the low sample scenario.

[0067] Compared to traditional methods that directly generate prototypes through feature pooling, ignoring shape information and resulting in low positioning accuracy (e.g., inability to distinguish between similar targets of different sizes), this application's embodiments decouple appearance and geometric information by combining appearance and shape queries. Furthermore, by iteratively adapting and dynamically fusing the target's appearance and shape information, the prototype's adaptability to target diversity is improved.

[0068] Furthermore, this embodiment can determine the target prototype through L iterations. Specifically, it can be done through... Figure 4 The multi-head attention module 404 is used for implementation. Each iteration includes the three steps mentioned above. First, initialization... = (Shape lookup is used as the initial state), L is a positive integer, for example L=3.

[0069] Step 1: Use Multi-Head Attention (MHA) to enable shape lookup. Appearance Inquiry Interactive, shape queries can be adjusted to incorporate local appearance features.

[0070] Updated query =MHA(LN( ), , )+ In this context, MHA represents multi-head attention, l represents the number of rounds, and LN represents layer normalization, which means normalizing the input data to make its mean 0 and variance 1, in order to accelerate training and improve model stability.

[0071] For example, if the shape query indicates "target aspect ratio is 2:1" and the appearance query contains "red circular texture", this step will associate the shape with the appearance (e.g., "red circular object may have a 2:1 aspect ratio").

[0072] Step 2: Update the query With global image features Interactive, incorporating semantic information from the entire image (such as "there are dense similar targets on the right side of the image").

[0073] Fusion results =MHA(LN(Qℓ′), , )+ .

[0074] Step 3: Further optimize the query representation using a feedforward network (FFN) to enhance nonlinear modeling capabilities. Output results of this iteration. =FFN(LN( ))+ .

[0075] After L iterations, the final query is reshaped into n target prototypes Each target prototype encodes shape, local appearance, and global context information.

[0076] In the first iteration, the shape and appearance of the target can be preliminarily associated, but may be affected by local noise (such as background interference). In subsequent iterations, the prototype is gradually corrected by global features, suppressing noise and enhancing the generalization ability to unannotated targets.

[0077] In this embodiment, shape information provides geometric constraints, and appearance information provides visual discrimination, both of which reduce false positives (such as misjudging similar backgrounds as targets).

[0078] Since appearance queries and shape queries are usually generated based on local features (such as the shape and appearance of the labeled detection box), they may not capture the distribution of the target in the global scene (such as the repetition pattern in dense areas); local features may contain background noise, leading to false positives (such as misjudging similar texture backgrounds as targets); the same class of targets may appear in different positions, angles, or occlusion conditions.

[0079] In this embodiment, by introducing global semantic information, the target prototype can dynamically adapt to the global context in the image, thus more accurately representing the common features of the target, suppressing noise interference, helping to distinguish the target from the complex background, and enhancing the generalization to unannotated targets. Further, through non-local calculation of the attention mechanism, the high computational overhead of pixel-by-pixel matching is avoided.

[0080] In the dense small target counting scene, shape queries can encode the size of small targets (such as 5x5 pixels), and appearance queries can extract the texture features of small targets (such as metal reflection), and through global features, it can be found that there are multiple similar areas in the image, enhancing the sensitivity of the prototype to small targets.

[0081] In the zero-shot counting scene, the shape query can be initialized as a learnable general shape template, and the potential shape and appearance of the target can be automatically inferred through global features without manual annotation.

[0082] In step 105, a density map is generated according to the similarity between the target prototype and each pixel position in the input image, and each pixel value in the density map represents the probability density of the existence of the target at that position.

[0083] In one non-limiting embodiment, the target prototype and the global image features of the input image are correlated to obtain a similarity tensor, which represents the matching degree of each position in the input image with the target prototype; and a density map is generated according to the similarity tensor.

[0084] Specifically, the prototype matching module 405 shown in the figure can be implemented. The input of the prototype matching module 405 is n target prototypes Figure 4 , and the encoded image features , n is a positive integer.

[0085] The prototype matching module 405 performs a channel-by-channel correlation calculation on each target prototype and the image features to obtain a similarity tensor .

[0086] The prototype matching module 405 performs a channel-by-channel maximum fusion on the n similarity tensors : R' = MAX( ) (take the maximum value in the channel dimension). The output is a joint response tensor R' , and the high-value area of the joint response tensor R' indicates the center position where the target may exist.

[0087] Further, a density map can be generated using a regression head structure. The joint response tensor R' is input into the regression head structure, and a density map R'' is output . The density map is shown in the figure. Figure 3 The higher the pixel value in the density map, the higher the probability that the target exists at that position. Therefore, the total number of targets to be counted can be determined based on the pixel values of each pixel in the density map. Figure 3

[0088] The regression head structure can specifically be a 3-layer convolution (channel number: 128 → 64 → 32), a bilinear upsampling, and a 1-layer 1x1 convolution.

[0089] The embodiment suppresses the interference of the background or the false response by maximum fusion, retains only the most significant prototype response at each position, and avoids repeated counting of the same target by multiple prototypes. In addition, by retaining the channel dimension, the multi-channel features can be maintained, and rich context information can be provided for the subsequent regression head.

[0090] In one specific embodiment, the number of target prototypes is multiple. Since a single prototype can only represent part of the characteristics of the target (such as a specific size or angle), and there is diversity in real scenes (such as different sizes, occlusions, and light changes). The embodiment of the present application covers multiple potential morphologies of the target through multiple prototypes, and then integrates the responses of all prototypes to ensure that no target instance is missed.

[0091] ​​Specifically, different prototypes focus on local features of the target (e.g., small-size prototypes focus on details, and large-size prototypes focus on overall shape), and the fusion covers a more comprehensive target distribution. In addition, the complementarity of multiple prototypes reduces missed detection (e.g., when a certain prototype fails, other prototypes can still detect the target), and the robustness of target counting is stronger.

[0092] In step 106, the total number of targets is determined according to the pixel value of each pixel in the density map.

[0093] It should be noted that the serial numbers of the steps in the embodiment do not represent the limitation of the execution order of the steps.

[0094] In one specific embodiment, the sum of all pixel values in the density map is obtained to obtain the total number of targets N, N=sum(R'').

[0095] Specifically, the sum of the density map is equivalent to the probability integral of the number of targets (e.g., a single target corresponds to a Gaussian peak in the density map, and the integral value is 1). If the model prediction is accurate, each target corresponds to a Gaussian peak with an approximate area of 1, and the sum is the target number. The representation of each target in the density map is an unnormalized Gaussian kernel, and the integral is equal to 1. The sum directly reflects the target number rather than the probability distribution. The sum of the density map is essentially the accumulation of the Gaussian kernel area of each target, which is equal to the target number.

[0096] The embodiment is based on the integral characteristics of the density map, and simply and efficiently realizes the estimation of the number of targets. In addition, the response tensor is mapped to the probability density map, which can intuitively represent the target distribution.

[0097] In one specific application scenario, the input image contains multiple doll toys (partially occluded and different sizes). The appearance query can capture the texture features of small-size doll toys. The shape query can reflect the shape information of large-size doll toys. In the fused target prototype, the response area of both small-size doll toys and large-size doll toys is retained. In the generated density map, each doll toy corresponds to a high-density area. The sum of the density map is obtained to obtain the total number of doll toys.

[0098] In another specific embodiment, the pixel points in the density map can be screened and clustered to enhance the accuracy of counting. For details, refer to Figure 5 , Figure 5 A specific counting method is shown.

[0099] In step 501, the density map is Gaussian smoothed. By Gaussian smoothing, noise in the density map can be removed, and the influence of noise can be reduced. Gaussian smoothing can be performed using a Gaussian kernel, for example, the Gaussian kernel core size is (31, 31), and the standard deviation is set to automatic.

[0100] In step 502, statistics are performed on the pixel values in the smoothed density map, and a value at a first percentile is selected as a threshold. For example, the first percentile is 90.

[0101] In step 503, first pixel points in the smoothed density map that are higher than the threshold are determined.

[0102] That is, only the first pixel points with pixel values greater than the threshold are considered as key points and participate in subsequent clustering.

[0103] In step 504, the first pixel points are clustered to obtain multiple categories, and the number of categories is determined as the target total number.

[0104] In this embodiment, the pixel points in the region where each target is located are clustered into one category through clustering, and the number of categories formed by clustering is the target total number to be counted.

[0105] Compared with directly summing all pixel values in the density map, the embodiment can re-cluster the pixel points segmented in the same target together through clustering of the first pixel points in the density map, further ensuring the accuracy of counting.

[0106] In addition, in actual applications, the target in the input image may not be complete, for example, only half of the target. In order to enhance the recognition of the counting model for these targets, the counting model can be trained using optimized training data.

[0107] In one non-limiting embodiment, the training data is constructed in the following manner:

[0108] Obtain a target data set, which includes multiple detection boxes, and each detection box includes a complete target;

[0109] Randomly crop the detection boxes, and the cropped detection boxes include partial textures of the target;

[0110] Add the target data set and the cropped detection boxes to the training data.

[0111] The embodiment optimizes the training data so that the training data includes the cropped detection boxes, so that the training data includes sample data with partial textures of the target. Therefore, training the counting model using the optimized training data can enhance the ability of the counting model to detect partial targets in the image.

[0112] Further, when counting, the counting model can use a non-maximum suppression algorithm to remove overlapping targets for repeated results of the same target, and count on this basis.

[0113] Please refer to Figure 6 , Figure 6A target counting apparatus 60 is shown, which can include:

[0114] An acquisition module 601 is configured to acquire an input image, the input image including a plurality of targets and a position label of at least one target;

[0115] A bounding box determination module 602 is configured to determine at least one bounding box according to the position label of the at least one target;

[0116] A query construction module 603 is configured to extract an appearance query from the at least one bounding box by RoI pooling and construct a shape query based on a width and a height of the at least one bounding box;

[0117] A target prototype determination module 604 is configured to fuse the appearance query and the shape query to obtain a target prototype;

[0118] A density map generation module 605 is configured to generate a density map according to a similarity between the target prototype and each pixel position in the input image, each pixel value in the density map representing a probability density of the target existing at the position;

[0119] A counting module 606 is configured to determine a total number of targets according to pixel values of each pixel in the density map.

[0120] In specific implementations, the target counting apparatus 60 described above can correspond to a chip with a communication function in a computer device, such as a System-On-a-Chip (SOC), a baseband chip, etc.; or a chip module including a chip with a communication function in a computer device; or a chip module including a chip with a data processing function, or a computer device.

[0121] Other related descriptions of the target counting apparatus 60 can refer to the related descriptions in the foregoing embodiments, which will not be described herein again.

[0122] The various modules / units included in the various devices and products described in the above embodiments can be software modules / units or hardware modules / units, or can be partially software modules / units and partially hardware modules / units. For example, for the various devices and products applied to or integrated into a chip, the various modules / units included therein can all be implemented in the form of hardware such as circuitry, or at least some of the modules / units can be implemented in the form of software programs running on a processing module integrated into the chip, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuitry; for the various devices and products applied to or integrated into a chip module, the various modules / units included therein can all be implemented in the form of hardware such as circuitry, and different modules / units can be located in the same component (e.g., a chip, a circuit module, etc.) or different components of the chip module, or at least some of the modules / units can be implemented in the form of software programs running on a processing module integrated into the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuitry; for the various devices and products applied to or integrated into a computer device, the various modules / units included therein can all be implemented in the form of hardware such as circuitry, and different modules / units can be located in the same component (e.g., a chip, a circuit module, etc.) or different components of the computer device, or at least some of the modules / units can be implemented in the form of software programs running on a processing module integrated into the computer device, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuitry.

[0123] The embodiments of the present application further disclose a storage medium, which is a computer-readable storage medium, and has a computer program stored thereon, where the computer program, when executed, can perform the steps of the method shown in the above embodiments. The storage medium can include a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc. The storage medium can also include a non-volatile memory or a non-transitory memory, etc.

[0124] For the sake of clarity, Figure 7 The embodiments of the present application further provide a hardware structure diagram of the target counting device. The device includes a processing module 701, a storage module 702, and a transceiver 703.

[0125] The processing module 701 can be a general central processing module (CPU), a micro processing module, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the program execution of the solutions of the present application. The processing module 701 can also include multiple CPUs, and the processing module 701 can be a single-CPU processing module or a multi-CPU processing module. The processing module herein can refer to one or more devices, circuits or processing cores for processing data (for example, computer program instructions).

[0126] The storage module 702 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magneto-optical disk storage (including compact disks, laser disks, optical disks, digital versatile disks, Blu-ray disks, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, and the embodiments of the present application do not make any limitation thereto. The storage module 702 can exist independently (in this case, the storage module 702 can be located outside the device or inside the device), or can be integrated with the processing module 701. The storage module 702 can contain computer program code. The processing module 701 is configured to execute the computer program code stored in the storage module 702, so as to implement the methods provided by the embodiments of the present application.

[0127] The processing module 701, the storage module 702 and the transceiver 703 are connected through a bus. The transceiver 703 is configured to communicate with other devices or communication networks. Optionally, the transceiver 703 can obtain the input image through an interface.

[0128] When Figure 7 The structure diagram shown in the figure is used to illustrate the structure of the computer device involved in the above embodiments, and the processing module 701 is configured to control and manage the actions of the computer device. For example, the processing module 701 is configured to support the computer device to perform the actions performed by the computer device in the other processes described in the embodiments of the present application.

[0129] It should be understood that the term "and / or" in this document merely describes an associated relationship between associated objects, and means that three relationships can exist, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in this document means that the front and rear associated objects are in an "or" relationship.

[0130] "Multiple" appearing in the embodiments of the present application means two or more.

[0131] The first, second, and the like appearing in the embodiments of the present application are only used for description and distinction of the described objects, and there is no order difference, nor does it represent a special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0132] The "connection" appearing in the embodiments of the present application means direct connection or indirect connection and various connection modes to realize communication between devices, and the embodiments of the present application do not make any limitation on this.

[0133] The above embodiments can be realized all or partially by software, hardware, firmware or other any combination. When realized by software, the above embodiments can be realized all or partially in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired or wireless mode.

[0134] It should be understood that in various embodiments of the present application, the size of the serial number of the above processes does not mean the order of execution, and the execution order of the processes should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0135] In several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other manners. For example, the described device embodiments are merely illustrative; the division of the units is merely logical function division; and other division manners can be used in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0136] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0137] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can be a physical unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of hardware plus software function units.

[0138] The integrated unit implemented in the form of software function units can be stored in a computer readable storage medium. The software function unit stored in the storage medium includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute some steps of the methods described in the embodiments of the present application.

[0139] Although the present application is disclosed as above, the present application is not limited thereto. Any person skilled in the art, without departing from the spirit and scope of the present application, can make various modifications and changes, and therefore the protection scope of the present application should be subject to the scope defined by the claims.

Claims

1. A target counting method characterized by comprising: The method comprises: obtaining an input image, the input image comprising a plurality of targets and position labels of at least one target; determining at least one detection box according to the position labels of the at least one target; extracting an appearance query from the at least one detection box through RoI pooling, and constructing a shape query based on the width and height of the at least one detection box; fusing at least the appearance query and the shape query to obtain a target prototype; generating a density map according to the similarity between the target prototype and each pixel position in the input image, each pixel value in the density map representing the probability density of the target existing at the position; determining the total number of targets according to the pixel values of each pixel in the density map.

2. The target counting method according to claim 1, wherein The method of extracting an appearance query from the at least one detection box through RoI pooling comprises: cropping the image feature region corresponding to each detection box and pooling it into a fixed-size tensor as the appearance query.

3. The target counting method according to claim 1, characterized by, The method of fusing at least the appearance query and the shape query comprises: fusing the appearance query and the shape query through multi-head attention.

4. The target counting method according to claim 1, characterized by, The method further comprises: fusing the appearance query and the shape query through multi-head attention to obtain an updated query; fusing the updated query and the global image features of the input image through multi-head attention to obtain a fusion result; transforming the fusion result through a feedforward network to obtain the target prototype.

5. The target counting method according to claim 1, wherein The method of generating a density map according to the similarity between the target prototype and each pixel position in the input image comprises: performing correlation calculation on the target prototype and the global image features of the input image to obtain a similarity tensor, the similarity tensor representing the matching degree of each position in the input image with the target prototype; generating the density map according to the similarity tensor.

6. The target counting method according to claim 5, wherein The method of determining the total number of targets according to the pixel values of each pixel in the density map comprises: summing all pixel values in the density map to obtain the total number of targets.

7. The target counting method according to claim 5, wherein The method of determining the total number of targets according to the pixel values of each pixel in the density map comprises: performing Gaussian smoothing on the density map; statistically analyzing the pixel values in the smoothed density map and selecting a value at the first percentile as a threshold value; determining first pixel points in the smoothed density map that are higher than the threshold value; clustering the first pixel points to obtain a plurality of categories and determining the number of categories as the total number of targets.

8. The target counting method according to claim 1, wherein The total number of targets is determined through a counting model, the counting model being obtained using training data, and the training data being constructed in the following manner: obtaining a target data set, the target data set comprising a plurality of detection boxes, each detection box comprising a complete target; randomly cropping the detection boxes, the cropped detection boxes comprising part of the texture of the target; adding the target data set and the cropped detection boxes to the training data.

9. A target counting device, characterized by The method comprises: an obtaining module, configured to obtain an input image, the input image comprising a plurality of targets and position labels of at least one target; a detection box determination module, configured to determine at least one detection box according to the position labels of the at least one target; a query construction module configured to extract an appearance query from the at least one bounding box by RoI pooling, and construct a shape query based on the width and height of the at least one bounding box; a target prototype determination module configured to fuse at least the appearance query and the shape query to obtain a target prototype; a density map generation module configured to generate a density map according to the similarity between the target prototype and each pixel position in the input image, wherein each pixel value in the density map represents the probability density of the target existing at the position; a counting module configured to determine the total number of targets according to the pixel value of each pixel in the density map.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is run by a processor to perform the steps of the target counting method in any one of claims 1 to 8.

11. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by a computer to implement the steps of the target counting method in any one of claims 1 to 8.

12. A computer device, comprising a storage module and a processing module, wherein the storage module stores a computer program capable of running on the processing module, and the computer device is characterized in that, The processing module performs the steps of the target counting method in any one of claims 1 to 8 when the computer program is run.

Images