A method and apparatus for counting and locating target objects

Abstract

This application provides a method and apparatus for counting and locating target objects. Relating to the field of data processing technology, the method first inputs a target image of a target scene into the backbone network of a counting network model, obtaining the output results of each of the M stages included in the backbone network. The counting network model is trained based on a training density map constructed using a kernel function. Then, the output results of the M stages are sampled to obtain a dimensionally unified output result. This dimensionally unified output result is then input into the fusion module of the counting network model to obtain a first density map. Finally, the density contributions of the first density map are summed to obtain the number of target objects in the target image. Thus, the number of target objects captured within a target image is counted.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for counting and locating target objects. Background Technology

[0002] With the hosting of various large-scale events, pedestrian-dense scenes are becoming increasingly common, such as concert venues, popular tourist attractions, and large commercial districts, where crowds are dense and flow is high. Real-time acquisition of crowd information is of great significance for safety monitoring, crowd control, and resource allocation.

[0003] Currently, people are often counted manually or by counting based on sensors. However, this method cannot meet the requirements of real-time performance and accuracy when dealing with large and dynamically changing crowds. Summary of the Invention

[0004] In view of this, this application provides a method and apparatus for counting and locating target objects, which aims to achieve real-time and accurate counting of the number of people in a crowd.

[0005] In a first aspect, this application provides a method for counting and locating target objects, including:

[0006] The target image of the target scene is input into the backbone network of the counting network model, and the output results of each of the M stages included in the backbone network are obtained. The counting network model is trained based on the training density map constructed by the kernel function.

[0007] The output results of the M stages are sampled to obtain a dimensionally unified output result;

[0008] The output result of the dimension unification is input into the fusion module of the counting network model to obtain the first density map;

[0009] The number of target objects in the target image is obtained by summing the density contributions of the first density map.

[0010] Optionally, the step of sampling the output results of the M stages to obtain dimensionally unified output results includes:

[0011] The dimension of the output result of the target stage is used as the target dimension, and the target stage is one of the M stages;

[0012] For the first stage in the backbone network that obtains the output result before the target stage, the output result of the first stage is downsampled to obtain the output result of the first stage of the target dimension. The first stage is one of the M stages.

[0013] For the second stage in the backbone network that obtains the output result after the target stage, the output result of the second stage is upsampled to obtain the output result of the second stage in the target dimension. The second stage is one of the M stages.

[0014] Optionally, the method further includes:

[0015] The first density map is clustered using a clustering algorithm to determine the location of target objects in the target scene, and the contour coefficient of the clustering algorithm is the number of target objects.

[0016] Optionally, the construction steps of the counting network model include:

[0017] Construct a dataset, which includes training images and training density maps corresponding to each training image, wherein the training images contain target objects;

[0018] Establish the network architecture of the aforementioned counting network model;

[0019] Establish the loss function for the counting network model;

[0020] The counting network model is trained based on the dataset to obtain the trained counting network model.

[0021] Optionally, the network architecture for establishing the counting network model includes:

[0022] Set up a backbone network consisting of N stages;

[0023] M stages are determined from the N stages, and the output of the target stage among the M stages is connected to the fusion module. The target stage is one of the M stages.

[0024] For the first stage that obtains the output result before the target stage of the backbone network, the output result of the first stage is connected to the fusion module after being downsampled. The first stage is one of the M stages. The downsampling module unifies the dimension of the output result of the first stage with the dimension of the output result of the target stage.

[0025] For the second stage that obtains the output result after the target stage of the backbone network, the output result of the second stage is connected to the fusion module after being passed through the upsampling module. The second stage is one of the M stages. The upsampling module unifies the dimension of the output result of the second stage with the dimension of the output result of the target stage. The fusion module fuses the input feature maps and outputs a second density map with 1 channel.

[0026] Optionally, the construction of the dataset includes:

[0027] Acquire multiple training images for training purposes;

[0028] Label the center point of each target object in the training image to obtain a labeled image;

[0029] Based on the label data of the center points of the target objects in the labeled image, the labeled image is converted into a training density map using a Gaussian kernel function;

[0030] The training images are divided into a training set, a validation set, and a test set in a ratio of 7:2:1. Each training image in the training set, validation set, and test set corresponds to a training density map obtained by converting the training image.

[0031] Optionally, the step of converting the labeled image into a training density map using a Gaussian kernel function based on the label data of the center points of the labeled objects in the labeled image includes:

[0032] A two-dimensional matrix with the same size as the labeled image is created to store the training density map, and all elements of the two-dimensional matrix are initialized to zero;

[0033] For each target center point annotated in the labeled image, the pixels on the training density map are traversed. The density contribution of each target center point to each traversed pixel is calculated using a Gaussian kernel function. The density contributions of all center points in the labeled image to each pixel in the training density map are accumulated as the value of the corresponding position in the training density map, thus forming the training density map.

[0034] Optionally, the loss function is the mean squared error; the loss function for establishing the counting network model includes:

[0035] Based on the density contribution of each pixel in the training density map corresponding to the training image and the second density map of the training image, the loss between the training density map and the second density map is quantified by mean square error, so as to minimize this loss through the backpropagation algorithm to train the counting network model.

[0036] Optionally, the method further includes:

[0037] The target image of the target scene is obtained by capturing it using a high-point camera.

[0038] Secondly, this application also provides a target object counting and positioning device, the device comprising:

[0039] The first processing unit is used to input the target image of the target scene into the backbone network of the counting network model, and obtain the output results of each of the M stages included in the backbone network. The counting network model is trained based on the training density map constructed by the kernel function.

[0040] The second processing unit is used to sample the output results of the M stages to obtain output results with uniform dimensions.

[0041] The fusion unit is used to input the dimension-unified output result into the fusion module of the counting network model to obtain the first density map;

[0042] The calculation unit is used to sum the density contributions of the first density map to obtain the number of target objects in the target scene.

[0043] Thirdly, this application provides an apparatus comprising a memory and a processor, the memory for storing instructions or code, and the processor for executing the instructions or code to cause the apparatus to perform a target counting and positioning method as described in any of the first aspects above.

[0044] Fourthly, this application provides a computer storage medium storing code, wherein when the code is executed, a device running the code implements a target counting and positioning method as described in any of the first aspects above.

[0045] This application provides a method and apparatus for counting and locating target objects. When executing the method, the target image of the target scene is first input into the backbone network of a counting network model. The output results of each of the M stages included in the backbone network are obtained. The counting network model is trained based on a training density map constructed using a kernel function. Then, the output results of the M stages are sampled to obtain a dimensionally uniform output result. This dimensionally uniform output result is then input into the fusion module of the counting network model to obtain a first density map. Finally, the density contributions of the first density map are summed to obtain the number of target objects in the target image, thereby achieving the statistical analysis of the number of target objects within the shooting range of the target image acquired from the target scene. In this way, by training the counting network model based on a density map constructed using a kernel function, the distribution of target object density is learned to indirectly calculate the number of target objects. Furthermore, the target image acquires feature maps from multiple stages through the backbone network. These feature maps are then fused through sampling and fusion modules to obtain a first density map reflecting the density of the target objects. In this way, even when the density of the target objects is high and they are mutually occluded, the feature information from multiple different stages is fully utilized, and the features from multiple stages are fused to enhance the model's adaptability to complex scenes and changes, improve the model's prediction accuracy, and achieve accurate counting of the number of target objects captured in the target image. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating a target object counting and positioning method provided in an embodiment of this application;

[0048] Figure 2 This is a schematic diagram of the architecture of a counting network model provided in an embodiment of this application;

[0049] Figure 3 This is a schematic diagram of the structure of a target counting and positioning device provided in an embodiment of this application. Detailed Implementation

[0050] Currently, when dealing with large and dynamically changing crowds, counting using manual techniques and sensor sensing methods cannot meet the requirements for real-time performance and accuracy. For large-scale scenarios, image recognition can be used to identify individuals within images. This often involves multi-camera stitching technology to combine images into a single image for head or body recognition to count people. However, this method is costly in terms of hardware, and image stitching and calibration increase system complexity and dimensionality. Furthermore, even with successful stitching, the high crowd density, significant occlusion, and individual differences make it difficult for recognition algorithms to distinguish and locate each individual, leading to missed and false detections. It also requires high-performance hardware, such as high-performance calculators and dedicated image recognition equipment, resulting in high costs.

[0051] To address the aforementioned issues, this application proposes a method and apparatus for target object counting and localization. The method trains a counting network model using a density map constructed based on a kernel function, learning the distribution of target object density to indirectly calculate the number of targets. The target image obtains feature maps at multiple stages through the backbone network of the counting network model. These feature maps are then fused using a sampling and fusion module to obtain a first density map reflecting the target object density, thereby enabling the statistical analysis of the number of targets within the target image's capture range. Thus, even in situations where the target object group density is high and mutual occlusion exists, the method can not only focus on the identification of individual targets but also analyze the entire target object group through density estimation, thereby achieving accurate counting of the number of targets.

[0052] Furthermore, this application employs a high-point camera, deploying a high-definition camera at a high point, such as by using a drone to take an overhead shot, or by using a high building or rooftop, to ensure that the high-point camera can obtain a sufficiently wide field of view, thereby reducing blind spots and helping to reduce the difficulty of recognition caused by individual height differences and occlusion.

[0053] Furthermore, in order to locate the target object, this application introduces a clustering algorithm, which automatically divides different clustering regions according to the density changes, thereby achieving a rough location of the individual.

[0054] The method provided in this application embodiment is executed by a processor and is used to analyze and process target images captured by a high-point camera on a target scene.

[0055] It should be noted that the embodiments of this application do not limit the execution subject of the processor. For example, a target counting and positioning method of this application embodiment can be applied to data processing devices such as terminal devices or servers. The terminal device can be a smartphone, computer, personal digital assistant (PDA), or tablet computer. The server can be a standalone server, a cluster server, or a cloud server.

[0056] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0057] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0058] Unless otherwise stated, the term "multiple" means two or more.

[0059] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0060] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0062] See Figure 1 , Figure 1 This application provides a flowchart illustrating a target object counting and positioning method, which includes:

[0063] S101. Input the target image of the target scene into the backbone network of the counting network model, and obtain the output results of each of the M stages included in the backbone network. The counting network model is trained based on the training density map constructed by the kernel function.

[0064] Optionally, the target scene described above is an image of the entire target scene obtained by taking a bird's-eye view of the scene from a high point, so as to facilitate subsequent analysis of the number of target objects within the target scene.

[0065] Optionally, the backbone network of the counting network model may include N stages, where M is a positive integer less than or equal to N. Thus, obtaining the output of each of the M stages means identifying M stages from all N stages in the backbone network and obtaining the feature map output of each of these M stages.

[0066] Optionally, the kernel function mentioned above can be a Gaussian kernel function, and the training density map mentioned above is generated based on the labeled training images using the Gaussian kernel function. In this way, the counting network model is trained using the training images and the corresponding training density map, enabling the counting network model to learn the distribution of target object density in order to subsequently calculate the number of target objects captured in the target image.

[0067] S102. Sample the output results of the M stages to obtain output results with uniform dimensions.

[0068] Understandably, the backbone network consists of N sequentially connected stages. The feature map output by each stage has a smaller dimension than the feature map of the previous stage. Therefore, in order to achieve the fusion of feature maps of the subsequent M stages, the feature maps of the current M stages need to be sampled (e.g., upsampling and downsampling) to unify the dimensions of the multiple feature map bitmaps.

[0069] S103. Input the output result of the dimension unification into the fusion module of the counting network model to obtain the first density map.

[0070] The fusion module combines multiple feature maps, ultimately outputting a single feature map with only one channel. This allows for full utilization of feature information from multiple different stages, improving the model's prediction accuracy. Furthermore, the fusion of features from multiple stages enhances the model's adaptability to complex scenarios and changes.

[0071] S104. Sum the density contributions of the first density map to obtain the number of target objects in the target image.

[0072] Each pixel in the first density map can correspond to a density contribution, which reflects the probability that the pixel is the target object.

[0073] Based on steps S101-S104 above, this application trains the counting network model using a density map constructed based on a kernel function, learning the distribution of target object density to indirectly calculate the number of target objects. Furthermore, the target image obtains feature maps from multiple stages through the backbone network, fully utilizing feature information from multiple different stages to improve the model's prediction accuracy. Simultaneously, the feature maps from multiple stages are fused through sampling processing and a fusion module to obtain a first density map reflecting the target object density, integrating features from multiple stages and enhancing the model's adaptability to complex scenes and changes. Thus, even in situations where the target object group density is high and there is mutual occlusion, it can not only focus on the identification of individual target objects but also analyze the entire target object group through density estimation, achieving accurate counting of the number of target objects.

[0074] In the embodiments of this application, the above Figure 1 There are several possible implementations of step S102, which will be described below. It should be noted that the implementations given below are merely illustrative examples and do not represent all implementations of the embodiments of this application.

[0075] The above sampling process of the output results of the M stages yields a dimensionally unified output result, including:

[0076] First, the dimension of the output result of the target stage is used as the target dimension, and the target stage is one of the M stages.

[0077] For example, the counting network model can be configured with N stages, which can be four stages: stage1, stage2, stage3, and stage4. M stages can be selected from these N stages, namely stage2, stage3, and stage4. Optionally, stage3 can be chosen as the target stage, with the dimension of the output (feature map) of stage3 used as the target dimension, and the output (feature map) of stage3 can be directly input into the fusion module in step S103.

[0078] Then, for the first stage in the backbone network that obtains output results before the target stage, the output results of the first stage are downsampled to obtain the output results of the first stage in the target dimension, where the first stage is one of the M stages; for the second stage in the backbone network that obtains output results after the target stage, the output results of the second stage are upsampled to obtain the output results of the second stage in the target dimension, where the second stage is one of the M stages.

[0079] Understandably, the dimension of the first stage (stage2) that obtains the output result before the target stage (stage3) is higher than the dimension of the target stage (stage3). Therefore, the first stage (stage2) is downsampled to obtain a feature map with the same dimension as the target stage (stage3).

[0080] Since the dimension of the second stage (stage4) that obtains the output result after the target stage (stage3) is lower than the dimension of the target stage (stage3), the second stage (stage4) is upsampled to obtain a feature map with the same dimension as the target stage (stage3).

[0081] In this way, by sampling, the dimensions of the output results of the M stages are unified, resulting in a feature map with unified M dimensions. This allows for the full utilization and fusion of feature information from different stages, thereby improving the prediction accuracy of the model.

[0082] Optionally, after step S104, the method further includes:

[0083] The first density map is clustered using a clustering algorithm to determine the location of target objects in the target scene, and the contour coefficient of the clustering algorithm is the number of target objects.

[0084] The density map is summed to obtain the total number of target objects. Using a clustering algorithm, such as K-Means, the total number of people is set as the silhouette coefficient (n_clusters), resulting in multiple center points corresponding to the number of target objects, which are then used as the locations of the target objects.

[0085] Optional, see Figure 2 The diagram shows an architecture of a counting network model. The construction steps of the aforementioned counting network model include:

[0086] First, a dataset is constructed, which includes training images and training density maps corresponding to each training image, wherein the training images contain target objects.

[0087] The above training images are obtained by manually labeling the center point of each target object in the training image. Then, based on the label data of the center points of the target objects in the labeled image (for example, label 1 indicates the probability that the center point is a target object), the labeled image is converted into a training density map by using a Gaussian kernel function.

[0088] Then, the network architecture of the counting network model is established.

[0089] Optionally, the network architecture of the above-mentioned technical network model mainly consists of a backbone network and a fusion module to obtain a density map reflecting the number of target objects and calculate the number of target objects. In addition, the output of the technical network model can also be combined with a clustering algorithm to locate the target objects in the density map and obtain the target object location results.

[0090] Secondly, the loss function of the counting network model is established.

[0091] Optionally, the loss function can be the mean squared error.

[0092] Finally, the counting network model is trained based on the dataset to obtain the trained counting network model.

[0093] Optionally, the dataset can be divided into a training set, a validation set, and a test set. The counting network model is trained using the training set. The difference between the training density map (determined by a Gaussian kernel function for the training image) and the second density map output by the counting network model for the same training image is quantified using mean squared error, and the model parameters are updated based on this difference. The model is trained iteratively, and its performance is evaluated using the validation set after each training round to obtain the optimal model parameters. Finally, based on the optimal model parameters, the performance of the counting network model on the test set is evaluated, including accuracy and recall. If the model performance meets the expected target, it can be deployed; otherwise, the model architecture or parameters are adjusted, and training is repeated.

[0094] Optionally, when deploying the aforementioned network model, the finally trained model can be deployed in the application environment, such as a server or embedded device. Then, an input interface is configured for the model to receive actual monitoring images or video streams (for video streams, frames are extracted to obtain images), and an output interface is configured for the model to output analysis results. Next, stress testing is performed on the deployment environment to ensure that the system with the counting network model deployed can operate stably. Furthermore, the system's operating status can be monitored, and abnormal events and log information can be recorded.

[0095] Subsequently, performance bottlenecks or anomalies in the model and system can be analyzed based on actual applications. The dataset and model can also be updated periodically to adapt to new types of anomalies and detection requirements. Optionally, for the counting network model, optimizations can be made in aspects such as model architecture, parameters, and data augmentation. For the system, optimizations can be made in aspects such as improving data loading methods and allocating computational resources.

[0096] See Figure 2 In one specific implementation, the network architecture for establishing the aforementioned counting network model may specifically include:

[0097] First, set up a backbone network consisting of N stages.

[0098] For example, see Figure 2 It has a backbone network with four stages: stage1, stage2, stage3 and stage4.

[0099] Then, M stages are determined from the N stages, and the output of the target stage among the M stages is connected to the fusion module. The target stage is one of the M stages.

[0100] For example, see Figure 2 From the N stages, M stages are determined as stage2, stage3, and stage4. Stage3 is the output directly connected to the fusion module.

[0101] Secondly, for the first stage that obtains the output result before the target stage of the backbone network, the output result of the first stage is connected to the fusion module after being downsampled. The first stage is one of the M stages. The downsampling module unifies the dimension of the output result of the first stage with the dimension of the output result of the target stage.

[0102] For example, see Figure 2 The output of stage2 (feature map) is downsampled and then input into the fusion module.

[0103] Finally, for the second stage after the target stage of the backbone network, the output of the second stage is passed through the upsampling module and then connected to the fusion module. The second stage is one of the M stages. The upsampling module unifies the dimension of the output of the second stage with the dimension of the output of the target stage. The fusion module fuses the input feature maps and outputs a second density map with 1 channel.

[0104] Optionally, the fusion module can also be connected to a counting module, which is used to sum the density contributions of the density map output by the fusion module to obtain the number of target objects in the density map.

[0105] Optionally, the output of the fusion module can also be configured with a clustering algorithm, which is used to cluster the density map of the fusion module to determine the location of the target object in the target scene.

[0106] For example, see Figure 2 The output (feature map) of stage4 is upsampled and then input into the fusion module.

[0107] Therefore, this application selects and fuses the feature maps from the last three stages (stage 2, stage 3, and stage 4) to generate a new feature map. In the fusion module, these feature maps are effectively fused, ultimately outputting a feature map with only one channel. This application can fully utilize the feature information from different stages, improving the model's prediction accuracy. Simultaneously, due to the fusion of features from multiple stages, the model's adaptability to complex scenes and changing conditions is significantly enhanced.

[0108] See Figure 2 In one specific implementation, the above-mentioned dataset construction includes:

[0109] First, acquire multiple training images for training.

[0110] The training images mentioned above can be sourced from publicly available video or image materials. For videos, frame extraction can be used to obtain training images. Alternatively, a dataset containing training images can be obtained directly.

[0111] Next, labels are added to the center point of each target object in the training image to obtain a labeled image.

[0112] For example, the target object can be a person, so that the center point of the head of each person in the training image can be labeled.

[0113] Then, based on the label data of the center points of the labeled objects in the labeled image, the labeled image is converted into a training density map using a Gaussian kernel function.

[0114] Specifically, a two-dimensional matrix of the same size as the labeled image can be first established to store the training density map, with all elements of the two-dimensional matrix initialized to zero. Then, for the center point of each object labeled in the labeled image, the pixels on the training density map are traversed, and the density contribution of each object center point to each traversed pixel is calculated using a Gaussian kernel function. The density contributions of all center points in the labeled image to each pixel in the training density map are accumulated as the value of the corresponding position in the training density map, thus forming the training density map.

[0115] The formula for calculating the Gaussian kernel function mentioned above can be:

[0116]

[0117] Where x is the coordinate of any pixel on the training density map, x i σ represents the coordinates of the i-th center point in the label data, σ is the standard deviation of the Gaussian kernel, which determines the smoothness of the density map, and ||xx i || is x and x i The Euclidean distance between them.

[0118] Finally, the training images are divided into a training set, a validation set, and a test set in a ratio of 7:2:1. Each training image in the training set, validation set, and test set corresponds to a training density map obtained by converting the training image.

[0119] See Figure 2 In one specific implementation, the loss function is the mean squared error; the loss function for establishing the aforementioned counting network model includes:

[0120] Based on the density contribution of each pixel in the training density map corresponding to the training image and the second density map of the training image, the loss between the training density map and the second density map is quantified by mean square error, so as to minimize this loss through the backpropagation algorithm to train the counting network model.

[0121] The formula for calculating the above loss function (mean squared error) is as follows:

[0122]

[0123] Where N is the number of pixels in the training density map, y i This contributes to the density of the i-th pixel in the training density map.

[0124] The above are some specific implementations of the methods provided in the embodiments of this application. Based on this, this application also provides corresponding apparatus. The apparatus provided in the embodiments of this application will be described below from the perspective of functional modularity.

[0125] See Figure 3 The diagram shows a target object counting and positioning device. The target object counting and positioning device includes:

[0126] The first processing unit 301 is used to input the target image of the target scene into the backbone network of the counting network model, and obtain the output results of each of the M stages included in the backbone network. The counting network model is trained based on the training density map constructed by the kernel function.

[0127] Optionally, a target image of the target scene can be acquired by capturing it using a high-point camera.

[0128] The above method trains the counting network model using a density map constructed based on a kernel function, learning the distribution of target object density to indirectly calculate the number of target objects.

[0129] The second processing unit 302 is used to sample the output results of the M stages to obtain output results with uniform dimensions.

[0130] The fusion unit 303 is used to input the dimension-unified output result into the fusion module of the counting network model to obtain the first density map;

[0131] The calculation unit 304 is used to sum the density contributions of the first density map to obtain the number of target objects in the target scene.

[0132] Based on the above-described apparatus, the first processing unit 301 acquires feature maps of multiple stages from the target image through a backbone network. The second processing unit 302 and the fusion unit 303 then fuse these feature maps through sampling and fusion modules to obtain a first density map reflecting the density of the target objects. Finally, the calculation unit 304 calculates the number of target objects based on the first density map. Thus, even in situations where the density of target objects is high and they overlap, the model fully utilizes feature information from multiple different stages, fusing features from multiple stages to enhance its adaptability to complex scenes and changes, improves prediction accuracy, and achieves accurate counting of the number of target objects captured within the target image.

[0133] Optionally, the device further includes a positioning unit, which is used to cluster the first density map using a clustering algorithm to determine the location of target objects in the target scene. The contour coefficient of the clustering algorithm is the number of target objects.

[0134] Optionally, the second processing unit 302 is specifically configured to use the dimension of the output result of the target stage as the target dimension, wherein the target stage is one of the M stages; for the first stage in the backbone network that obtains the output result before the target stage, downsample the output result of the first stage to obtain the output result of the first stage with the target dimension, wherein the first stage is one of the M stages; for the second stage in the backbone network that obtains the output result after the target stage, upsample the output result of the second stage to obtain the output result of the second stage with the target dimension, wherein the second stage is one of the M stages.

[0135] Optionally, the device further includes a network construction unit for constructing a dataset, the dataset including training images and training density maps corresponding to each training image, the training images containing target objects; establishing the network architecture of the counting network model; establishing the loss function of the counting network model; and training the counting network model based on the dataset to obtain a trained counting network model.

[0136] Optionally, the network construction unit is specifically configured to set up a backbone network including N stages; determine M stages from the N stages, connect the output of the target stage in the M stages to the fusion module, wherein the target stage is one of the M stages; for a first stage that obtains an output before the target stage in the backbone network, connect the output of the first stage to the fusion module after passing through a downsampling module, wherein the first stage is one of the M stages, and the downsampling module unifies the dimension of the output of the first stage with the dimension of the output of the target stage; for a second stage that obtains an output after the target stage in the backbone network, connect the output of the second stage to the fusion module after passing through an upsampling module, wherein the second stage is one of the M stages, and the upsampling module unifies the dimension of the output of the second stage with the dimension of the output of the target stage; and the fusion module fuses the input feature maps and outputs a second density map with 1 channel.

[0137] Optionally, the network construction unit is specifically used to acquire multiple training images for training; label the center point of each target object in the training images to obtain a labeled image; based on the label data of the center points of the target objects in the labeled image, convert the labeled image into a training density map using a Gaussian kernel function; divide the multiple training images into a training set, a validation set, and a test set in a 7:2:1 ratio, wherein each training image in the training set, validation set, and test set corresponds to a training density map obtained by converting the training image.

[0138] Optionally, the network construction unit is specifically used to establish a two-dimensional matrix with the same size as the labeled image to store the training density map. All elements of the two-dimensional matrix are initialized to zero. For each center point of the labeled object in the labeled image, the pixels on the training density map are traversed. The density contribution of each center point of the object to each traversed pixel is calculated using a Gaussian kernel function. The density contributions of all center points in the labeled image to each pixel in the training density map are accumulated as the value of the corresponding position in the training density map to form the training density map.

[0139] Optionally, the loss function is mean squared error; the network construction unit is specifically used to quantify the loss between the training density map and the second density map based on the density contribution of each pixel in the training density map corresponding to the training image and the second density map of the training image, so as to train the counting network model by minimizing this loss through the backpropagation algorithm.

[0140] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0141] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to enable the device to perform a target counting and positioning method according to any embodiment of this application.

[0142] The computer storage medium stores code, and when the code is executed, the device running the code implements a target object counting and positioning method according to any embodiment of this application.

[0143] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0144] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0145] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0146] The above description is merely an exemplary implementation of this application and is not intended to limit the scope of protection of this application.

Claims

1. A method of target counting and positioning, characterized by, The method comprises the following steps: inputting a target image of a target scene into a backbone network of a counting network model, obtaining an output result of each stage in M stages included in the backbone network, the counting network model being trained based on a training density map constructed by a kernel function, the training density map being formed by traversing pixel points on the training density map, calculating a density contribution of each target center point to each pixel point traversed by a Gaussian kernel function, and accumulating density contributions of all center points in the labeled image to each pixel point in the training density map as a value of a corresponding position of the training density map; sampling the output results of the M stages to obtain output results with unified dimensions; inputting the output results with unified dimensions into a fusion module of the counting network model to obtain a first density map; summing density contributions of the first density map to obtain a number of target objects in the target image; determining positions of the target objects in the target scene by clustering the first density map by a clustering algorithm, a contour coefficient of the clustering algorithm being the number of target objects.

2. The method of claim 1, wherein, The sampling of the output results of the M stages to obtain output results with unified dimensions comprises the following steps: taking a dimension of an output result of a target stage as a target dimension, the target stage being one of the M stages; for a first stage in the backbone network that obtains an output result before the target stage, performing down-sampling on the output result of the first stage to obtain an output result of the first stage with the target dimension, the first stage being one of the M stages; for a second stage in the backbone network that obtains an output result after the target stage, performing up-sampling on the output result of the second stage to obtain an output result of the second stage with the target dimension, the second stage being one of the M stages.

3. The method of claim 1, wherein, The counting network model comprises the following steps: constructing a data set, the data set comprising training images and training density maps corresponding to the training images, the training images containing target objects; establishing a network architecture of the counting network model; establishing a loss function of the counting network model; training the counting network model based on the data set to obtain a trained counting network model.

4. The method of claim 3, wherein, The establishment of the network architecture of the counting network model comprises the following steps: setting a backbone network comprising N stages; determining M stages from the N stages, connecting an output result of a target stage in the M stages to a fusion module, the target stage being one of the M stages; for a first stage in the backbone network that obtains an output result before the target stage, connecting an output result of the first stage to the fusion module after passing through a down-sampling module, the first stage being one of the M stages, the down-sampling module being used to unify a dimension of the output result of the first stage and a dimension of the output result of the target stage. For a second stage obtaining an output result after the target stage of the backbone network, the output result of the second stage is connected with the fusion module after passing through an upsampling module, the second stage is one of the M stages, and the upsampling module is to unify the dimension of the output result of the second stage and the dimension of the output result of the target stage; the fusion module is to fuse the input feature maps and output a second density map with one channel.

5. The method of claim 3, wherein, the constructing the dataset comprises: obtaining a plurality of training images for training; annotating a label for a center point of each target object in the training image to obtain an annotated image; based on the label data of the center point of the annotated target object in the annotated image, converting the annotated image into a training density map through a Gaussian kernel function; dividing the plurality of training images into a training set, a validation set and a test set according to a ratio of 7:2:1, each training image included in the training set, the validation set and the test set corresponding to a training density map obtained by converting the training image.

6. The method of claim 5, wherein, the converting the annotated image into a training density map based on the label data of the center point of the annotated target object in the annotated image through a Gaussian kernel function comprises: establishing a two-dimensional matrix with the same size as the annotated image, for storing the training density map, all elements of the two-dimensional matrix being initialized to zero; for each center point of the annotated target object in the annotated image, traversing the pixel points on the training density map, calculating the density contribution of each center point of the target object to each traversed pixel point through a Gaussian kernel function, and accumulating the density contribution of all center points in the annotated image to each pixel point in the training density map as the value of the corresponding position of the training density map to form the training density map.

7. The method of claim 4, wherein, the loss function is a mean square error; the establishing the loss function of the counting network model comprises: quantifying the loss between the training density map and the second density map through a mean square error according to the density contribution of each pixel point of the training density map corresponding to the training image and the second density map of the training image, so as to train the counting network model by minimizing this loss through a back propagation algorithm.

8. The method of claim 1, wherein, the method further comprises: obtaining a target image of the target scene by shooting through a high point camera.

9. A target counting and positioning device, characterized by the device comprises: a first processing unit configured to input a target image of a target scene into a backbone network of a counting network model, obtain an output result of each stage of M stages included in the backbone network, and train the counting network model based on a training density map constructed by a kernel function, wherein the training density map is a training density map formed by, for each center point of an annotated target object in an annotated image, traversing pixel points on the training density map, calculating the density contribution of each center point of the target object to each traversed pixel point through a Gaussian kernel function, and accumulating the density contribution of all center points in the annotated image to each pixel point in the training density map as the value of the corresponding position of the training density map. A second processing unit is configured to sample the output results of the M stages to obtain output results with unified dimensions; A fusion unit is configured to input the output results with unified dimensions into a fusion module of the counting network model to obtain a first density map; A calculation unit is configured to sum density contributions of the first density map to obtain a target object quantity of the target scene; A positioning unit is configured to cluster the first density map by using a clustering algorithm to determine a position of a target object in the target scene, and a contour coefficient of the clustering algorithm is the target object quantity.

Images