A large-scale scene target perception method based on position distribution

Abstract

A large-scale scene target perception method based on position distribution, comprising the steps of: inputting a given large-scale scene into a backbone network for feature extraction; inputting the scene data after feature extraction in the backbone network into a multiple convolution layer for processing; based on the data after processing by the multiple convolution layer, calculating the output predicted positioning map, target offset and width and height of the target bounding box; based on the positioning map, setting a local scaling module LSM to adaptively search the concentrated area in the positioning map image and appropriately scaling it to fit the capacity of the detector; modifying part of the loss function of the original data so that it can be easier to train, which can realize the detection of dense small targets and alleviate the problem of uneven target distribution.

Description

Technical Field

[0001] This invention relates to the field of image processing, specifically to a method for large-scale scene target perception based on location distribution. Background Technology

[0002] Object perception, a fundamental task in computer vision, has a wide range of applications, such as disaster forecasting and traffic flow control. Object perception is a multi-task learning framework designed to simultaneously estimate the category of a target and accurately locate it.

[0003] With the rapid development of deep learning, especially the development of convolutional neural networks (CNNs), the performance of object perception has significantly improved. Several representative paradigms of existing high-performing object perceptrons, such as the two-stage Faster-RCNN, the single-stage YOLO and SSD, and the anchor-free FCOS and CenterNet, have demonstrated excellent performance on datasets depicting natural scenes. Images in natural scenes typically have low resolution, for example, 600×400 in MS COCO. However, when these methods are applied to aerial photography, their performance is unsatisfactory.

[0004] These methods demonstrate good performance in natural images; however, their performance is not optimal in large-scale scenes, typically captured by drones, airplanes, satellites, and high-definition smart cameras. Target perception within these images is of practical significance, for applications such as autonomous driving, target perception in remote sensing images, smart urban transportation, and smart city construction.

[0005] Target perception in large-scale scenes is more challenging, and its poor performance is mainly due to the following two aspects: 1) Large-scale scene datasets contain a large number of small targets. These small targets constitute a significant portion of the dataset, and their size is much smaller than the overall image resolution. This makes it difficult for detectors to identify small targets within the limited resolution; for example, Visdrone's resolution is 2000×1500. Due to the limited resolution and lack of coarse detail information, detectors struggle to identify small targets. 2) Targets in large-scale scenes exhibit a sparse and uneven distribution. For example, vehicles tend to cluster together at intersections or parking lots, while they are sparse in other areas. This severely imbalanced data distribution also leads to poor detector performance.

[0006] To address these issues, current popular solutions primarily rely on image cropping strategies. These methods typically divide a large-scale image into smaller, lower-resolution patches, perform target detection on each patch, and then merge the detection results into a larger image. The simplest and most direct approach is to uniformly divide the image or crop it randomly. This simple strategy can improve the detection accuracy of small targets to some extent because cropping and then enlarging the image can increase the scale of small targets. However, this method results in image patches containing a large amount of background, causing a severe imbalance between positive and negative samples, leading to extremely low efficiency. Furthermore, for larger targets, the image may be split in half. To improve efficiency and maximize the utilization of target information, ClusDet designed a cluster proposal sub-network (CPNet) and a scale estimation sub-network (ScaleNet), then fused the detection of clustered regions with the global detection. DMNet utilizes the idea that density maps can intuitively reflect target distribution, and then crops the image's clustered regions based on a set density threshold. UFPMP-Det uses a simple clustering method (K-means clustering) to obtain clustered regions of targets, and then assembles these regions into an image using a mosaic effect, thus greatly improving the utilization of target information. This type of method exhibits good detection performance because it maximizes the preservation of regions containing dense small targets and suppresses useless background information, thereby accelerating inference. However, designing a dedicated network to find clustered regions of targets, then cropping and storing them offline before finally performing target perception, means their detection performance heavily depends on the network design for generating clustered regions. Furthermore, the offline cropping and storage method, followed by separate feeding into the detector, significantly slows down detection efficiency. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a large-scale scene target perception method based on location distribution, which can realize the detection of dense small targets and alleviate the problem of uneven target distribution.

[0008] This invention provides a method for large-scale scene target perception based on location distribution, comprising the following steps performed sequentially:

[0009] (1) Input the given large-scale scene into the backbone network for feature extraction, wherein the backbone network is the high-resolution feature pyramid network HRFPN;

[0010] (2) The scene data after feature extraction in the backbone network is input into multiple convolutional layers for processing, where each convolutional layer is followed by a ReLU activation layer;

[0011] (3) Based on the data processed by multiple convolutional layers, calculate the predicted localization map, target offset, and the width and height of the target bounding box; wherein, the position coordinates of the target center point are obtained by calculating the local maximum value in the localization map, and the width and height of the target bounding box are obtained by the position coordinates of the target center point;

[0012] (4) Based on the localization map, the local scaling module LSM adaptively searches for clustered regions in the localization map image and scales them appropriately to suit the capacity of the detector.

[0013] (5) Modify part of the loss function of the original data to make training easier.

[0014] In a preferred embodiment, the multiple convolutional layers in step (2) include convolutional layers and deconvolutional layers, wherein the convolutional layer is represented as “Conv-convolution kernel-channel-stride”, and the deconvolutional layer is represented as “TransConv-convolution kernel-channel-stride”.

[0015] In a preferred embodiment, the multiple convolutional layers in step (2) are configured sequentially as Conv-1-384-1, TransConv-4-64-1, TransConv-4-64-1 and Conv-1-1.

[0016] In a preferred embodiment, the predicted location map calculated in step (3) is generated by using a distance transformation map as a location regression method.

[0017] In the preferred embodiment, the calculation of the predicted localization map in step (3) specifically involves:

[0018] Assume S represents the set of all target center points, (x,y) represents the x and y coordinates on the Euclidean distance transformation graph, and (x',y') represents the coordinates of the target center points in set S. Then, the Euclidean distance transformation graph is represented as:

[0019]

[0020] Two distance transformation maps, the inverse distance transformation map FIDTM and the Gaussian weighted distance transformation map GDTM, are used for processing, as shown below:

[0021]

[0022] GD = e -γD(x,y)

[0023] α, β, and γ are hyperparameters, which are empirically set to 0.02, 0.75, and 0.02, respectively; C is a constant to avoid a denominator of 0, and is set to C = 1.

[0024] Both distance transformation map methods use the High Resolution Feature Pyramid Network (HRFPN) as the backbone network, and then add one convolutional layer and two deconvolutional layers for localization regression.

[0025] In the preferred embodiment, step (4) specifically includes:

[0026] (4.1) Given the predicted distance transformation map, the predicted localization map is obtained by the local maximum detection method;

[0027] (4.2) Divide the predicted location map into 16×10 grids, count the number of targets in each grid, and then sort them from largest to smallest according to the number;

[0028] (4.3) Select the 15 grids with the largest number and obtain larger candidate regions through the octet region algorithm, that is, select the clustered regions in the image;

[0029] In a preferred embodiment, step (4.3) further includes: first enlarging the connected region to an appropriate scale so that the clipping region can cover the entire target and filtering out clipping blocks smaller than a certain threshold; then enlarging the clipping region to a size suitable for the network and filling the remaining part with zeros.

[0030] In the preferred embodiment, step (5) specifically comprises:

[0031] The original loss function consists of three parts: the prediction loss L of the center point. k The offset loss of the center point L off and size regression loss L size That is, the final loss is a combination of three losses:

[0032] L det =L k +λ size L size +λ off L off

[0033] Where L k Using Gaussian-weighted focal loss, L off Using L1 loss, the target size regression loss L size Using L1 loss, λ size and λ off As control parameters, they are set to 0.1 and 1 respectively;

[0034] The size regression loss L is adjusted using the modified GIoU loss M-IoU. size Make the necessary modifications;

[0035] The original loss function is modified based on the regression loss using Gaussian Wasserstein distance.

[0036] In the preferred embodiment, the regression loss based on Gaussian Wasserstein distance modifies the original loss function, specifically as follows:

[0037] Boundary box of the target Convert to a two-dimensional Gaussian distribution Where x, y, h, w represent the horizontal and vertical coordinates of the bounding box, as well as the height and width of the bounding box, respectively; m represents the mean; and Σ represents the covariance matrix.

[0038]

[0039] m = (x, y)

[0040] The Wasserstein distance between two distinct probability distributions can be defined as:

[0041]

[0042] Where μ and v represent two different probability distributions, and the variables include all random variables (X, Y) in Rn×Rn, X~μ, Y~v, we can obtain:

[0043]

[0044] Where m1 and m2 represent the means of two different two-dimensional Gaussian distributions, and Σ1 and Σ2 represent the covariance matrices of these two distributions.

[0045] It can be written as:

[0046] in,

[0047] Since this is a horizontal bounding box detection process, according to the commutative law, Σ1Σ2=Σ2Σ1, therefore:

[0048]

[0049] Among them, || || F Denotes the Frobenius norm;

[0050] Using a nonlinear transformation, d 2 Convert to an affine function Then, following the form of GIoU, this affine function can be written as:

[0051]

[0052] Where f(·) represents a nonlinear function that represents the Wasserstein distance d 2 It becomes smoother and easier to train; the choice is f(·) = ln(D+1), where τ controls the entire loss, and τ = 1;

[0053] Using L GWD When using it as regression loss, since the center point offset has already been considered, the center point offset loss L is ignored. off This results in the final loss being:

[0054] L det =L k +λ GWD L GWD ;

[0055] Where, λ GWD The control parameter is set to 2.

[0056] The large-scale scene target perception method based on location distribution of the present invention can achieve:

[0057] 1) An effective and efficient target perception algorithm without anchor boxes, with a simple and elegant overall network framework.

[0058] 2) The unsupervised local scaling module (LSM) takes the adaptive search cluster region and scales it. This module is lightweight and resource-efficient.

[0059] 3) Extensive experiments were conducted on two large-scale sets of aerial target perceptrons, demonstrating the effectiveness of the proposed algorithm and proving its superiority by comparing it with state-of-the-art methods. Attached Figure Description

[0060] Figure 1 A comparative diagram of the YOLC network framework and CenterNet;

[0061] Figure 2 A schematic diagram of the YOLC framework that ignores offset and size regression branches and includes a local scaling module;

[0062] Figure 3 This is a schematic diagram illustrating a visual detection result of the YOLC network. Detailed Implementation

[0063] The specific implementation of the present invention will be described in detail below. It should be noted that the following implementation is only for further illustration of the present invention and should not be construed as a limitation on the scope of protection of the present invention. Some non-essential improvements and adjustments made to the present invention by those skilled in the art based on the above description of the present invention still fall within the scope of protection of the present invention.

[0064] This invention provides a large-scale scene target perception method based on location distribution, which can detect dense small targets and alleviate the problem of uneven target distribution.

[0065] Based on observation and analysis, this invention proposes a simple and effective target perception framework: a You Only Look Clusters (YOLC) aerial target perception network that requires no anchor boxes and can adaptively search dense regions online. The proposed YOLC network is somewhat similar to CenterNet, except that the predicted heatmap is replaced with a localization map. CenterNet was chosen as the baseline algorithm because of its simplicity, scalability, and high inference speed. Most importantly, CenterNet directly predicts the center point of the target, rather than designing complex anchor boxes, such as Faster R-CNN. In the original CenterNet framework, it infers the target's localization by predicting a heatmap (also called a density map). However, density maps typically cannot provide accurate target localization because they contain a series of fuzzy Gaussian clusters that severely overlap in dense scenes, causing the target's position to shift. Therefore, directly applying CenterNet to dense, small target perception tasks in large-scale scenes is not very suitable. By modifying the density map to a localization map, the rest remains largely consistent with CenterNet. Furthermore, to mitigate uneven distribution and improve the detection performance of small targets, a Local Scale Module (LSM) is designed. This module adaptively selects target clustering regions in the image, performs online cropping, and adaptively scales the images to a size suitable for detection. Compared to previous image cropping strategies, the proposed LSM is an unsupervised module that eliminates the need for a dedicated clustering region generation network, making the entire network framework easier to implement, more memory-efficient, and maintaining high inference efficiency. A detailed description follows.

[0066] CenterNet is an anchor-free framework for general object perception proposed in recent years. It represents the object by predicting the center coordinates, and other features such as the object's size, orientation, or pose can be obtained by regressing directly from the center position.

[0067] Therefore, object perception can be described as keypoint detection. Specifically, the model predicts a heatmap (or density map) using a fully convolutional network, such as an hourglass network. The coordinates of the center point can then be obtained by calculating local maxima in the heatmap. The width and height of the object's bounding box can then be obtained from the coordinates of the center point. The network framework is as follows: Figure 1 As shown in (a) of the diagram, the CenterNet model is simple, requires no NMS post-processing, boasts high-speed detection, and can be easily extended to other tasks, such as 3D object perception and human pose estimation. Its simplicity, high-speed detection efficiency, and strong scalability have resulted in excellent detection performance. This is why this network was chosen as the benchmark algorithm.

[0068] Density maps can intuitively reflect the distribution of targets, and changes in density intensity can naturally infer the presence or absence of targets. Inspired by this, this is very helpful for image cropping or extracting regions of interest in large-scale scenes. CenterNet can directly predict the density map of target distribution, which is consistent with the above analysis. However, if CenterNet is directly applied to large scenes, it may lead to less than ideal results. This is because there are a large number of densely packed small targets, and in dense scenes, the density maps are likely to have severe overlap, causing multiple targets to collapse to a single point.

[0069] To address this issue, a localization map is used instead of a density map to prevent overlapping of predicted target locations. The YOLC framework is proposed, allowing the model to focus only on regions of clustered targets within the image. Furthermore, to further improve the detection performance of clustered small targets and alleviate the problem of uneven target distribution in large-scale scenes, a Local Scale Module (LSM) is proposed. This module is derived from clustered regions in the adaptive search image and appropriately scaled to suit the detector's capacity. The entire framework is simple and efficient.

[0070] In the YOLC method, the proposed YOLC framework is as follows: Figure 1 As shown in (b) of the figure (the convolutional layers in the figure are represented as "Conv-convolutional kernel-channel-stride", TransConv represents a deconvolutional layer, and each convolutional layer is followed by a linear rectified unit (ReLU) activation layer), Additionally, Figure 1(The local scaling module is omitted). It adopts a form similar to CenterNet, but with some adjustments to the backbone network design, regression method, and loss function. Specifically, a given large-scale scene is input into a fully convolutional network, which outputs a localization map to estimate the precise location of the target center point. Then, like CenterNet, other features can be inferred from the location of the center point. Because there are many dense small targets in large-scale scene images, a High Resolution Feature Pyramid Network (HRFPN) is used as the feature extraction backbone to avoid information loss caused by multiple convolution and pooling operations. For the generation of the localization map, a distance transformation map is used for localization regression. In addition, considering the uneven distribution and local clustering of small targets in large-scale scenes, a Local Scaling (LSM) module is added. Finally, for the designed network, some of the loss functions of the original CenterNet are modified to make the network easier to train. Next, we will mainly introduce the generation of the ground truth localization map, LSM, and the modified loss function.

[0071] • Generation of truth location maps

[0072] First, assume that S represents the set of all target center points, (x,y) represents the horizontal and vertical coordinates on the Euclidean distance transformation graph, and (x',y') represents the coordinates of the target center points in set S. Then, the Euclidean distance transformation graph can be represented as follows:

[0073]

[0074] Because the Euclidean distance can range from [0, +∞), it is difficult to regress. To address this issue, two distance transformation maps are used: the Focal Inverse Distance Transform Map (FIDTM) and the Gaussian-weighted Distance Transform Map (GDTM), as shown below:

[0075]

[0076] GD = e -γD(x,y) (3)

[0077] α, β, and γ are hyperparameters, empirically set to 0.02, 0.75, and 0.02, respectively. C is a constant to avoid a denominator of 0, set to C = 1.

[0078] To accurately regress target localization in large-scale scenes, especially for small targets, both distance transformation schemes use HRFPN as the backbone network, then add one convolutional layer and two deconvolutional layers for localization regression. It is worth noting that any localization method can be used in the network.

[0079] • Local scaling module LSM

[0080] In large-scale scenarios, besides numerous dense small targets, there is also severe uneven data distribution, with some target instances clustered together while others are more dispersed. To alleviate this problem, a Local Scaling Module (LSM) is proposed. It adaptively searches for clustered regions, then crops and scales them to fit the detector's size. LSM is somewhat similar to AutoScale, but the difference is that AutoScale was originally used for crowd counting and localization tasks, and therefore is only suitable for single-class targets, while large-scale scenarios involve multiple target classes. Furthermore, AutoScale only searches for the largest clustered region based on a set distance threshold. This may lead to suboptimal results in large-scale scenarios because multiple clustered regions exist in large-scale scenarios, and their density is not as high as that of a crowd. The proposed LSM will be described in detail below, with its specific structure as follows: Figure 3 As shown.

[0081] Specifically, given the distance transformation map predicted earlier, a localization map is first obtained using the local maximum detection method. Next, it is divided into a 16×10 grid. Then, the number of targets in each grid is counted, and the grids are sorted from largest to smallest. The top 15 grids with the highest number of targets are selected, and the largest candidate regions are obtained using the octal connected region algorithm, i.e., the clustered regions in the image are selected. Before this, to prevent target truncation, the connected regions are enlarged by an appropriate scale (e.g., 1.2 times) so that the cropped region can cover the complete target, filtering out cropped blocks smaller than a certain threshold (e.g., 96×96). Finally, the cropped region is enlarged to fit the network size, and the remaining parts are padded with zeros.

[0082] Loss function

[0083] The original CenterNet loss function consists of three parts: the prediction loss of the center point (L... k ), center point offset loss (L off ) and size regression loss (L size That is, the final loss is a combination of three losses:

[0084] L det =L k +λ size L size +λoff L off (4)

[0085] Where L k Using Gaussian-weighted Focal loss, L off The L1 loss is used for the target size regression loss, λ. size and λ off To control parameters, they were set to 0.1 and 1 respectively. The size regression loss was modified because it is sensitive to changes in target scale, regressing only the width and height of the bounding box and thus ignoring the correlation between attributes within the same target. Furthermore, this loss leads to inconsistencies between measurement and evaluation: training measures differences in target size, while testing evaluates the IoU between the predicted and ground truth boxes. Therefore, a high training loss does not guarantee high detection accuracy. In contrast, GIoU considers the optimization process holistically, independent of target size. Therefore, GIoU is used instead of L1 loss. However, this loss was originally used in anchor-based detectors and is not suitable for direct application in the proposed anchor-free detector. Therefore, following the approach in AFSM, a modified GIoU loss (M-IoU) is used. Thus, the final loss simply replaces the size regression loss; other losses and parameter settings remain consistent with CenterNet.

[0086] Furthermore, considering the presence of numerous small targets in large-scale scenes, where even slight shifts in these targets can lead to significant losses, a regression loss based on Gaussian Wasserstein Distance (GWD) is introduced. Specifically, the bounding boxes of the targets are first... Convert to a two-dimensional Gaussian distribution m represents the mean, and Σ represents the covariance matrix.

[0087]

[0088] m = (x, y)

[0089] The Wasserstein distance between two distinct probability distributions can be defined as:

[0090]

[0091] The variables include all random variables (X,Y) in Rn×Rn, where X~μ, Y~v, from which we can obtain:

[0092]

[0093] It can be written as:

[0094] It is important to note that:

[0095]

[0096] Since this is a horizontal bounding box detection process, according to the commutative law, Σ1Σ2=Σ2Σ1, and Equation 1.7 can be rewritten as:

[0097]

[0098] Among them, || || F This represents the Frobenius norm.

[0099] It should be noted that GWD alone may be sensitive to large errors, therefore a nonlinear transformation is used to transform d 2 Convert to an affine function Then, following the form of GIoU, this affine function can be written as:

[0100]

[0101] Where f(·) represents a nonlinear function that represents the Wasserstein distance d 2 It becomes smoother and easier to train; it is chosen as f(·) = ln(D+1), and τ controls the entire loss, which is set to τ = 1 in this method.

[0102] It is important to note that when using L GWD When using it as regression loss, since the center point offset has already been considered, the center point offset loss L is ignored. off This results in the final loss being:

[0103] L det =L k +λ GWD L GWD (0.1)

[0104] Where, λ GWD Set to 2.

[0105] For the experimental results and analysis, we first introduce the dataset, evaluation criteria, and implementation details. Then, through experimental comparisons with state-of-the-art methods, we further demonstrate the superiority of the proposed algorithm. Finally, we prove the effectiveness of the proposed method through extensive ablation experiments.

[0106] • Dataset

[0107] Experiments were conducted on two publicly available aerial photography datasets, Visdrone and UAVDT. Details of the datasets are as follows:

[0108] Visdrone contains 10,209 high-resolution images with an average resolution of approximately 2,000 × 1,500. The dataset includes ten object classes: pedestrians, people, bicycles, cars, vans, trucks, tricycles, awing-tricycles, buses, and motorcycles. 6,471 images are used as the training set, 548 as the validation set, and the remaining 3,190 as the test set. Because the test set does not provide ground truth labels and the evaluation platform server is offline, evaluation is performed on the validation set, following the approach used by ClusDet and DMNet.

[0109] UAVDT contains 38,327 images with an average resolution of approximately 1080×540. 23,258 images are used as the training set, and the remaining 15,069 are used as the test set. This dataset includes three object classes: cars, buses, and trucks.

[0110] Evaluation indicators

[0111] Following the evaluation protocol of the MSCOCO

[35] dataset, average precision (AP), precision at an intersection-over-union ratio of 0.5 (AP_50), and precision at an intersection-over-union ratio of 0.75 (AP_75) are used. Specifically, AP is obtained by averaging the precision across all classes, and AP_50 and AP_75 are calculated by setting a single IoU threshold of 0.5 and 0.75 for all classes, respectively. In addition to the above three metrics, single-class precision is also provided to evaluate the detection performance of a single class, and there are also metrics for different scales (the detection precision of small, medium, and large targets are represented as AP_small, AP_medium, and AP_large, respectively). Detection efficiency is measured by calculating the number of images that the detector needs to process, and the average time is the time it takes for the detector to process an image and a locally scaled image patch. In addition, the number of images processed refers to the sum of the global image and the local image, which is represented as #img in subsequent experiments.

[0112] Implementation details

[0113] The proposed method is based on the open-source MMDetection toolkit. CenterNet, with Hourglass as the backbone network, is used as the baseline algorithm. Unless otherwise specified, this setup is maintained in all experiments. The proposed model is trained for 240 epochs using stochastic gradient descent (SGD). Momentum and weight decay are set to 0.9 and 0.001, respectively. The initial learning rate is set to 0.01, and linear warm-up is performed. For both datasets, the detector input resolution is set to 1000×600.

[0114] Quantitative and qualitative experimental results

[0115] The performance of the proposed YOLC method on the VisDrone and UAVDT datasets is evaluated. Table 1 shows that YOLC consistently improves performance on the VisDrone dataset. Specifically, general object detection methods perform poorly on this dataset, mainly due to the presence of numerous small objects and uneven distribution. However, comparing the single-stage CenterNet with the two-stage Faster R-CNN, the single-stage method clearly performs better, validating the previous analysis that two-stage detection methods are inconvenient for setting anchor boxes for a large number of densely packed small objects. Anchor-free methods avoid this operation and are more suitable for small object detection, which is one of the reasons for choosing CenterNet. Furthermore, due to the severe uneven distribution of objects in the images, methods that crop the image based on density to speed up inference have become mainstream. Compared to these methods, such as ClusDet and DMNet, the proposed method surpasses their performance without density cropping. The performance is further improved by adding a local magnification module for clustered region cropping. Finally, by modifying the optimization scheme, the best AP value of 33.8% was achieved. Furthermore, under multi-scale testing conditions, performance was improved by nearly 3%. On another note, the proposed method significantly improves the detection performance of small targets, while the performance improvement for medium and large targets is not very significant; this is also to improve the detection of small targets. In addition, considering the number of images processed (#img) and the detection rate (s / img), the method proposed in this invention achieves higher point detection performance with a relatively smaller number of images and a faster inference speed.

[0116] Table 2 shows the performance comparison with other state-of-the-art algorithms on the UAVDT dataset. Consistent with the results on the Visdrone dataset, it still significantly improves detection performance. In particular, consistent performance improvements were achieved for small-scale, medium-scale, and large-scale targets, thanks to the choice of framework, the construction of the local scaling module, and the design of the optimization scheme. However, from an overall performance perspective, this metric still has considerable room for improvement. Besides the challenges of the high-density target distribution and extremely small target size in this dataset, it also labels an invalid region: the "ignore." These regions contain a large number of instances of valid classes, but these classes are either not labeled or are incorrectly labeled. Therefore, this significantly reduces detection performance, which is a direction for future improvement.

[0117] Furthermore, to further verify the superiority of the method, Table 3 shows the detection results for each target class. The table reveals that, longitudinally, compared to the benchmark algorithm, the accuracy is generally improved by 6%-14% for each class. However, looking at individual metrics, the detection method performs well in some common classes, such as cars, vans, and buses, but its performance needs improvement in some rare classes, such as bicycles, tricycles, and awning tricycles. This is due to a severe class imbalance in the dataset, which is an area planned for future improvement.

[0118] In addition to the quantitative analysis above demonstrating the effectiveness of the algorithm, it also... Figure 3 The visualization of the detection results shows that the proposed method can achieve good detection results in both sparse and dense scenes, especially for small targets.

[0119] Ablation experiment

[0120] The effectiveness of the designed localization map scheme, local scaling module, and loss function was verified through ablation experiments.

[0121] Impact of different backbone networks

[0122] To verify the performance impact of different backbone networks, the backbone network in the original CenterNet was replaced, aiming to address the information loss issues caused by multiple convolutions and pooling processes for densely packed small targets in large-scale scenes. Table 1 shows that simply replacing the backbone network improved accuracy by nearly 3%. Furthermore, this invention focuses on improving the detection performance of small targets; the table shows that replacing the backbone network improved accuracy by approximately 5% for small targets. Surprisingly, it also improved accuracy by approximately 8% for medium and large targets.

[0123] Different center point positioning methods have an impact

[0124] One of the motivations was to replace the original CenterNet heatmap (density map) with a localization map, so two localization methods were used: FIDTM and GDTM. As shown in Table 1, these two localization methods offer nearly 2% performance improvement over the previous methods. However, the GDTM-based approach is slightly better than the FIDTM approach. Therefore, the GDTM-based approach was used in subsequent experiments.

[0125] • The function of the local scaling module

[0126] Uneven target distribution in large-scale scenes results in targets clustering in some areas but sparse distribution in others. To address this issue, an adaptive local scaling module is proposed, which sets different thresholds for image cropping, i.e., different k values. Table 1 shows that when k=1, the accuracy is improved by 3.2% compared to the previous results, but when k=2, the performance improvement is only 0.2% compared to k=1. When k=3, there is almost no performance improvement. To prevent some targets from being excluded if the cropping is too small, the maximum cropping area is set to k=3.

[0127] The impact of different loss functions

[0128] The original CenterNet designed three loss functions. It was observed that the original L1 loss was sensitive to scale changes and exhibited inconsistencies between measurement and evaluation. Two losses were modified to address this: a modified GIoU (M-GIoU) ​​and GWD. As shown in Table 1, replacing the original L1 loss with the M-GIoU loss improves accuracy by 0.4%. Replacing it with GWD further improves performance by 0.4%. This demonstrates the importance of the designed loss functions.

[0129] Table 1 shows the performance comparison on the VisionDrone2019 dataset.

[0130]

[0131]

[0132]

[0133] Note: #img indicates the number of images fed into the detector. “” indicates multi-scale testing, while ★ indicates data augmentation using mosaic. “o” indicates the original validation set images, “c” indicates images with evenly partitioned images (EIP), “ca” indicates images cropped according to clustering, “aug” indicates data-augmented images, and “FRCNN” stands for Faster R...

[0134] Table 2 Performance Comparison on UAVDT

[0135]

[0136] Table 3 shows the single-class detection results on the VisDrone 2019 validation set.

[0137]

[0138]

[0139] Note: RS stands for random sampling.

[0140] Although exemplary embodiments of the invention have been described for illustrative purposes, those skilled in the art will understand that various modifications, additions, and substitutions in form and detail may be made without departing from the scope and spirit of the invention disclosed in the appended claims, and all such modifications and substitutions should fall within the scope of protection of the appended claims. Furthermore, the various parts of the product and the various steps of the method claimed in this invention can be combined in any combination. Therefore, the description of the embodiments disclosed in this invention is not intended to limit the scope of the invention, but rather to describe the invention. Accordingly, the scope of the invention is not limited by the above embodiments, but is defined by the claims or their equivalents.

Claims

1. A large-scale scene target perception method based on position distribution, characterized in that, The method comprises the following steps in sequence: (1) inputting a given large-scale scene into a backbone network for feature extraction, wherein the backbone network is a high-resolution feature pyramid network (HRFPN); (2) inputting the scene data after feature extraction in the backbone network into multiple convolution layers for processing, wherein each convolution layer is followed by a linear rectification unit (ReLu) activation layer; (3) calculating an output predicted positioning map, target offset, and width and height of the target bounding box based on the data processed by the multiple convolution layers, wherein the position coordinates of the target center point are obtained by calculating the local maximum value in the positioning map, and the width and height of the target bounding box are obtained by the position coordinates of the target center point; (4) setting a local scaling module (LSM) to adaptively search the aggregation area in the positioning map image and appropriately scaling the aggregation area to adapt to the capacity of the detector based on the positioning map; (5) modifying part of the loss function of the original data to make it easier to train; The step (3) of calculating the output predicted positioning map specifically comprises: Assuming that S represents a set of all target center points, (x, y) represents the horizontal and vertical coordinates on the Euclidean distance transform map, and (x', y') represents the target center point coordinates in the set S, then the Euclidean distance transform map is expressed as: Two distance transform maps, inverse distance transform map (FIDTM) and Gaussian weighted distance transform map (GDTM), are used for processing, and are specifically expressed as follows: GD = e -γD(x,y) Wherein, alpha, beta and gamma are hyperparameters, which are empirically set to 0.02, 0.75 and 0.02; C is a constant to avoid a denominator of 0, which is set to C=1; Both the two distance transform maps use the high-resolution feature pyramid network (HRFPN) as the backbone network, and then add a convolution layer and two deconvolution layers as the positioning regression; The step (4) specifically comprises: (4.1) obtaining the predicted positioning map by the local maximum value detection method given the predicted distance transform map; (4.2) dividing the predicted positioning map into 16x10 grids, counting the number of targets in each grid, and then sorting according to the number from large to small; (4.3) selecting the top 15 grids with the largest number, and obtaining the larger candidate region by the eight-connected region algorithm, i.e. selecting the aggregation area in the image; The step (4.3) further comprises: first, appropriately enlarging the connected region to make the cropped region cover the complete target, and filtering out the cropped blocks smaller than a certain threshold; and then enlarging the cropped region to a size suitable for the network, and filling the remaining part with zero; The step (5) specifically comprises: The original loss function consists of three parts: the prediction loss L of the center point. k The offset loss of the center point L off and size regression loss L size That is, the final loss is a combination of three losses: L det = L k + λ size L size + λ off L off wherein L k Focal loss, L off L1 loss, the size regression loss L size L1 loss, λ size and λ off are control parameters, respectively set to 0.1 and 1; Adopting modified GIoU loss M-IoU for size regression loss L size Modifications made; The regression loss based on the Gaussian Wasserstein distance is used to modify the original loss function.

2. The method of claim 1, wherein: The multiple convolution layers in the step (2) comprise convolution layers and deconvolution layers, wherein the convolution layer is represented as "Conv-convolution kernel-channel-stride", and the deconvolution layer is represented as "TransConv-convolution kernel-channel-stride".

3. The method of claim 2, wherein: The multiple convolution layers in the step (2) are sequentially set as Conv-1-384-1, TransConv-4-64-1, TransConv-4-64-1 and Conv-1-1.

4. The method of claim 1, wherein: The regression loss based on the Gaussian Wassertein distance modifies the original loss function, specifically: convert the bounding box of the target into a two-dimensional Gaussian distribution where x, y, h, w represent the horizontal and vertical coordinates of the bounding box and the height and width of the bounding box, respectively, m represents the mean, and Σ represents the covariance matrix ​ m = (x, y) The Wassertein distance between two different probability distributions is defined as: Wherein, mu, v represent two different probability distributions, and the variable contains all random variables (X, Y) in RnxRn, X~mu, Y~v, and the following is obtained: Wherein, m1 and m2 represent the mean values of two different two-dimensional Gaussian distributions, and Sigma1 and Sigma2 represent the covariance matrices of the two distributions. is written as: wherein, Because it is a horizontal frame detection process, according to the commutative law, Sigma1 Sigma2 = Sigma2 Sigma1, so: Among them, || || F Denotes the Frobenius norm; With a nonlinear transformation, d 2 is converted into an affine function Then, according to the form of GIoU, the affine function is written as: where f(·) denotes a nonlinear function that makes the Wasserstein distance d 2 more smooth and easy to train; choose f(·) = ln(D + 1), τ controls the whole loss, where τ = 1; In using L GWD As the regression loss, because the offset of the center point has been considered, the center point offset loss L off is ignored, so the final loss is: L det = L k + λ GWD L GWD ; where λ GWD is a control parameter, set to 2.

Images