Evaluation method for crowd positioning algorithm

Abstract

The invention relates to the technical field of computer vision, and discloses an evaluation method for a crowd positioning algorithm, which comprises the following steps: describing the matching between a prediction point and a marking point as an optimal transmission problem, and solving the optimal transmission problem to obtain a global optimal transmission scheme. And evaluating the positioning performance of the crowd positioning algorithm on the crowd graph based on the global optimal transmission scheme. According to the invention, the problem of poor evaluation accuracy of a crowd positioning algorithm in the prior art is solved.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, and in particular to an evaluation method for a crowd positioning algorithm. Background Art

[0002] The dense crowd localization task aims to locate the center position of individual heads in dense crowd scenes, which plays a vital role in subsequent upstream crowd analysis tasks, such as crowd monitoring. Dense crowd scenes have the characteristics of no category information in the annotation points, diverse annotation methods of datasets, and different sizes of human heads. With the development of crowd localization methods, designing a fair evaluation metric has become an important research field. These metrics are crucial to evaluate the consistency of crowd localization prediction results with human judgment, demonstrating their effectiveness in the real world. How to fairly evaluate model performance by measuring the difference between annotation points and prediction points while considering the characteristics of the crowd localization task is the primary issue in designing effective crowd localization evaluation metrics.

[0003] Existing evaluation metrics have been used to evaluate many crowd localization algorithms. However, to the best of our knowledge, they often do not fully consider the characteristics of crowd localization. For example, evaluation metrics such as grid mean absolute error (GAME) and mean location error (MLE) obviously ignore scale information and are difficult to meet scale adaptability. The most commonly used precision, recall, and F1 value evaluation metrics lack universality for datasets with different categories of annotations in threshold selection, and it is difficult to achieve scale adaptivity in threshold setting for point datasets. In addition, the greedy algorithm used in the matching process may also lead to incorrect matching schemes, thus affecting the final evaluation results. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides an evaluation method for a crowd positioning algorithm, which solves the problem that the evaluation accuracy of the crowd positioning algorithm in the prior art is poor.

[0005] The technical solution adopted by the present invention to solve the above problems is:

[0006] An evaluation method for crowd localization algorithm describes the matching between predicted points and labeled points as an optimal transmission problem. By solving the optimal transmission problem, a global optimal transmission scheme is obtained. The positioning performance of the crowd localization algorithm on the crowd graph is evaluated based on the global optimal transmission scheme.

[0007] As a preferred technical solution, the following steps are included:

[0008] S1, discrete measure definition: define the labeled point information of the crowd graph as a weighted labeled discrete measure, and define the predicted point information of the crowd graph as a weighted predicted discrete measure;

[0009] S2, global optimal transmission scheme calculation: calculate the global optimal transmission scheme to minimize the total transmission cost of transmitting the predicted points to the marked points;

[0010] S3, performance evaluation: The total transmission cost is used as the evaluation score of the positioning performance of the crowd localization algorithm on the crowd graph.

[0011] As a preferred technical solution, in step S1, the calculation formula of the weighted labeling discrete measure is:

[0012]

[0013] Among them, G represents the weighted labeling discrete measure, i represents the number of the labeling point, m represents the total number of labeling points, and g i represents the coordinates of the i-th annotation point, represents the x-axis coordinate of the i-th annotation point, represents the y-axis coordinate of the i-th labeled point, R represents the set of real numbers, represents the two-dimensional real number space, represents the weight of the i-th annotation point,

[0014] The calculation formula of weighted prediction discrete measure is:

[0015]

[0016] Among them, E represents the weighted prediction discrete measure, j represents the number of the prediction point, n represents the total number of prediction points, and e j represents the coordinates of the j-th prediction point, represents the x-axis coordinate of the j-th prediction point, Represents the y-axis coordinate of the predicted point, represents the weight of the j-th prediction point,

[0017] As a preferred technical solution, in step S2, an optimal transmission scheme P is sought from a set P of all transmission schemes between G and E. * , so that each prediction point e j Transfer to the marked point g i The total transmission cost is the smallest; where P represents the set of all transmission schemes between G and E, P * Represents the global optimal transmission scheme between G and E.

[0018] As a preferred technical solution, the calculation formula of the global optimal transmission solution is:

[0019]

[0020] Among them, C i,j represents the transmission cost between the i-th annotation point in G and the j-th prediction point in E, P i,j represents the amount of transmission from the i-th marked point in G to the j-th predicted point in E, Represents the transmission scheme with the minimum total transmission cost in P as P * .

[0021] As a preferred technical solution, in step S2, the global optimal transmission solution constraint condition is:

[0022]

[0023] Where P i,j ≥0.

[0024] As a preferred technical solution, in step S2, C i,j The calculation formula is:

[0025]

[0026] Where l represents the width of the crowd graph, and h represents the height of the crowd graph.

[0027] As a preferred technical solution, in step S3, P is calculated * After that, P * Normalization to obtain the target transmission solution Elements It represents the transmission value from the i-th annotation point in G to the j-th prediction point in E. The calculation formula is:

[0028]

[0029] In the formula, for The i-th row and j-th column element of represents the transmission value from the i-th annotation point in G to the j-th prediction point in E, P * The i-th row and j-th column element of represents the optimal transmission scheme from the i-th marked point in G to the j-th predicted point in E;

[0030] Then calculate the final total transmission cost and use it as the score for evaluating the performance of the crowd positioning algorithm. The calculation formula for the final total transmission cost is:

[0031]

[0032] Where S-OTC represents the final total transmission cost.

[0033] As a preferred technical solution, in step S1, and The calculation formulas for and are:

[0034]

[0035] In the formula,

[0036] in, Represents the set of inverses of the scale information of all annotation points, min(·) represents the minimum value calculation, max(·) represents the maximum value calculation, s i Indicates the head size of the i-th annotation point;

[0037]

[0038] As a preferred technical solution, in step S1, considering the scale prior information, correcting the head size of the outlier point includes the following steps:

[0039] S11, according to the local outlier factor scores of the marked points, the marked points are divided into two categories: internal points and outlier points, so as to distinguish individuals in the crowd from isolated individuals; wherein, the internal points correspond to individuals in the crowd, and the outlier points correspond to isolated individuals;

[0040] S12, calculate the head size of all annotation points based on the k nearest neighbor distance, the formula is: scale i =knn i ; Among them, scale i represents the head size of the i-th annotation point, km i represents the k-nearest neighbor distance of the i-th annotation point; and, obtains the head size of the intrinsic point, the formula is s i =scale i ;

[0041] S13, implementing a distance correction strategy for the outlier point, and correcting the head size of the outlier point by capturing the relationship with the adjacent intrinsic points of the outlier point, including the following steps:

[0042] S131, quantify the discreteness between the outlier point and the adjacent intrinsic point of the outlier point: the calculation formula of discreteness is: disp i =d ij / knn j ; where disp represents the discreteness of the i-th annotation point, d ij represents the Euclidean distance between the i-th annotation point and the j-th prediction point, knn j Represents the k-nearest neighbor distance of the j-th prediction point;

[0043] S132, according to the dispersion disp i Calculate the scaling factor i , the calculation formula is: factor i =Sigmoid(disp i );where Sigmoid(·) represents the S-type function;

[0044] S133, estimated scale based on outliers i , scaling factor i , and the head size s of the inner points adjacent to the outlier j , derive the corrected head size s of the outlier i =scale i +factor i ×s j .

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] (1) The present invention proposes an evaluation index for dense crowd positioning based on optimal transmission, which provides an overall description of the differences between predicted points and annotated points by seeking a stable matching solution between predicted points and annotated points;

[0047] (2) The present invention proposes a scale-sensitive adaptive method for head estimation, which estimates the size of a head based only on point annotation information by discovering outliers in a point group and correcting the k-nearest neighbor distances of the outliers.

[0048] (3) The present invention proposes a normalized cost matrix calculation method, which normalizes the distance between the predicted point and the labeled point based on the image resolution, thereby enhancing scale invariance to reflect the actual scene;

[0049] (4) Experiments on different dense crowd datasets show that the proposed solution is scale-sensitive and provides a stable, fair and effective evaluation result that is better than the most advanced evaluation indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the overall structure of the present invention;

[0051] Figure 2 for Figure 1 One of the partial enlarged pictures;

[0052] Figure 3 for Figure 1 The second partial enlarged picture;

[0053] Figure 4This is the error sensitivity experimental verification diagram of the evaluation index. DETAILED DESCRIPTION

[0054] The present invention will be further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0055] Example 1

[0056] like Figures 1 to 4 As shown in the figure, the present invention proposes a new evaluation index method, which aims to achieve fair and effective evaluation of the performance of crowd positioning algorithm in the scenario with only point annotation data. First, the scheme describes the matching between predicted points and annotated points as an optimal transmission problem, which is convenient for exploring stable and accurate matching results. Among them, the predicted point refers to the position result of the center point of the human head predicted by the crowd positioning algorithm, and the annotated point refers to the position of the real center point of the human head.

[0057] Specifically, the present invention first converts the prediction point information and the annotation point information into two weighted discrete measures: the prediction measure and the annotation measure. Then, an adaptive scale-sensitive method is proposed to integrate the scale prior information into the annotation measure, thereby enhancing the scale invariance. At the same time, a simple and effective normalization technique is used to correct the cost matrix between the two measures. This method effectively reduces the impact of scale changes caused by changes in image resolution. The evaluation index is based on the corrected cost matrix and two discrete weighted measures. By solving the optimal transmission problem, a globally optimal transmission scheme is obtained to minimize the total transmission cost from the prediction measure to the annotation measure. Finally, the total transmission cost is defined as a new scale-invariant evaluation index.

[0058] The goal of this invention is to provide a stable, fair and effective evaluation index for dense crowd positioning algorithms, and to meet the characteristics of crowd positioning scenarios where the annotated points have no category information, the annotation methods are different, and the scale varies greatly. Figure 1 shown.

[0059] The framework evaluates the performance of the crowd localization algorithm by calculating the cost of the optimal transmission scheme between the predicted points and the annotated points. First, two weighted discrete measures are defined to convert the predicted point information and the annotated point information into corresponding discrete measures (discrete measures can be understood as a distribution containing weights and location information), while considering the scale (in this embodiment, the size of the human head) prior information. Then, a cost matrix is ​​constructed using the distance function normalized by the image scale, which consists of paired unit costs between the predicted points and the annotated points. In order to find the optimal transmission scheme from the predicted measure to the annotated measure, the optimal transmission problem is solved on the cost matrix. Finally, the total transmission cost of the transmission scheme is calculated as an evaluation index, and the score represents the difference between the predicted point and the annotated point.

[0060] I. Formulation of the Optimal Transport Problem

[0061] The labeled point information of each crowd map is re - defined as a weighted discrete measure, denoted as where represents the coordinates of the \(i\) - th labeled point, refers to the adaptive weight of the head region around the \(i\) - th labeled point, and \(m\) is the total number of labeled points. Similarly, the information of the predicted points is represented as another weighted discrete measure where are the coordinates, is the weight of the \(j\) - th predicted point, and \(n\) is the total number of predicted points. It should be noted that represents the \(x\) - axis coordinate of the \(i\) - th labeled point, represents the \(y\) - axis coordinate of the \(i\) - th labeled point, represents the \(x\) - axis coordinate of the \(j\) - th predicted point, represents the \(y\) - axis coordinate of the predicted point. Here, the \(x\) - axis represents the horizontal direction and the \(y\) - axis represents the vertical direction. However, such settings of the \(x\) - axis and \(y\) - axis are only for example. In fact, the \(x\) - axis and \(y\) - axis can be set in various ways, which does not affect the implementation of the present invention nor the embodiment of the inventive concept of the present invention.

[0062] Based on \(G\) and \(E\), the optimal transport problem aims to find a globally optimal transport plan \(P\) * = \(P\) i,j \(\in\{P|i = 1,\ldots,m,j = 1,\ldots,n\}\) such that the total transport cost of transporting each predicted point \(e\) j to the labeled point \(g\) i is minimized.

[0063] The formula is as follows:

[0064]

[0065] \(P\) i,j \(\geq0(i = 1,\ldots,m\) and \(j = 1,\ldots,n)\),

[0066] where \(C\) i,j is the element in the \(i\) - th row and \(j\) - th column of the cost matrix \(C\), and \(C\) i,j reflects the cost of the transport relationship between \(g\) i and \(e\) j . The weights \(w\) g and \(w\) e are regarded as the transport quality and ensure mass conservation (i.e., the marginal distribution of the transport plan \(P\) with respect to variable \(i\) is equal to the weight of the \(j\) - th predicted point, the marginal distribution of the transport plan \(P\) with respect to variable \(j\) is equal to the weight of the \(i\) - th labeled point, the sum of the weights of the measure \(G\) is 1, the sum of the weights of the measure \(E\) is 1, and the transport volume in the transport plan \(P\) cannot be negative) to balance the problem of non - uniformity caused by over - or under - estimation of the prediction quantity.

[0067] Finally, the linear programming and solver are used to obtain the exact optimal transmission solution. Therefore, the total transmission cost from G to E is to use the global optimal transmission solution P * The calculated value is used as the final evaluation score. The average value of the evaluation scores of all test images is used to evaluate the performance of the positioning algorithm on the dataset.

[0068] 2. Construction of Weighted Discrete Measures

[0069] First, we assume that each prediction point contributes equally to the prediction result. Therefore, the prediction measure is considered to be uniformly distributed, where all points have the same weight, as follows:

[0070]

[0071] Secondly, the position of the human annotation point is located at the center of the pedestrian's head. However, in an image, the size of the human head is very different. The head scale has a great impact on the evaluation of most crowd localization methods. Therefore, according to the head size of the annotation point, different weight parameters w are further introduced in the annotation measure G. g :

[0072]

[0073] where s i is the head size of the i-th annotation point. Each weight is inversely proportional to the corresponding head size, and the entire weight is normalized.

[0074] 3. Adaptive scale-sensitive algorithm

[0075] However, estimating head size for a dataset that only contains point annotation information is a challenging task. Recent crowd analysis work uses the k-nearest neighbor distance of each point to reflect the head size in crowded scenes. However, in real scenes, there are a large number of isolated individuals far away from the crowd. Although the k-nearest neighbor distance estimates the head size of individuals in crowded areas relatively accurately, the head size estimates of these isolated individuals often deviate significantly from the actual size. Therefore, the focus of solving scale estimation is to estimate the head size of these isolated individuals' outliers, aiming to provide a more accurate scale prior for annotation measurement. To this end, the present invention proposes an adaptive scale-sensitive estimation method, which can obtain more accurate head size estimation using only point annotations.

[0076] First, according to the local outlier factor score of the annotation points (which can be achieved by the existing technology and will not be repeated in this embodiment), they are divided into two categories: internal points and outliers, so as to distinguish individuals in the crowd from isolated individuals. Secondly, since the k nearest neighbor distance effectively captures the correlation between the head size and distance of these internal points, the head size scale of all annotation points is directly calculated based on the k nearest neighbor distance. i =knn i (The existing technology can be used to achieve this, and the specific process will not be repeated in this embodiment); wherein, scale i represents the head size of the i-th annotation point, knn i represents the k nearest neighbor distance points of the i-th annotation point. For the internal point, its head size is directly determined by scale i Get, that is, s i =scale i ; Third, implement a distance correction strategy for outliers. The goal is to correct the outlier head size s by capturing the relationship with its adjacent intrinsic points i The strategy initially quantifies the discreteness between outliers and their adjacent intrinsic points. The discreteness of the i-th annotation point can be formulated as disp i =d ij / knn j ; where d ij represents the Euclidean distance between the i-th annotation point and the j-th prediction point, knn j represents the k nearest neighbor distance of the j-th prediction point. Then, according to the dispersion disp i , the scaling factor is calculated using nonlinear transformation i , factor i It reflects the degree of perspective between the outlier point and its adjacent internal points in the graph. The formula is expressed as factor i =Sigmoid(disp i ), where Sigmoid(·) represents the S-type function. Finally, according to the estimated size scale of the outlier i , scaling factor i and the head size s of the inner points adjacent to the outlier j , derive the corrected head size s of the outlier i =scale i +factor i ×s j The adaptive scale-sensitive algorithm not only provides a scale prior for the annotation measurement, but is also applicable to all crowd positioning datasets, ensuring the universality of the evaluation index.

[0077] 4. Normalized cost matrix

[0078] A common approach is to define the squared Euclidean distance between the coordinates of two points as the value of each unit cost. However, images evaluated in crowd localization tasks usually have different resolutions. This causes the Euclidean distance to vary with resolution, which affects the final total transmission cost and leads to unfair evaluation. To address this issue, a normalized Euclidean distance function is designed to calculate the value of each unit cost. Specifically, the distance between two points in the horizontal and vertical coordinates is normalized by dividing the distance between the two points by the width and height of the image, respectively.

[0079] The element C of the cost matrix C i,j It is obtained by normalizing the Euclidean distance between the i-th annotation point in G and the j-th prediction point in E, C i,j The calculation formula is as follows:

[0080]

[0081] in and They are marked points g i and the predicted point e j The coordinates of , l and h represent the width and height of the image respectively. Therefore, the proposed normalized distance can keep the cost matrix unaffected by changes in image resolution, thereby ensuring the fairness of the evaluation index.

[0082] 5. Evaluation score calculation

[0083] After computing the two discrete measures G and E and the corresponding cost matrix C, the optimal transmission problem is solved to find the global optimal transmission solution P * Then P * Normalization to obtain the target transmission solution Medium Element represents the transmission value from the i-th annotation point in G to the j-th prediction point in E, as shown below:

[0084]

[0085] Finally, the final total transmission cost is the score of the evaluation index:

[0086]

[0087] Example 2

[0088] like Figures 1 to 4As shown, due to the adoption of the technical solution of the present invention, the following technical effects are achieved: stability verification is performed on the public dataset UCF-QNRF. In order to verify the sensitivity of the evaluation index to the prediction result error, we perform three disturbance operations of adding, deleting, and moving on the prediction results of the existing algorithm LSCCNN. Each disturbance step performs a single disturbance operation on 5%, 10%, and 20% of the prediction points respectively. Then calculate the score ranking of the prediction results before and after the disturbance by the evaluation index. Finally, the sensitivity of the evaluation index to these error disturbances is evaluated by statistically analyzing the consistency between the evaluation index ranking and human preference. From Figure 4 It can be seen from the statistics that the evaluation indicators we proposed are more consistent with human preferences than the common accuracy, recall and F1 value, reflecting that the proposed method is more sensitive to both counting errors and spatial deviations, providing a more reliable and robust indicator for crowd positioning evaluation.

[0089] As described above, the present invention can be preferably implemented.

[0090] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0091] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. According to the technical essence of the present invention, within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement made to the above embodiment still falls within the protection scope of the technical solution of the present invention.

Claims

1. An evaluation method for crowd positioning algorithm, characterized in that: The matching between predicted points and marked points is described as an optimal transmission problem. The global optimal transmission solution is obtained by solving the optimal transmission problem. The positioning performance of the crowd localization algorithm on the crowd graph is evaluated based on the global optimal transmission solution.

2. The evaluation method for crowd positioning algorithm according to claim 1, characterized in that: The following steps are involved: S1, discrete measure definition: define the labeled point information of the crowd graph as a weighted labeled discrete measure, and define the predicted point information of the crowd graph as a weighted predicted discrete measure; S2, global optimal transmission scheme calculation: calculate the global optimal transmission scheme to minimize the total transmission cost of transmitting the predicted points to the marked points; S3, performance evaluation: The total transmission cost is used as the evaluation score of the positioning performance of the crowd localization algorithm on the crowd graph.

3. The evaluation method for crowd positioning algorithm according to claim 2, characterized in that: In step S1, the calculation formula of the weighted labeling discrete measure is: Among them, G represents the weighted labeling discrete measure, i represents the number of the labeling point, m represents the total number of labeling points, and g i represents the coordinates of the i-th annotation point, represents the x-axis coordinate of the i-th annotation point, represents the y-axis coordinate of the i-th labeled point, R represents the set of real numbers, represents the two-dimensional real number space, represents the weight of the i-th annotation point, The calculation formula of weighted prediction discrete measure is: Among them, E represents the weighted prediction discrete measure, j represents the number of the prediction point, n represents the total number of prediction points, and e j represents the coordinates of the j-th prediction point, represents the x-axis coordinate of the j-th prediction point, Represents the y-axis coordinate of the predicted point, represents the weight of the j-th prediction point, 4. The evaluation method for crowd positioning algorithm according to claim 3, characterized in that: In step S2, an optimal transmission scheme P is sought from the set P of all transmission schemes between G and E. * , so that each prediction point e j Transfer to the marked point g i The total transmission cost is the smallest; where P represents the set of all transmission schemes between G and E, P * Represents the global optimal transmission scheme between G and E.

5. The evaluation method for crowd positioning algorithm according to claim 4, characterized in that: The calculation formula of the global optimal transmission solution is: Among them, C i,j represents the transmission cost between the i-th annotation point in G and the j-th prediction point in E, P i,j represents the amount of transmission from the i-th marked point in G to the j-th predicted point in E, Represents the transmission scheme with the minimum total transmission cost in P as P * .

6. The evaluation method for crowd positioning algorithm according to claim 5, characterized in that: In step S2, the global optimal transmission solution constraint is: Where P i,j ≥0.

7. The evaluation method for crowd positioning algorithm according to claim 6, characterized in that: In step S2, C i,j The calculation formula is: Where l represents the width of the crowd graph, and h represents the height of the crowd graph.

8. The evaluation method for crowd positioning algorithm according to claim 7, characterized in that: In step S3, P is calculated * After that, P * Normalization to obtain the target transmission solution Elements It represents the transmission value from the i-th annotation point in G to the j-th prediction point in E. The calculation formula is: In the formula, for The i-th row and j-th column element of represents the transmission value from the i-th annotation point in G to the j-th prediction point in E, P * The i-th row and j-th column element of represents the optimal transmission scheme from the i-th marked point in G to the j-th predicted point in E; Then calculate the final total transmission cost and use it as the score for evaluating the performance of the crowd positioning algorithm. The calculation formula for the final total transmission cost is: Where S-OTC represents the final total transmission cost.

9. The evaluation method for a crowd positioning algorithm according to any one of claims 3 to 8, characterized in that: In step S1, and The calculation formulas for and are: In the formula, in, Represents the set of inverses of the scale information of all annotation points, min(·) represents the minimum value calculation, max(·) represents the maximum value calculation, s i Indicates the head size of the i-th annotation point; 10. The evaluation method for crowd positioning algorithm according to claim 9, characterized in that: In step S1, the head size of the outlier is corrected by considering the scale prior information, including the following steps: S11, according to the local outlier factor scores of the marked points, the marked points are divided into two categories: internal points and outlier points, so as to distinguish individuals in the crowd from isolated individuals; wherein, the internal points correspond to individuals in the crowd, and the outlier points correspond to isolated individuals; S12, calculate the head size of all annotation points based on the k nearest neighbor distance, the formula is: scale i =knn i ; Among them, scale i represents the head size of the i-th annotation point, knn i represents the k-nearest neighbor distance of the i-th annotation point; and, obtains the head size of the intrinsic point, the formula is s i =scale i ; S13, implementing a distance correction strategy for the outlier point, and correcting the head size of the outlier point by capturing the relationship with the adjacent intrinsic points of the outlier point, including the following steps: S131, quantify the discreteness between the outlier point and the adjacent intrinsic point of the outlier point: the calculation formula of discreteness is: disp i =d ij / knn j ; where disp represents the discreteness of the i-th annotation point, d ij represents the Euclidean distance between the i-th annotation point and the j-th prediction point, knn j Represents the k-nearest neighbor distance of the j-th prediction point; S132, according to the dispersion disp i Calculate the scaling factor i , the calculation formula is: factor i =Sigmoid(disp i );where Sigmoid(·) represents the S-type function; S133, estimated scale based on outliers i , scaling factor i , and the head size s of the inner points adjacent to the outlier j , derive the corrected head size s of the outlier i =scale i +factor i ×s j .

Images