Evaluation method and device for dense small target detection task, equipment and medium
By calculating the average accuracy and target density in intensive small object detection tasks and analyzing their correlation, an evaluation method for intensive small object detection tasks is provided, which solves the problem of inapplicability of existing evaluation indicators and improves the detection performance of the model in dense scenarios.
Patent Information
- Application Number
- CN202411735382.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-04-11
AI Technical Summary
The existing target detection evaluation indicators are mainly suitable for sparse large target scenarios and are not completely applicable to intensive small target detection tasks, which makes it difficult for traditional target detection deep learning models to accurately detect the boundaries of each cell in cell clusters, and the detection performance is degraded.
An evaluation method for intensive small object detection tasks is provided. By obtaining the image to be detected and the truth value is marked, the average accuracy and target density are calculated, and the evaluation results are determined based on the correlation coefficient and significance level indicators, and the performance of the target detection model under different target densities is determined.
This method can effectively evaluate the performance of the model under various target densities, provide important support information, provide guidance for the research and development, testing and application of the model, and improve the accuracy and efficiency of intensive small-objective detection.
Smart Images
Figure CN119810594A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an evaluation method, device, equipment and medium for dense small target detection tasks. Background Art
[0002] Although existing cell image analysis technology and deep learning models have achieved remarkable results in sparse large target detection tasks, in actual clinical processes, cell clusters often exist in cytopathology smears, that is, multiple cells are closely arranged and overlapped, which poses a huge challenge to cell detection.
[0003] Detecting the rectangular bounding box of each cell in a cell cluster is a typical dense small target detection task. In this case, traditional target detection deep learning models and evaluation methods may fail because they are usually designed to detect independent and scattered targets and have difficulty handling dense and overlapping cell clusters. In cell clusters, due to the dense arrangement and mutual interference between cells, traditional target detection deep learning models often find it difficult to accurately define the boundaries of each cell, resulting in reduced detection performance. In addition, existing target detection evaluation indicators are mainly applicable to sparse large target scenarios and are not fully applicable to dense small target detection tasks. Summary of the invention
[0004] The present invention provides an evaluation method, device, equipment and medium for dense small target detection tasks, so as to solve the defect that the existing target detection evaluation indicators in the prior art are mainly applicable to sparse large target scenes and are not fully applicable to dense small target detection tasks.
[0005] The present invention provides an evaluation method for a dense small target detection task, comprising the following steps: Acquire an image to be detected and a true value of an annotation of the image to be detected, and determine the number of objects in the true value of the annotation; Inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model, and determining an average precision based on the target detection result and the labeled true value; Determining a target density based on the target quantity, and determining an evaluation result based on the target density and the average accuracy; Based on the evaluation result, a target density interval in which the target detection model has poor performance is determined.
[0006] According to an evaluation method for a dense small target detection task provided by the present invention, the evaluation result is determined based on the target density and the average accuracy, including: Determining a correlation coefficient and a significance level indicator based on the target density and the average precision; The evaluation result is determined based on the correlation coefficient and the significance level indicator.
[0007] According to an evaluation method for a dense small target detection task provided by the present invention, the evaluation result is determined based on the correlation coefficient and the significance level index, including: When the significance level index is greater than a first threshold, determining that the evaluation result is that the average accuracy has no correlation with the target density; When the significance level index is less than the first threshold and the correlation coefficient is less than the second threshold, determining that the evaluation result is that the average accuracy is negatively correlated with the target density, and as the target density increases, the average accuracy decreases; When the significance level index is less than the first threshold and the correlation coefficient is greater than the second threshold, it is determined that the evaluation result is that the average accuracy is positively correlated with the target density, and as the target density increases, the average accuracy increases.
[0008] According to an evaluation method for a dense small target detection task provided by the present invention, the target density is determined based on the number of targets, including: In the case where the image pixel areas of the images to be detected are different, determining the image pixel areas of the images to be detected, and determining the target density based on the image pixel areas, the linear scaling factor and the number of targets; In a case where the image pixel areas of the images to be detected are the same, the target density is determined based on the target quantity and the linear scaling factor.
[0009] According to an evaluation method for a dense small target detection task provided by the present invention, the average accuracy is determined based on the target detection result and the labeled true value, including: Determine the degree of overlap between the predicted box in the target detection result and the real box in the labeled true value; Based on the degree of overlap, an average precision is determined.
[0010] According to an evaluation method for a dense small target detection task provided by the present invention, the target density interval with poor performance of the target detection model is determined based on the evaluation result, and then the method further includes: A training data set of the target density interval is constructed, and the target detection model is optimized based on the training data set.
[0011] The present invention also provides an evaluation device for a dense small target detection task, comprising the following units: An acquisition unit, used to acquire an image to be detected and a true value of an annotation of the image to be detected, and determine the number of targets in the true value of the annotation; An average precision determining unit is used to input the image to be detected into a target detection model, obtain a target detection result output by the target detection model, and determine an average precision based on the target detection result and the annotation true value; An evaluation result determination unit, configured to determine a target density based on the target quantity, and to determine an evaluation result based on the target density and the average accuracy; The target density interval determination unit is used to determine the target density interval where the performance of the target detection model is poor based on the evaluation result.
[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an evaluation method for dense small target detection tasks as described in any one of the above is implemented.
[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described evaluation methods for dense small target detection tasks.
[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-described evaluation methods for dense small target detection tasks.
[0015] The evaluation method, device, equipment and medium for dense small target detection tasks provided by the present invention are oriented to dense small target monitoring, cell detection in a typical scene-type cell cluster image, and average accuracy and target density are calculated for each image separately, and the correlation between average accuracy and target density is further evaluated, and the correlation coefficient is calculated to determine the evaluation result. The evaluation method for dense cell target detection tasks described in the present invention can effectively evaluate the model performance under various target densities, and can provide important support information for the research and development, testing, and application of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1This is one of the flow charts of the evaluation method for dense small target detection tasks provided by the present invention.
[0018] Figure 2 This is the second flow chart of the evaluation method for dense small target detection tasks provided by the present invention.
[0019] Figure 3 It is a schematic diagram of the evaluation of dense small target detection tasks combined with target density provided by the present invention.
[0020] Figure 4 It is a schematic diagram of the optimization training of the guidance model provided by the present invention on images of different densities.
[0021] Figure 5 It is a schematic diagram of the cell cluster image data provided by the present invention.
[0022] Figure 6 It is a schematic diagram of the target detection results of the Faster R-CNN model provided by the present invention in cell-dense and cell-sparse scenarios.
[0023] Figure 7 It is the target detection result of the CenterNet model provided by the present invention in the cell-dense and cell-sparse scenarios.
[0024] Figure 8 It is a schematic diagram of the correlation between AP and target density provided by the present invention.
[0025] Fig. 9 It is a schematic diagram of the structure of an evaluation device for dense small target detection tasks provided by the present invention.
[0026] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] Among the related technologies, cytopathology is one of the key tools for diagnosing many diseases. Cell samples are collected from patient tissues or body fluids and then processed through a series of sophisticated processing steps, such as centrifugation, smearing, fixation, and staining, so that the cell structure can be clearly displayed. Pathologists observe and analyze the morphology, size, nuclear morphology and other characteristics of these cells under a microscope to determine whether there are abnormal cells, providing a basis for the preliminary diagnosis of the disease.
[0029] In recent years, with the rapid development of computer science and artificial intelligence technology, computer-aided diagnosis (CAD) has been increasingly used in the field of cell image analysis. By training deep learning models, CAD systems can quickly identify and locate each cell in pathological images, greatly improving the work efficiency of pathologists. In addition, accurate detection of each cell helps pathologists more accurately determine whether the cell is abnormal, thereby detecting mild and minor lesions and avoiding missed detection and misdiagnosis. The trained target detection deep learning model can quickly detect cells in the image and obtain a rectangular bounding box for each cell.
[0030] In the field of object detection, the main indicators for evaluating model performance include intersection over union (IoU), precision, recall, average precision (AP), mean average precision (MAP), and accuracy, etc. These indicators perform well in sparse large object detection tasks and can fully reflect the detection capabilities of the model.
[0031] Although existing cell image analysis technology and deep learning models have achieved remarkable results in sparse large target detection tasks, in actual clinical processes, cell clusters often exist in cytopathology smears, that is, multiple cells are closely arranged and overlapped, which poses a huge challenge to cell detection.
[0032] Detecting the rectangular bounding box of each cell in a cell cluster is a typical dense small target detection task. In this case, traditional target detection deep learning models and evaluation methods may fail because they are usually designed to detect independent and scattered targets and have difficulty handling dense and overlapping cell clusters. In cell clusters, due to the dense arrangement and mutual interference between cells, traditional target detection deep learning models often find it difficult to accurately define the boundaries of each cell, resulting in reduced detection performance. In addition, existing target detection evaluation indicators are mainly applicable to sparse large target scenarios and are not fully applicable to dense small target detection tasks.
[0033] Figure 1 is one of the flow charts of the evaluation method for dense small target detection tasks provided by the present invention, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .
[0034] Step 110, obtaining an image to be detected and a true value of an annotation of the image to be detected, and determining the number of objects in the true value of the annotation.
[0035] Specifically, the image to be detected and the annotated true value of the image to be detected can be obtained. Here, the image to be detected, that is, the image that needs to be subsequently subjected to target detection, can be pre-captured by an image acquisition device, can be captured in real time, or can be downloaded or scanned from the Internet, and the embodiments of the present invention do not make specific limitations on this.
[0036] The annotation truth value marks the spatial position of all objects in each image, and is usually obtained manually or semi-automatically. Preferably, the annotation truth value is usually presented in the form of a rectangular bounding box, which can also be a rectangle, polygon, etc. The annotation software includes Labelme, LabelImg, Label Studio, CVAT, COCO Annotator, X-anything labeling, etc., which is not specifically limited in the embodiment of the present invention.
[0037] Furthermore, the target number in the labeled true value may be determined, that is, the number of cells in the labeled true value may be determined.
[0038] Step 120: input the image to be detected into a target detection model to obtain a target detection result output by the target detection model, and determine the average accuracy based on the target detection result and the annotation true value.
[0039] Specifically, the image to be detected is input into the target detection model to obtain the target detection result output by the target detection model. The target detection model can be a deep learning model, which can detect the spatial positions of all targets in each image. Preferably, the monitoring result is presented in the form of a rectangular bounding box, and the score of each bounding box is recorded.
[0040] It can be understood that the annotation true value and the target detection result are the basis for executing the evaluation method described in the present invention, and the average precision AP can be determined based on the target detection result and the annotation true value.
[0041] For example, determine the degree of overlap (IoU) between the predicted box in the target detection result and the real box in the labeled truth value, and then determine the average precision based on the degree of overlap.
[0042] Here, IoU is the ratio of the overlapping area between the predicted box and the true box to the total area of the predicted box and the true box, which is used to quantify the degree of overlap between the predicted box and the true box.
[0043] In the field of general object detection, the two commonly used AP indicators are AP50 and AP75. AP50 represents the average accuracy of the model when the loU threshold is set to 0.5, that is, the predicted box and the real box have more than 50% overlap, which is considered to be a correct detection. AP75 represents the average accuracy of the model when the loU threshold is set to 0.75, that is, the predicted box and the real box have more than 75% overlap, which is considered to be a correct detection.
[0044] In the sparse target detection scenario, the target detection results of all images are usually integrated together to calculate the average precision AP index. In the dense small target scenario that this application focuses on, each image has many target objects, so the average precision AP index is calculated for each image separately.
[0045] Here, commonly used target detection deep learning models include Faster R-CNN (Faster Region-based Convolutional Neural Network), Grid R-CNN (Grid Region-based Convolutional Neural Network), SSD (Single Shot MultiBox Detector), CenterNet, ATSS (Adaptive Training Sample Selection), YOLO, etc., which are not specifically limited in the embodiments of the present invention.
[0046] Here, the average precision AP evaluates the performance of an object detection model by calculating the precision and recall under different thresholds.
[0047] Step 130, determining the target density based on the target quantity, and determining the evaluation result based on the target density and the average accuracy.
[0048] Specifically, the target density may be determined based on the target quantity, and the evaluation result may be determined based on the target density and the average accuracy.
[0049] For example, based on the target density and the average precision, the correlation coefficient and the significance level index are determined, and based on the correlation coefficient and the significance level index, the evaluation result is determined.
[0050] Step 140: Determine a target density interval where the target detection model has poor performance based on the evaluation result.
[0051] Specifically, after obtaining the evaluation results, a target density interval in which the performance of the target detection model is poor may be determined based on the evaluation results.
[0052] It can be understood that the evaluation method for the dense small target detection task described in the embodiment of the present invention is applicable to different types of labeling software and different types of target detection deep learning models.
[0053] It should be noted that the traditional target detection evaluation scheme combines the annotation results of all images in the test set with the model detection results, and uniformly calculates the classic evaluation index scheme. The disadvantage is that it does not distinguish the impact of target density on model performance. The method provided in the embodiment of the present invention is aimed at dense small target monitoring, cell detection in a typical scene-type cell cluster image, and calculates the average accuracy and target density for each image separately, and further evaluates the correlation between the average accuracy and the target density, calculates the correlation coefficient, and determines the evaluation result. The evaluation method for dense cell target detection tasks described in the present invention can effectively evaluate the model performance of the model under various target densities, and can provide important support information for the research and development, testing, and application of the model.
[0054] Based on the above embodiment, the step 130 of determining the evaluation result based on the target density and the average accuracy includes: Step 131, determining a correlation coefficient and a significance level index based on the target density and the average precision; Step 132: Determine the evaluation result based on the correlation coefficient and the significance level indicator.
[0055] Specifically, based on the target density and average precision, a scatter plot is drawn, and the Pearson correlation analysis method is used, with the target density of all images as X and the average precision (AP index) as Y, to determine the correlation coefficient r and the significance level index p. Tools for performing this Pearson correlation analysis include MATLAB, Python, etc. For example, for the Matlab tool, the execution code is [r,p] = corr(X,Y). When the size of all images in the test set is the same, a scatter plot of the AP index versus the number of targets can also be directly drawn, and the linear correlation parameters r and p can be calculated.
[0056] Based on the above embodiment, step 132 includes: When the significance level index is greater than a first threshold, determining that the evaluation result is that the average accuracy has no correlation with the target density; When the significance level index is less than the first threshold and the correlation coefficient is less than the second threshold, determining that the evaluation result is that the average accuracy is negatively correlated with the target density, and as the target density increases, the average accuracy decreases; When the significance level index is less than the first threshold and the correlation coefficient is greater than the second threshold, it is determined that the evaluation result is that the average accuracy is positively correlated with the target density, and as the target density increases, the average accuracy increases.
[0057] Specifically, when the significance level index p is greater than a first threshold, it is determined that the evaluation result is that the average precision (AP index) has no correlation with the target density, wherein the first threshold is 0.5.
[0058] When the significance level index p is less than the first threshold and the correlation coefficient r is less than the second threshold, the evaluation result is determined to be that the average accuracy is negatively correlated with the target density, and as the target density increases, the average accuracy (AP index) decreases, where the second threshold is 0.
[0059] When the significance level index p is less than the first threshold and the correlation coefficient r is greater than the second threshold, it is determined that the evaluation result is that the average accuracy is positively correlated with the target density, and as the target density increases, the average accuracy (AP index) increases.
[0060] Based on the above embodiment, determining the target density based on the target quantity in step 130 includes: Step 210, when the image pixel areas of the images to be detected are different, determining the image pixel areas of the images to be detected, and determining the target density based on the image pixel areas, the linear scaling factor and the number of targets; Step 220 , when the image pixel areas of the images to be detected are the same, determine the target density based on the number of targets and the linear scaling factor.
[0061] Specifically, when the image pixel areas of the images to be detected are different, the image pixel area of the images to be detected (ie, the length×height of the images to be detected) is determined.
[0062] Then, based on the image pixel area, linear scaling factor, and number of objects, the object density is determined as follows: Target density = k*number of targets / image pixel area The parameter k is a linear scaling factor. A suitable k value can make the target density value more readable. For example, if k=10000, the target density means how many targets there are per 10000 pixels in the image on average.
[0063] It should be noted that in some scenarios, the pixel areas of all images are the same. For example, all cell images are cropped from whole slide pathology images, and the image size is unified to 512×512. In this case, the target density is directly linearly related to the number of targets, and the target density can be abbreviated as: Target density = k*number of targets That is, when the image pixel areas of the images to be detected are the same, the target density is determined based on the number of targets and the linear scaling factor.
[0064] For each image, calculate the AP index of the object detection model, such as AP50 and AP75. The AP index can be calculated using COCO Evaluation or other Python and MALTAB tools.
[0065] In the related technology, in the dense cell detection scenario, these evaluation indicators cannot fully and accurately reflect the performance of the model, which makes it difficult to guide the model to perform further optimization training on different target density images, and it is difficult to select the deep learning model that best matches the image density.
[0066] Based on the above problem, step 140 further includes: Step 141, constructing a training data set of the target density interval, and optimizing the target detection model based on the training data set.
[0067] Specifically, a training data set of target density intervals is constructed, and the target detection model is optimized based on the training data set.
[0068] The length of the target density interval depends on the effective density interval of the data set, for example, 1 / 4 or 1 / 8 of the effective density interval length is selected, or the target density interval with poor performance is manually determined based on a scatter plot of the target density of the target detection model AP indicator.
[0069] For the density range with poor model performance, more training data sets in this density range are constructed in a targeted manner to further optimize the model.
[0070] When the model is applied, the target density of the image to be processed is first estimated by manual setting or automatic estimation. Automatic estimation can be evaluated based on the average gray value and color scale curve of the image. For example, for the dense cell detection task in the embodiment of the present invention, the darker the image, the lower the average gray value, and the greater the cell density.
[0071] Prioritize deep learning models with excellent performance within the estimated target range, and then consider the efficiency of the model.
[0072] Based on any of the above embodiments, Figure 2This is the second flow chart of the evaluation method for dense small target detection task provided by the present invention, such as Figure 2 As shown, the evaluation method for dense small target detection tasks described in the embodiment of the present invention is mainly divided into three steps.
[0073] S1, obtain the true value of the target detection test data and the target detection results of the deep learning model.
[0074] S2, evaluation of dense small target detection task combined with target density.
[0075] S3 guides the optimization training of the model on images of different densities, as well as the model selection in practical applications.
[0076] Based on the above embodiments, Figure 3 is a schematic diagram of the evaluation of the dense small target detection task combined with the target density provided by the present invention, such as Figure 3 As shown: S21, obtain the annotation results of each image and the deep learning target detection results.
[0077] S22, for each image, count the pixel area of the image (i.e., the length × height of the image), mark the number of targets in the true value, and determine the target density.
[0078] Target density = k*number of targets / image pixel area The parameter k is a linear scaling factor. A suitable k value can make the target density value more readable. For example, if k=10000, the target density means how many targets there are per 10000 pixels in the image on average.
[0079] It should be noted that in some scenarios, the pixel areas of all images are the same. For example, all cell images are cropped from whole slide pathology images, and the image size is unified to 512×512. In this case, the target density is directly linearly related to the number of targets, and the target density can be abbreviated as: Target density = k*number of targets S23, for each image, calculate the AP index of the model, such as AP50 and AP75. The AP index can be calculated using COCO Evaluation or other Python and MALTAB tools.
[0080] S24, for the entire test set, calculate the correlation between the evaluation index AP and the target density.
[0081] Based on the above embodiments, Figure 4 is a schematic diagram of the optimization training of the guidance model provided by the present invention on images of different densities, such as Figure 4As shown, S31, the correlation between model performance and target density is evaluated, and the performance of the model on images with different target densities is clarified.
[0082] S32, according to the evaluation result of S31, the target density interval with poor model performance is determined. The length of the density interval depends on the effective density interval of the data set, for example, 1 / 4 or 1 / 8 of the effective density interval length is selected, or the target density interval with poor performance is manually determined according to the scatter plot of the model AP index and the target density in step S24.
[0083] S33, for the density intervals with poor model performance, more training data sets are specifically constructed for the density intervals to further optimize the model.
[0084] S34, when the model is applied, the target density of the image to be processed is first estimated by manual setting or automatic estimation. Automatic estimation can be evaluated based on the average gray value and color scale curve of the image. For example, for the dense cell detection task in the embodiment of the present application, the darker the image, the lower the average gray value, and the greater the cell density.
[0085] S35, give priority to deep learning models with excellent performance within the estimated target range, and then consider the efficiency of the model.
[0086] Based on any of the above embodiments, taking the field of cytopathology and detecting each cell in a cell cluster as an example, a specific embodiment of the present invention is described.
[0087] 1) Cell cluster image data Images for cell detection were cropped from whole slide pathology images, and the image size was unified to 512×512. Figure 5 is a schematic diagram of the cell cluster image data provided by the present invention, such as Figure 5 As shown, the cells in the left image are extremely dense, while the cells in the right image are relatively sparse.
[0088] 2) Labeling results and deep learning target detection results The embodiment of the present invention adopts two deep learning models: CenterNet and Faster R-CNN. Figure 6 is a schematic diagram of the target detection results of the Faster R-CNN model provided by the present invention in cell-dense and cell-sparse scenarios, Figure 7 is the target detection result of the CenterNet model provided by the present invention in the cell-dense and cell-sparse scenes, such as Figure 6 , Figure 7 As shown, the annotation results are represented by a red rectangular box, and the deep learning target detection results are represented by a blue rectangular box.
[0089] 3) Calculation of cell density and AP index In this embodiment, the size of all images is 512×512 pixels, so the cell density is linearly related to the cell number, and the cell number can be used instead of the cell density for subsequent evaluation.
[0090] Cell-dense image: The number of labeled cells is 348, the AP50 of the CenterNet model is 0.8684, and the AP50 of the FasterR-CNN model is 0.6682.
[0091] Cell sparse image: The number of labeled cells is 68, the AP50 of the CenterNet model is 0.9002, and the AP50 of the Faster R-CNN model is 0.8775.
[0092] This example shows that in the cell sparse scene, the AP50 index of the CenterNet model and the Faster R-CNN model is not much different (0.9002 - 0.8775 = 0.0227); however, in the cell dense scene, the AP50 index of the CenterNet model and the Faster R-CNN model is very different (0.8684 - 0.6682 = 0.2002). This example verifies the necessity and effectiveness of calculating the AP index for each image separately in the small target detection scene.
[0093] 4) Calculate the correlation between AP and target density Figure 8 is a schematic diagram of the correlation between AP and target density provided by the present invention, such as Figure 8 As shown, a scatter plot of the AP50 metric versus the number of cells in the image. Left: CenterNet model. Right: Faster R-CNN model.
[0094] The Pearson correlation method was used to calculate the correlation between AP50 and cell number of the CenterNet model and the Faster R-CNN model.
[0095] CenterNet model: r = -02521, p = 0.0463, indicating that the performance of the CenterNet model decreases with the increase of cell density.
[0096] Faster R-CNN model: r = -0.4406, p = 0.0003, indicating that the performance of the Faster R-CNN model decreases rapidly with the increase of cell density.
[0097] The results of the embodiments of the present invention show that target detection in dense scenes is more difficult than target detection in sparse scenes, and the performance of the CenterNet model and the Faster R-CNN model are inversely proportional to the cell density. In the sparse target detection scenario, the performance of the CenterNet model and the Faster R-CNN model are both better, but in the dense scene, the performance of the CenterNet model is significantly better than the Faster R-CNN model.
[0098] The following is a description of an evaluation device for a dense small target detection task provided by the present invention. The evaluation device for a dense small target detection task described below and the evaluation method for a dense small target detection task described above can refer to each other.
[0099] Based on any of the above embodiments, the present invention provides an evaluation device for dense small target detection tasks, Fig. 9 is a schematic diagram of the structure of the evaluation device for dense small target detection tasks provided by the present invention, such as Fig. 9 As shown, the device comprises: An acquisition unit 910 is used to acquire an image to be detected and a true value of an annotation of the image to be detected, and determine the number of objects in the true value of the annotation; An average precision determination unit 920 is used to input the image to be detected into a target detection model, obtain a target detection result output by the target detection model, and determine an average precision based on the target detection result and the annotation true value; An evaluation result determination unit 930, configured to determine a target density based on the target quantity, and to determine an evaluation result based on the target density and the average accuracy; The target density interval determination unit 940 is used to determine a target density interval where the performance of the target detection model is poor based on the evaluation result.
[0100] The device provided by the embodiment of the present invention is aimed at monitoring dense small targets, detecting cells in a typical scene-type cell cluster image, calculating the average accuracy and target density for each image separately, and further evaluating the correlation between the average accuracy and the target density, calculating the correlation coefficient, and determining the evaluation result. The evaluation method for dense cell target detection tasks described in the present invention can effectively evaluate the model performance under various target densities, and can provide important support information for the research and development, testing, and application of the model.
[0101] Based on any of the above embodiments, the evaluation result determination unit 930 is specifically used to: A first determining unit, configured to determine a correlation coefficient and a significance level index based on the target density and the average precision; The second determining unit is used to determine the evaluation result based on the correlation coefficient and the significance level indicator.
[0102] Based on any of the foregoing embodiments, the second determining unit is specifically configured to: When the significance level index is greater than a first threshold, determining that the evaluation result is that the average accuracy has no correlation with the target density; When the significance level index is less than the first threshold and the correlation coefficient is less than the second threshold, determining that the evaluation result is that the average accuracy is negatively correlated with the target density, and as the target density increases, the average accuracy decreases; When the significance level index is less than the first threshold and the correlation coefficient is greater than the second threshold, it is determined that the evaluation result is that the average accuracy is positively correlated with the target density, and as the target density increases, the average accuracy increases.
[0103] Based on any of the above embodiments, the evaluation result determination unit 930 is specifically used to: In the case where the image pixel areas of the images to be detected are different, determining the image pixel areas of the images to be detected, and determining the target density based on the image pixel areas, the linear scaling factor and the number of targets; In a case where the image pixel areas of the images to be detected are the same, the target density is determined based on the target quantity and the linear scaling factor.
[0104] Based on any of the above embodiments, the average precision determining unit 920 is specifically configured to: Determine the degree of overlap between the predicted box in the target detection result and the real box in the labeled true value; Based on the degree of overlap, an average precision is determined.
[0105] Based on any of the above embodiments, it further includes an optimization unit, wherein the optimization unit is specifically used to: A training data set of the target density interval is constructed, and the target detection model is optimized based on the training data set.
[0106] Fig.10 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig.10As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute an evaluation method for a dense small target detection task, the method comprising: obtaining an image to be detected and a true value of the annotation of the image to be detected, and determining the number of targets in the true value of the annotation; inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model, and determining an average precision based on the target detection result and the true value of the annotation; determining a target density based on the number of targets, and determining an evaluation result based on the target density and the average precision; and determining a target density interval with poor performance of the target detection model based on the evaluation result.
[0107] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0108] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the evaluation method for dense small target detection tasks provided by the above methods, which method includes: obtaining an image to be detected and a labeled true value of the image to be detected, and determining the number of targets in the labeled true value; inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model, and determining an average accuracy based on the target detection result and the labeled true value; determining a target density based on the number of targets, and determining an evaluation result based on the target density and the average accuracy; and determining a target density interval with poor performance of the target detection model based on the evaluation result.
[0109] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute an evaluation method for dense small target detection tasks provided by the above-mentioned methods, the method comprising: obtaining an image to be detected and a labeled true value of the image to be detected, and determining the number of targets in the labeled true value; inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model, and determining an average precision based on the target detection result and the labeled true value; determining a target density based on the number of targets, and determining an evaluation result based on the target density and the average precision; and determining a target density interval with poor performance of the target detection model based on the evaluation result.
[0110] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An evaluation method for dense small target detection tasks, characterized in that: include: Acquire an image to be detected and a true value of an annotation of the image to be detected, and determine the number of objects in the true value of the annotation; Inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model, and determining an average precision based on the target detection result and the labeled true value; Determining a target density based on the target quantity, and determining an evaluation result based on the target density and the average accuracy; Based on the evaluation result, a target density interval in which the target detection model has poor performance is determined.
2. The evaluation method for dense small target detection task according to claim 1, characterized in that: The step of determining an evaluation result based on the target density and the average accuracy includes: Determining a correlation coefficient and a significance level indicator based on the target density and the average precision; The evaluation result is determined based on the correlation coefficient and the significance level indicator.
3. The evaluation method for dense small target detection task according to claim 2, characterized in that: The step of determining the evaluation result based on the correlation coefficient and the significance level indicator includes: When the significance level index is greater than a first threshold, determining that the evaluation result is that the average accuracy has no correlation with the target density; When the significance level index is less than the first threshold and the correlation coefficient is less than the second threshold, determining that the evaluation result is that the average accuracy is negatively correlated with the target density, and as the target density increases, the average accuracy decreases; When the significance level index is less than the first threshold and the correlation coefficient is greater than the second threshold, it is determined that the evaluation result is that the average accuracy is positively correlated with the target density, and as the target density increases, the average accuracy increases.
4. The evaluation method for dense small target detection task according to any one of claims 1 to 3, characterized in that: The step of determining the target density based on the target quantity includes: In the case where the image pixel areas of the images to be detected are different, determining the image pixel areas of the images to be detected, and determining the target density based on the image pixel areas, the linear scaling factor and the number of targets; In a case where the image pixel areas of the images to be detected are the same, the target density is determined based on the target quantity and the linear scaling factor.
5. The evaluation method for dense small target detection task according to any one of claims 1 to 3, characterized in that: The determining the average precision based on the target detection result and the labeled true value includes: Determine the degree of overlap between the predicted box in the target detection result and the real box in the labeled true value; Based on the degree of overlap, an average precision is determined.
6. The evaluation method for dense small target detection task according to any one of claims 1 to 3, characterized in that: The step of determining a target density interval with poor performance of the target detection model based on the evaluation result further includes: A training data set of the target density interval is constructed, and the target detection model is optimized based on the training data set.
7. An evaluation device for dense small target detection tasks, characterized in that: include: An acquisition unit, used to acquire an image to be detected and a true value of an annotation of the image to be detected, and determine the number of targets in the true value of the annotation; An average precision determining unit is used to input the image to be detected into a target detection model, obtain a target detection result output by the target detection model, and determine an average precision based on the target detection result and the annotation true value; An evaluation result determination unit, configured to determine a target density based on the target quantity, and to determine an evaluation result based on the target density and the average accuracy; The target density interval determination unit is used to determine the target density interval where the performance of the target detection model is poor based on the evaluation result.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the evaluation method for dense small target detection tasks according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the evaluation method for dense small target detection tasks as described in any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the evaluation method for dense small target detection tasks as described in any one of claims 1 to 6 is implemented.