Target counting method and device, equipment and medium
By combining semantic segmentation and deep visual feature matching with a correction model, the problem of missed and false detections of small-sized targets in target counting is solved, and high-precision counting is achieved in complex scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-13
AI Technical Summary
Existing technologies struggle to accurately count small targets in scenarios with dense target distribution, complex pose changes, uneven lighting, and severe occlusion, leading to missed or false detections.
The target region is extracted by semantic segmentation, high-dimensional features are extracted by deep vision model and matched with a preset template library, and the number is estimated and adaptively corrected by calibration model, so as to achieve accurate determination of target category and number calculation.
It significantly improves the stability and accuracy of counting in complex scenarios, achieves generalized adaptation to different environmental conditions, and ensures the accuracy and reliability of counting results.
Smart Images
Figure CN121661624A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of target counting, and more particularly to a target counting method, apparatus, device, and medium. Background Technology
[0002] In scenarios such as warehousing and logistics, industrial production, and retail management, the rapid and accurate counting of targets in images is of great value. However, these scenarios generally present challenges such as dense target distribution, complex pose variations (e.g., tilting, stacking), uneven lighting, and severe occlusion, making it difficult for traditional counting methods to meet the requirements of accuracy and robustness. Furthermore, practical applications involve diverse target categories (e.g., packaging boxes of different sizes, industrial parts of different shapes), requiring counting systems to have multi-category adaptability. Therefore, there is an urgent need for a universal target counting method that can balance versatility, accuracy, and engineering feasibility.
[0003] In existing technologies, common target counting methods involve processing the scene to be identified frame by frame using a target detection model, and then identifying and locating each target instance individually. Specifically, features are first extracted from the input image or video frame, then multiple candidate regions are generated using a pre-trained detection network, and finally the category and bounding box of each target are determined based on the classification and regression results.
[0004] However, existing solutions struggle to handle small targets, leading to missed or false detections. Summary of the Invention
[0005] This application provides a target counting method, apparatus, device, and medium to solve the problem in the prior art that it is difficult to handle small-sized targets, resulting in missed detections or false detections.
[0006] In a first aspect, embodiments of this application provide a target counting method, including:
[0007] The target image to be counted is acquired, and semantic segmentation processing is performed on the target image to obtain multiple target regions;
[0008] Feature extraction is performed on each target region to obtain multiple target feature vectors;
[0009] The similarity of each target feature vector with a preset category feature template library is calculated to determine the target category corresponding to each target feature vector and the standard pixel area corresponding to the target category. The category feature template library stores the relationship between feature vectors, categories and standard pixel areas.
[0010] For each target category, the initial number of the target category is calculated based on all the target regions corresponding to the target category and the standard pixel area;
[0011] Based on the calibration model, the final number of target categories is calculated according to the target region and the occlusion rate. The calibration model is trained in advance based on historical annotation data, which includes the annotation categories, number of annotations, annotation pixel area, annotation connected component area, and annotation occlusion rate of multiple historical annotation regions.
[0012] In one possible implementation, the process of constructing the category feature template library includes:
[0013] For each sample category, a preset number of sample images corresponding to the category are obtained, and the sample images include multiple typical poses of the sample category;
[0014] Perform preprocessing operations on each of the sample images to obtain a sample standardized image. The preprocessing operations include at least image denoising and size normalization.
[0015] The target region of each standardized image is determined by manually annotating bounding boxes, and the pixel area of each target region is calculated.
[0016] After removing outliers in the pixel area based on the Grubbs criterion, the statistical mean of the other pixel areas is determined as the standard pixel area of the sample category.
[0017] A preset feature extraction network is used to extract features from each of the target regions of the sample to obtain a sample feature vector. The sample category, the sample feature vector, and the standard pixel area of the sample category are then associated to construct a category feature template library.
[0018] In one possible implementation, calculating the initial number of target categories for each target category based on all target regions corresponding to the target category and the standard pixel area includes:
[0019] Calculate the pixel area of the target region based on the target region;
[0020] The total pixel area of the target category is obtained by summing up the areas of all pixels corresponding to the target category.
[0021] Calculate the ratio of the total pixel area to the standard pixel area to obtain the initial number of the target category.
[0022] In one possible implementation, calculating the final number of target categories based on the correction model, according to the target region and the occlusion rate, includes:
[0023] Based on the pixel area and connected component area of the target region, the occlusion rate of the target region is calculated, where the connected component area is the area of the largest consecutive set of pixels containing the target region.
[0024] Determine the correction feature set based on the occlusion rate;
[0025] The correction feature set is input into the correction model to obtain the correction factor corresponding to the target category;
[0026] The final number of target categories is calculated based on the correction factor and the initial number of target categories.
[0027] In one possible implementation, determining the correction feature set based on the occlusion rate includes:
[0028] If the occlusion rate does not exceed the preset occlusion rate, then the first feature set is determined as the correction feature set. The first feature set includes camera tilt angle, target density, image brightness, and the percentage of image edge detection pixels.
[0029] If the occlusion rate exceeds the preset occlusion rate, then a second feature set is determined as the correction feature set. The second feature set includes the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the image grayscale variance.
[0030] In one possible implementation, the training process of the correction model includes:
[0031] Obtain historical labeled data;
[0032] Based on the labeled occlusion rate, the historical labeled data is divided into a first training set and a second training set;
[0033] For the first training set, the camera tilt angle, target density, image brightness, and the percentage of pixels detected at image edges are extracted to construct the first training input features;
[0034] For the second training set, the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the gray-level variance of the image are extracted to construct the second training input features;
[0035] Based on the number of annotations in the historical annotation data and the initial number calculated from the area of the annotated pixels and the area of the standard pixels, a historical correction factor is determined.
[0036] The first and second training input features are mapped to their corresponding historical correction factors, and the correction model is trained using supervised learning so that the correction model can output the correction factor of the target category based on the input correction feature set.
[0037] In one possible implementation, the step of calculating the similarity between each target feature vector and a preset category feature template library to determine the target category corresponding to each target feature vector includes:
[0038] The target feature vector is compared with the feature vectors of each sample in the preset category feature template library by cosine similarity calculation to obtain multiple similarity values;
[0039] Select the sample category corresponding to the highest similarity value from the similarity values as the candidate category;
[0040] When the maximum similarity is greater than or equal to a preset similarity threshold, the candidate category is determined as the target category corresponding to the target feature vector;
[0041] When the maximum similarity is less than the preset similarity threshold, the target feature vector is marked as an unknown category and reported.
[0042] Secondly, embodiments of this application provide a target counting device, comprising:
[0043] The first processing module is used to acquire the target image to be counted and perform semantic segmentation processing on the target image to obtain multiple target regions;
[0044] The extraction module is used to extract features from each of the target regions to obtain multiple target feature vectors;
[0045] The determination module is used to perform similarity calculation between each target feature vector and a preset category feature template library to determine the target category corresponding to each target feature vector and the standard pixel area corresponding to the target category. The category feature template library stores the association relationship between feature vectors, categories and standard pixel areas.
[0046] The calculation module is used to calculate the initial number of target categories for each target category based on the area of all target regions corresponding to the target category and the area of the standard pixels.
[0047] The second processing module is used to calculate the final number of target categories based on the calibration model, according to the target region and the occlusion rate. The calibration model is trained in advance based on historical annotation data, which includes the annotation categories, number of annotations, annotation pixel area, annotation connected component area, and annotation occlusion rate of multiple historical annotation regions.
[0048] In one possible implementation, the target counting device further includes a construction module for:
[0049] For each sample category, a preset number of sample images corresponding to the category are obtained, and the sample images include multiple typical poses of the sample category;
[0050] Perform preprocessing operations on each of the sample images to obtain a sample standardized image. The preprocessing operations include at least image denoising and size normalization.
[0051] The target region of each standardized image is determined by manually annotating bounding boxes, and the pixel area of each target region is calculated.
[0052] After removing outliers in the pixel area based on the Grubbs criterion, the statistical mean of the other pixel areas is determined as the standard pixel area of the sample category.
[0053] A preset feature extraction network is used to extract features from each of the target regions of the sample to obtain a sample feature vector. The sample category, the sample feature vector, and the standard pixel area of the sample category are then associated to construct a category feature template library.
[0054] In one possible implementation, the computing module is specifically used for:
[0055] Calculate the pixel area of the target region based on the target region;
[0056] The total pixel area of the target category is obtained by summing up the areas of all pixels corresponding to the target category.
[0057] Calculate the ratio of the total pixel area to the standard pixel area to obtain the initial number of the target category.
[0058] In one possible implementation, the second processing module is specifically used for:
[0059] Based on the pixel area and connected component area of the target region, the occlusion rate of the target region is calculated, where the connected component area is the area of the largest consecutive set of pixels containing the target region.
[0060] Determine the correction feature set based on the occlusion rate;
[0061] The correction feature set is input into the correction model to obtain the correction factor corresponding to the target category;
[0062] The final number of target categories is calculated based on the correction factor and the initial number of target categories.
[0063] In one possible implementation, the second processing module is specifically used for:
[0064] If the occlusion rate does not exceed the preset occlusion rate, then the first feature set is determined as the correction feature set. The first feature set includes camera tilt angle, target density, image brightness, and the percentage of image edge detection pixels.
[0065] If the occlusion rate exceeds the preset occlusion rate, then a second feature set is determined as the correction feature set. The second feature set includes the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the image grayscale variance.
[0066] In one possible implementation, the second processing module is specifically used for:
[0067] Obtain historical labeled data;
[0068] Based on the labeled occlusion rate, the historical labeled data is divided into a first training set and a second training set;
[0069] For the first training set, the camera tilt angle, target density, image brightness, and the percentage of pixels detected at image edges are extracted to construct the first training input features;
[0070] For the second training set, the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the gray-level variance of the image are extracted to construct the second training input features;
[0071] Based on the number of annotations in the historical annotation data and the initial number calculated from the area of the annotated pixels and the area of the standard pixels, a historical correction factor is determined.
[0072] The first and second training input features are mapped to their corresponding historical correction factors, and the correction model is trained using supervised learning so that the correction model can output the correction factor of the target category based on the input correction feature set.
[0073] In one possible implementation, the determining module is specifically used for:
[0074] The target feature vector is compared with the feature vectors of each sample in the preset category feature template library by cosine similarity calculation to obtain multiple similarity values;
[0075] Select the sample category corresponding to the highest similarity value from the similarity values as the candidate category;
[0076] When the maximum similarity is greater than or equal to a preset similarity threshold, the candidate category is determined as the target category corresponding to the target feature vector;
[0077] When the maximum similarity is less than the preset similarity threshold, the target feature vector is marked as an unknown category and reported.
[0078] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;
[0079] The memory stores computer-executed instructions;
[0080] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.
[0081] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.
[0082] The target counting method, apparatus, device, and medium provided in this application first perform semantic segmentation on the image to be counted, thereby accurately extracting multiple candidate target regions. Then, feature extraction is performed on each candidate region to ensure stable representation of targets of the same category under different poses and lighting conditions in the feature space, providing a reliable foundation for subsequent recognition. Next, the similarity of each target feature vector with a category feature template library is calculated. Cosine similarity is used to measure the similarity between the target region and registered category samples, achieving accurate determination of the target category for each region and simultaneously obtaining the standard pixel area related to the category, laying the basis for subsequent counting. Furthermore, for multiple regions of the same category, the initial number is calculated by accumulating the region areas and comparing them with the standard pixel area, thus obtaining a high-accuracy count estimate even without correction. Finally, to further eliminate area deviations caused by factors such as occlusion, dense stacking, and shooting angle, a pre-trained correction model is used to adaptively correct the initial number, thereby obtaining the final number of target categories. This significantly improves the counting stability and accuracy in complex scenes, achieves generalization adaptation to different environmental conditions, and ensures that the final counting result is more accurate and reliable. Attached Figure Description
[0083] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0084] Figure 1 A flowchart illustrating the target counting method provided in the embodiments of this application. Figure 1 ;
[0085] Figure 2 A flowchart illustrating the target counting method provided in the embodiments of this application. Figure 2 ;
[0086] Figure 3This is a schematic diagram of the structure of the target counting device provided in the embodiments of this application;
[0087] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0088] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0089] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0090] In scenarios such as warehousing and logistics, industrial production, and retail management, the rapid and accurate counting of targets in images is of paramount importance. For example, in warehousing, it is necessary to count the number of goods on shelves in real time for inventory management; in production line inspection, it is necessary to efficiently count products on conveyor belts to ensure the accuracy of the production process; in retail scenarios, supermarket shelf inventory counting requires automated counting to reduce labor costs. However, these scenarios generally present problems such as dense target distribution, complex pose changes (e.g., tilting, stacking), uneven lighting, and severe occlusion, making it difficult for traditional counting methods to meet the requirements of accuracy and robustness. In addition, practical applications involve diverse target categories (e.g., packaging boxes of different sizes, industrial parts of different shapes), requiring the counting system to have multi-category adaptability. Existing technologies often struggle to be implemented when dealing with these complex scenarios due to insufficient algorithm adaptability, poor model generalization ability, or high computational resource consumption. There is an urgent need for a general target counting method that can balance versatility, accuracy, and engineering feasibility.
[0091] In existing technologies, common target counting methods involve processing the scene to be identified frame by frame using a target detection model, and then identifying and locating each target instance individually. Specifically, features are first extracted from the input image or video frame, then multiple candidate regions are generated using a pre-trained detection network, and finally the category and bounding box of each target are determined based on the classification and regression results.
[0092] However, existing solutions are only applicable to scenarios where targets are sparsely distributed. They are prone to missed detections or false detections when targets are densely packed and heavily occluded, and they are difficult to handle small targets, thus limiting their application scenarios.
[0093] Based on this, this application proposes a target counting method to address the problem that existing technologies generally rely on target detection bounding boxes or fixed rules for identification and statistics, leading to large counting bias and weak generalization ability. The inventors recognized that if the target region in the image can be accurately segmented first, then the stability of deep features in high-dimensional space can be used to complete category identification, and further, the statistical regularity of single target area can be combined for quantity estimation. Finally, by introducing a scene adaptive correction model to correct the area bias, a scheme that can still achieve high-precision counting in complex environments can be constructed. Specifically: First, a semantic segmentation model is used to extract pixel-level regions from the image, completely separating the target region from the background; then, a deep vision model is used to extract high-dimensional features from each region and perform similarity matching with a pre-set template library to determine the target category and its corresponding standard pixel area; next, a preliminary quantity estimate is established based on the area of each category region and the standard pixel area; finally, a pre-trained correction model is used to generate correction factors based on occlusion rate, region morphology, and scene features to adaptively calibrate the initial quantity. Through this series of steps, this application can maintain high robustness and high accuracy under complex conditions such as occlusion, stacking, and dense scenes, achieving technical effects that are significantly better than traditional detection-based counting methods.
[0094] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0095] Figure 1 A flowchart illustrating the target counting method provided in the embodiments of this application. Figure 1 ;like Figure 1 As shown, the method includes:
[0096] S101. Obtain the target image to be counted, and perform semantic segmentation on the target image to obtain multiple target regions.
[0097] It should be understood that traditional object counting methods based on detection boxes are prone to problems such as missed detections, box offsets, and region overlap in scenes with dense targets, occlusion, or deformation, leading to unstable counting results. Semantic segmentation can perform pixel-level region recognition on images, completely separating foreground targets from the background, and preserving independent region structures even when targets are in contact or partially overlapping. Therefore, it is more suitable as a preprocessing step for counting. Specifically, after acquiring the image of the targets to be counted, a pre-trained semantic segmentation model is used to process the image at the pixel level. The model can be based on DINOv3 feature encoding and fine-tuned with a large number of multi-scene samples. The segmentation model outputs multiple target foreground regions, each containing attributes such as pixel position, region shape, and pixel area, which are used for subsequent feature extraction and category determination. For example, in a warehouse image containing multiple water bottles and cardboard boxes, the segmentation model can separate each item independently. Even if the water bottles are partially in contact, they can be segmented into multiple independent connected components, laying the foundation for accurate subsequent statistics.
[0098] Understandably, by replacing detection box technology with semantic segmentation, the boundaries of the target area become more accurate, significantly reducing the error in area segmentation caused by occlusion, dense stacking, or changes in lighting, thereby improving the accuracy of subsequent recognition and counting. This is particularly suitable for complex scenarios such as warehousing and e-commerce picking lines.
[0099] S102. Extract features from each target region to obtain multiple target feature vectors.
[0100] It should be understood that because the visual features of the same type of target vary greatly under different angles, lighting conditions, and occlusions, direct classification based on simple features such as area and color is prone to misclassification. Deep vision models can express the semantic features of targets through a high-dimensional embedding space, maintaining a stable distribution of the same type of target under different forms, thereby improving the robustness of category determination.
[0101] For each target region obtained by S101, the region image is cropped and input into a feature extraction network (such as the DINOv3 model based on ViT), and the output is a target feature vector with dimensions of 768–1024.
[0102] Understandably, the target feature vector obtained through feature extraction can effectively distinguish targets with similar shapes but different categories, while maintaining high robustness to rotation, lighting, and pose changes, making category recognition in natural scenes more stable and reliable.
[0103] S103. Calculate the similarity between each target feature vector and the preset category feature template library to determine the target category corresponding to each target feature vector and the standard pixel area corresponding to the target category.
[0104] The category feature template library stores the relationship between feature vectors, categories, and standard pixel areas.
[0105] In one feasible approach, the cosine similarity of the target feature vector with each sample feature vector in a pre-defined category feature template library is first calculated to obtain multiple similarity values. Then, the sample category corresponding to the maximum similarity value is selected as a candidate category. When the maximum similarity is greater than or equal to a pre-defined similarity threshold, the candidate category is determined as the target category corresponding to the target feature vector. When the maximum similarity is less than the pre-defined similarity threshold, the target feature vector is marked as an unknown category and reported.
[0106] It should be understood that by calculating the similarity between the feature vector of the target region and the category feature template library, high-precision category determination can be achieved without retraining the classification model. Using cosine similarity to measure the consistency of vector direction can effectively measure the semantic proximity between targets. Simultaneously, each category has already calculated its standard pixel area in the category feature template library, establishing a natural association between recognition and counting.
[0107] Specifically, each target feature vector is sequentially compared with all sample feature vectors stored in the template library using cosine similarity calculation, resulting in multiple similarity values. The category corresponding to the highest similarity is selected as the candidate category. When the highest similarity is greater than a preset threshold (e.g., 0.85), the candidate category is determined as the final category of the target, and its corresponding standard pixel area is read from the template library. If the highest similarity is lower than the threshold, the region is marked as "unknown category" and reported for manual processing. For example, in a warehouse image, after calculating the similarity of a certain region, its similarity with the "water bottle template" reaches 0.92, while it is lower than 0.7 with other categories. Therefore, the region is determined to be a water bottle, and a standard pixel area of 8500px is automatically associated. The specific formula for calculating cosine similarity is as follows:
[0108]
[0109] in, For the target feature vector, This refers to the feature vectors of all samples stored in the template library.
[0110] Understandably, this step achieves high-precision category recognition based on feature similarity and establishes a direct mapping between categories and standard areas, providing a reliable scale benchmark for subsequent counting; at the same time, the threshold mechanism can effectively avoid misidentification and improve system stability.
[0111] S104. For each target category, calculate the initial number of target categories based on the area of all target regions and standard pixels corresponding to the target category.
[0112] In one possible approach, the pixel area of the target region is first calculated based on the target region; then, the pixel areas corresponding to all target categories are summed to obtain the total pixel area of the target category; finally, the ratio of the total pixel area to the standard pixel area is calculated to obtain the initial number of target categories.
[0113] It should be understood that for targets of the same category, under close-range acquisition and uniform camera parameters, the pixel area of a single target exhibits statistical stability. Therefore, by summing the areas of all regions and dividing by the standard pixel area, a relatively accurate preliminary quantity estimate can be obtained; the area statistical method is particularly suitable for dense scenes and partially occluded conditions.
[0114] In the specific implementation process, for each identified category, the pixel area of all target regions is calculated and summed to obtain the total pixel area. Then, the total pixel area is divided by the standard pixel area of that category to obtain the initial number. If the initial number is not an integer, one decimal place can be retained as the basis for subsequent corrections. The specific calculation formula is as follows:
[0115]
[0116]
[0117] in, Total pixel area The pixel area of a single target region; For the initial quantity, The standard pixel area for the target category.
[0118] For example, if the area of the three water bottle regions is 8600, 8100, and 6000 pixels respectively, the total area is 22700 pixels; calculated using a standard area of 8500 pixels: This indicates that there is partial occlusion, and further adjustments are needed through the calibration model.
[0119] Understandably, this method achieves fast and relatively accurate counting without the need to detect each target individually. For occluded parts (such as stacked objects side by side), the change in quantity can still be reflected by the area ratio, which improves the usability and generalization ability in complex scenarios.
[0120] S105. Based on the calibration model, calculate the final number of target categories according to the target area and occlusion rate.
[0121] The calibration model is trained in advance based on historical annotation data, which includes the annotation category, number of annotations, annotation pixel area, area of connected components, and occlusion rate of multiple historical annotation regions.
[0122] It should be understood that, due to factors such as changes in camera angle, uneven lighting, target occlusion, and region fusion in real-world scenes, simply counting based on area is prone to bias. By introducing a correction model trained on historical labeled data, a correction factor can be predicted based on the target region characteristics and the degree of occlusion, thereby adaptively compensating for the initial results.
[0123] It should be noted that the calibration model can be an XGBoost regression model or a lightweight neural network. When the calibration model is an XGBoost regression model, the parameters are set to a tree depth of 6, a learning rate of 0.1, and 100 iterations to reduce training time and make it suitable for edge devices. When the calibration model is a lightweight neural network, the input dimension is equal to the feature dimension, the hidden layer is a 2-layer structure, the ReLU activation function is used, and the output is a linear node, which is suitable for scenarios that need to capture non-linear relationships.
[0124] In actual counting, the region features and occlusion rate of the current image are first extracted, input into the correction model to obtain the correction factor, and finally the final number of target categories is calculated.
[0125] Understandably, by automatically compensating for area bias through the model, the system can significantly improve counting accuracy under complex conditions such as strong occlusion, dense stacking, and varying shooting angles, achieving adaptive generalization capabilities across multiple scenarios. Even if the area estimate is too low or too high, a stable output can be obtained through a correction factor.
[0126] It should be noted that the calculation process for the final quantity of specific target categories is detailed in this application. Figure 2 The embodiments are described in detail, and the embodiments of this application will not be repeated here.
[0127] The target counting method provided in this application first performs semantic segmentation on the image to be counted, thereby accurately extracting multiple candidate target regions. Then, feature extraction is performed on each candidate region to ensure stable representation of targets of the same category under different poses and lighting conditions in the feature space, providing a reliable foundation for subsequent recognition. Next, the similarity of each target feature vector with a category feature template library is calculated. Cosine similarity is used to measure the similarity between the target region and registered category samples, achieving accurate determination of the target category for each region and simultaneously obtaining the standard pixel area related to the category, laying the basis for subsequent counting. Furthermore, for multiple regions of the same category, the initial number is calculated by accumulating the region areas and comparing them with the standard pixel area, thus obtaining a high-accuracy count estimate even without correction. Finally, to further eliminate area deviations caused by factors such as occlusion, dense stacking, and shooting angle, a pre-trained correction model is used to adaptively correct the initial number, thereby obtaining the final number of target categories. This significantly improves the counting stability and accuracy in complex scenes, achieves generalization adaptation to different environmental conditions, and ensures that the final counting result is more accurate and reliable.
[0128] In one feasible approach, the construction process of the category feature template library is as follows:
[0129] First, for each sample category, a predetermined number of sample images corresponding to the category are obtained. Then, preprocessing is performed on each sample image to obtain a standardized sample image. Next, the target region of each standardized sample image is determined by manually annotated bounding boxes, and the pixel area of each target region is calculated. Then, outliers in pixel area are removed based on the Grubbs criterion, and the statistical mean of the other pixel areas is determined as the standard pixel area of the sample category. Finally, a preset feature extraction network is used to extract features from each target region of the sample to obtain a sample feature vector. The sample category, the sample feature vector, and the standard pixel area of the sample category are then correlated to construct a category feature template library.
[0130] The sample images include multiple typical poses of the sample categories; the preprocessing operations include at least image denoising and size normalization.
[0131] It should be understood that, in order to accurately characterize the shape, pose, and visual variability of each type of target, a sufficient number of reference sample images need to be collected for each category to ensure the representativeness and robustness of the feature template library. Furthermore, according to the principles of statistical learning, the higher the sample coverage, the more accurately the extracted features will reflect the true distribution of that type of target in the real-world scene.
[0132] Therefore, for each sample category to be registered, at least a preset number of sample images (e.g., ≥15 images) need to be obtained. These sample images should cover the typical poses and common appearance differences of the target in that category, such as the upright, tilted, and inverted poses of a water bottle; and the front, side, and exposed corners of a cardboard box when stacked. Then, image denoising processing is performed on each sample image, i.e., Gaussian filtering is used. Imaging noise is removed while preserving edge information as much as possible; size normalization is also performed, adjusting the image scale to a uniform 512×512 pixels to reduce feature shift caused by resolution differences. It should also be noted that if the image has uneven illumination, contrast-limited adaptive histogram equalization (CLAHE) can be used to improve the illumination unevenness and enhance local contrast. This processing method significantly improves the stability of subsequent feature extraction, making samples from different acquisition conditions more consistent in the feature space, reducing noise interference, and improving the robustness of the template library.
[0133] Furthermore, the standard pixel area is a key parameter for pixel area ratio counting, and its accuracy directly affects the precision of the final quantity estimation. Therefore, it is necessary to accurately label the sample target regions and remove outliers. Specifically, for each standardized sample image, the precise range of the target is determined by manually annotating bounding boxes; then, the pixel area of each sample target region is calculated. To avoid statistical bias caused by annotation errors, the Grubbs criterion (significance level) must be used. Outlier detection is performed on the area data, removing areas that significantly deviate from the mean. The arithmetic mean of the remaining areas is then used as the standard pixel area for that category. This outlier removal and mean calculation method results in a more stable standard pixel area that accurately reflects the typical size of this type of target in the image, laying a solid foundation for subsequent quantitative estimation based on pixel area ratios.
[0134] To enable high-precision comparison using feature vectors in subsequent identification, it is necessary to extract discriminative and stable deep visual features from each sample target region. Employing advanced transformer-like visual models can significantly improve feature representation capabilities. For each sample target region, a pre-defined deep feature extraction network (such as DINOv3, ViT-B / 16, etc.) is used for feature extraction. A 1024-dimensional feature vector is preferably extracted to fully represent the texture, shape, edges, and overall semantic information of the target type.
[0135] Finally, in order to achieve rapid retrieval of target features, categories, and standard pixel areas during the recognition process, it is necessary to establish a clear relationship between sample categories, sample feature vectors, and standard pixel areas to form a unified template library structure.
[0136] Figure 2 A flowchart illustrating the target counting method provided in the embodiments of this application. Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the examples, the process of calculating the final number of target categories is described in detail, and the method includes:
[0137] S201. Calculate the occlusion rate of the target region based on the pixel area and connected component area of the target region.
[0138] The area of the connected region is the area of the largest consecutive set of pixels that contains the target region.
[0139] It should be understood that in densely stacked, multi-target occlusion scenarios, the actual visible area of a single target region will be significantly reduced. Directly using pixel area for quantity estimation will lead to serious errors. Therefore, it is necessary to measure the degree of occlusion by calculating the ratio of the target region's pixel area to its connected component area, thus providing a basis for subsequent quantity correction. Specifically, firstly, the pixel area of each target region is determined; then, based on image connected component analysis methods (such as 8-neighborhood connectivity analysis), the largest set of consecutive pixels containing that target region is extracted, obtaining its connected component area. The ratio of the target region area to its corresponding connected component area is used as the occlusion rate of that target region. The lower the occlusion rate, the more severely the target is occluded. For example, in a stacked water bottle scene, if a target region has a pixel area of 1000 pixels, and its connected component area reaches 3000 pixels, the occlusion rate is 0.33, indicating that only one-third of the target is visible.
[0140] Understandably, quantifying the visibility of a target by occlusion rate enables the system to accurately identify whether the target is in a severely occluded state and determine the correction feature set to be used subsequently, thereby improving the adaptability and robustness of the overall counting framework to occluded scenarios.
[0141] S202. Determine the correction feature set based on the occlusion rate.
[0142] In one possible approach, if the occlusion rate does not exceed a preset occlusion rate, a first feature set is determined as a correction feature set. The first feature set includes the camera tilt angle, target density, image brightness, and the percentage of pixels used for image edge detection. If the occlusion rate exceeds the preset occlusion rate, a second feature set is determined as a correction feature set. The second feature set includes the number of connected components, the aspect ratio, perimeter, and area of a single connected component, and the image grayscale variance.
[0143] It should be understood that the factors affecting the initial counting accuracy differ depending on the degree of occlusion: slightly occluded images are more susceptible to the effects of shooting angle, lighting, and scene complexity, while severely occluded scenes rely more on the geometry of connected components and grayscale texture to determine the stacking situation. Therefore, it is necessary to adaptively select different feature sets based on the occlusion rate to ensure the effectiveness of the correction model input. In specific implementation, when the occlusion rate does not exceed a preset threshold (e.g., 0.6), the first feature set is used as the correction feature set, including lightly occluded sensitive parameters such as camera tilt angle, target density, image brightness, and the percentage of pixels used for image edge detection; when the occlusion rate exceeds this threshold, a second feature set is determined, including the number of connected components, the aspect ratio, perimeter, area, and grayscale variance of a single connected component, to better characterize the geometric and textural features of connected components under occlusion. For example, in a scene of densely stacked cardboard boxes, the number of connected components and grayscale variance are often significantly higher than in lightly occluded scenes, making them more suitable as model input.
[0144] Understandably, by selecting hierarchical feature sets, the calibration model can obtain the most discriminative input information under different occlusion conditions, avoiding feature redundancy or feature loss, and improving the prediction accuracy of the correction factor.
[0145] S203. Input the correction feature set into the correction model to obtain the correction factor corresponding to the target category.
[0146] In one feasible approach, the training process of the calibration model is as follows: First, historical labeled data is acquired; then, based on the labeled occlusion rate, the historical labeled data is divided into a first training set and a second training set; next, for the first training set, camera tilt angle, target density, image brightness, and the percentage of pixels used for image edge detection are extracted to construct the first training input features; for the second training set, the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the image gray-level variance are extracted to construct the second training input features; further, based on the number of labels in the historical labeled data and the initial number calculated from the labeled pixel area and the standard pixel area, a historical correction factor is determined; finally, the first and second training input features are mapped to their corresponding historical correction factors, and the calibration model is trained using supervised learning, enabling the calibration model to output the correction factor for the target category based on the input calibration feature set.
[0147] It should be understood that the initial number is calculated based on the ratio of the total pixel area to the standard pixel area, which is prone to deviation under complex conditions such as occlusion, stacking, and uneven lighting. By establishing a correction model, it can learn the mapping relationship between the feature set and the correction factor, thereby automatically compensating for the deviation and making the final number closer to the true labeled number. In the specific implementation process, historical labeled data is first collected, including information such as labeled category, labeled number, labeled pixel area, connected component area, and labeled occlusion rate. Based on the occlusion rate, the historical data is divided into a first training set (light occlusion) and a second training set (heavy occlusion). For the first training set, camera tilt angle, target density, image brightness, and edge pixel ratio are extracted to construct the training input; for the second training set, the number of connected components, aspect ratio, perimeter, area, and gray-level variance are extracted as training input. Then, the historical correction factor is calculated based on the labeled number and the initial number obtained by the pixel area method. Finally, supervised learning methods (such as random forest, lightweight MLP, or XGBoost) are used to fit and train the two types of training features to their correction factors, enabling the calibration model to output the corresponding correction factor based on the input calibration feature set. For example, in a historical sample, the initial estimated number is 10, while the manually labeled number is 12, then the corresponding correction factor is 1.2.
[0148] It should be understood that the calibration model can learn the distribution pattern of the initial quantity error under different scene conditions, realize automatic compensation, make the final quantity closer to the true value, and significantly reduce the counting deviation caused by occlusion and stacking.
[0149] S204. Calculate the final number of target categories based on the correction factor and the initial number of target categories.
[0150] It should be understood that the initial number is a theoretical estimate based on pixel area, while the correction factor is used to compensate for deviations caused by factors such as occlusion rate, shooting angle, and density variations. Combining the two allows for a composite estimate that simultaneously considers pixel proportion patterns and scene influences, significantly improving the accuracy of the final result. Therefore, by multiplying the correction factor output by the calibration model by the initial number, the final number of target categories is obtained. For example, if the initial number of a certain target category is estimated to be 20, and the calibration model outputs a correction factor of 0.85, the final number will be 17, which is closer to the actual labeled number.
[0151] Understandably, the final result combines the advantages of the area method and machine learning correction method, enabling the system to maintain high accuracy under different shooting conditions, different target densities, and different degrees of occlusion. It is particularly suitable for actual industrial scenarios with dense stacking, partial occlusion, and significant changes in viewing angle.
[0152] Figure 3 This is a schematic diagram of the structure of the target counting device provided in the embodiments of this application; as shown below. Figure 3 As shown, the device includes:
[0153] The first processing module 301 is used to acquire the target image to be counted and perform semantic segmentation processing on the target image to obtain multiple target regions;
[0154] Extraction module 302 is used to extract features from each target region to obtain multiple target feature vectors;
[0155] The determination module 303 is used to calculate the similarity between each target feature vector and the preset category feature template library, determine the target category corresponding to each target feature vector, and the standard pixel area corresponding to the target category. The category feature template library stores the relationship between feature vectors, categories and standard pixel areas.
[0156] The calculation module 304 is used to calculate the initial number of target categories for each target category based on the area of all target regions and standard pixels corresponding to the target category.
[0157] The second processing module 305 is used to calculate the final number of target categories based on the calibration model, the target region, and the occlusion rate. The calibration model is trained in advance based on historical annotation data, which includes the annotation categories, number of annotations, annotation pixel area, annotation connected component area, and annotation occlusion rate of multiple historical annotation regions.
[0158] In one possible implementation, the target counting device further includes a construction module for:
[0159] For each sample category, a preset number of sample images are obtained, and the sample images include multiple typical poses of the sample category;
[0160] Perform preprocessing operations on each sample image to obtain a sample-normalized image. The preprocessing operations include at least image denoising and size normalization.
[0161] The target region of each sample in the standardized image is determined by manually annotating bounding boxes, and the pixel area of each target region is calculated.
[0162] After removing outliers in pixel area based on the Grubbs criterion, the statistical mean of the other pixel areas is determined as the standard pixel area for the sample category.
[0163] A pre-defined feature extraction network is used to extract features from the target region of each sample to obtain the sample feature vector. The sample category, the sample feature vector, and the standard pixel area of the sample category are then associated to construct a category feature template library.
[0164] In one possible implementation, the computing module 304 is specifically used for:
[0165] Calculate the pixel area of the target region based on the target region;
[0166] The total pixel area of the target category is obtained by summing up the areas of all pixels corresponding to the target category.
[0167] Calculate the ratio of the total pixel area to the standard pixel area to obtain the initial number of target categories.
[0168] In one possible implementation, the second processing module 305 is specifically used for:
[0169] The occlusion rate of the target region is calculated based on the pixel area and connected component area of the target region. The connected component area is the area of the largest consecutive set of pixels containing the target region.
[0170] Determine the correction feature set based on the occlusion rate;
[0171] The correction feature set is input into the correction model to obtain the correction factor corresponding to the target category;
[0172] The final number of target categories is calculated based on the correction factor and the initial number of target categories.
[0173] In one possible implementation, the second processing module 305 is specifically used for:
[0174] If the occlusion rate does not exceed the preset occlusion rate, then the first feature set is determined as the correction feature set. The first feature set includes the camera tilt angle, target density, image brightness, and the percentage of pixels used for image edge detection.
[0175] If the occlusion rate exceeds the preset occlusion rate, then the second feature set is determined as the correction feature set. The second feature set includes the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the image grayscale variance.
[0176] In one possible implementation, the second processing module 305 is specifically used for:
[0177] Obtain historical labeled data;
[0178] Based on the annotation occlusion rate, the historical annotation data is divided into the first training set and the second training set;
[0179] For the first training set, the camera tilt angle, target density, image brightness, and the percentage of pixels used for image edge detection are extracted to construct the first training input features.
[0180] For the second training set, the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the gray-level variance of the image are extracted to construct the second training input features;
[0181] The historical correction factor is determined based on the number of annotations in the historical annotation data and the initial number calculated from the area of the annotated pixels and the area of the standard pixels.
[0182] The first and second training input features are mapped to their corresponding historical correction factors, respectively. The correction model is trained using supervised learning so that it can output the correction factor of the target category based on the input correction feature set.
[0183] In one possible implementation, the determining module 303 is specifically used for:
[0184] The cosine similarity between the target feature vector and the feature vectors of each sample in the preset category feature template library is calculated to obtain multiple similarity values.
[0185] Select the sample category corresponding to the highest similarity value from the similarity values as the candidate category;
[0186] When the maximum similarity is greater than or equal to the preset similarity threshold, the candidate category is determined as the target category corresponding to the target feature vector;
[0187] When the maximum similarity is less than the preset similarity threshold, the target feature vector is marked as an unknown category and reported.
[0188] The target counting device provided in this application embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0189] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the device 40 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.
[0190] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.
[0191] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0192] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0193] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0194] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0195] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.
[0196] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0197] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0198] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0199] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0200] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0201] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0202] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0203] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A target counting method, characterized in that, include: The target image to be counted is acquired, and semantic segmentation processing is performed on the target image to obtain multiple target regions; Feature extraction is performed on each target region to obtain multiple target feature vectors; The similarity of each target feature vector with a preset category feature template library is calculated to determine the target category corresponding to each target feature vector and the standard pixel area corresponding to the target category. The category feature template library stores the relationship between feature vectors, categories and standard pixel areas. For each target category, the initial number of the target category is calculated based on all the target regions corresponding to the target category and the standard pixel area; Based on the calibration model, the final number of target categories is calculated according to the target region and the occlusion rate. The calibration model is trained in advance based on historical annotation data, which includes the annotation categories, number of annotations, annotation pixel area, annotation connected component area, and annotation occlusion rate of multiple historical annotation regions.
2. The method according to claim 1, characterized in that, The construction process of the category feature template library includes: For each sample category, a preset number of sample images corresponding to the category are obtained, and the sample images include multiple typical poses of the sample category; Perform preprocessing operations on each of the sample images to obtain a sample standardized image. The preprocessing operations include at least image denoising and size normalization. The target region of each standardized image is determined by manually annotating bounding boxes, and the pixel area of each target region is calculated. After removing outliers in the pixel area based on the Grubbs criterion, the statistical mean of the other pixel areas is determined as the standard pixel area of the sample category. A preset feature extraction network is used to extract features from each of the target regions of the sample to obtain a sample feature vector. The sample category, the sample feature vector, and the standard pixel area of the sample category are then associated to construct a category feature template library.
3. The method according to claim 1, characterized in that, For each target category, the initial number of target categories is calculated based on all target regions corresponding to the target category and the standard pixel area, including: Calculate the pixel area of the target region based on the target region; The total pixel area of the target category is obtained by summing up the areas of all pixels corresponding to the target category. Calculate the ratio of the total pixel area to the standard pixel area to obtain the initial number of the target category.
4. The method according to claim 1, characterized in that, The step of calculating the final number of target categories based on the correction model, according to the target region and occlusion rate, includes: Based on the pixel area and connected component area of the target region, the occlusion rate of the target region is calculated, where the connected component area is the area of the largest consecutive set of pixels containing the target region. Determine the correction feature set based on the occlusion rate; The correction feature set is input into the correction model to obtain the correction factor corresponding to the target category; The final number of target categories is calculated based on the correction factor and the initial number of target categories.
5. The method according to claim 4, characterized in that, The step of determining the correction feature set based on the occlusion rate includes: If the occlusion rate does not exceed the preset occlusion rate, then the first feature set is determined as the correction feature set. The first feature set includes camera tilt angle, target density, image brightness, and the percentage of image edge detection pixels. If the occlusion rate exceeds the preset occlusion rate, then a second feature set is determined as the correction feature set. The second feature set includes the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the image grayscale variance.
6. The method according to claim 4 or 5, characterized in that, The training process of the correction model includes: Obtain historical labeled data; Based on the labeled occlusion rate, the historical labeled data is divided into a first training set and a second training set; For the first training set, the camera tilt angle, target density, image brightness, and the percentage of pixels detected at image edges are extracted to construct the first training input features; For the second training set, the number of connected components, the aspect ratio, perimeter, area of a single connected component, and the gray-level variance of the image are extracted to construct the second training input features; Based on the number of annotations in the historical annotation data and the initial number calculated from the area of the annotated pixels and the area of the standard pixels, a historical correction factor is determined. The first and second training input features are mapped to their corresponding historical correction factors, and the correction model is trained using supervised learning so that the correction model can output the correction factor of the target category based on the input correction feature set.
7. The method according to claim 1 or 2, characterized in that, The step of calculating the similarity between each target feature vector and a preset category feature template library to determine the target category corresponding to each target feature vector includes: The target feature vector is compared with the feature vectors of each sample in the preset category feature template library by cosine similarity calculation to obtain multiple similarity values; Select the sample category corresponding to the highest similarity value from the similarity values as the candidate category; When the maximum similarity is greater than or equal to a preset similarity threshold, the candidate category is determined as the target category corresponding to the target feature vector; When the maximum similarity is less than the preset similarity threshold, the target feature vector is marked as an unknown category and reported.
8. A target counting device, characterized in that, include: The first processing module is used to acquire the target image to be counted and perform semantic segmentation processing on the target image to obtain multiple target regions; The extraction module is used to extract features from each of the target regions to obtain multiple target feature vectors; The determination module is used to perform similarity calculation between each target feature vector and a preset category feature template library to determine the target category corresponding to each target feature vector and the standard pixel area corresponding to the target category. The category feature template library stores the association relationship between feature vectors, categories and standard pixel areas. The calculation module is used to calculate the initial number of target categories for each target category based on the area of all target regions corresponding to the target category and the area of the standard pixels. The second processing module is used to calculate the final number of target categories based on the calibration model, according to the target region and the occlusion rate. The calibration model is trained in advance based on historical annotation data, which includes the annotation categories, number of annotations, annotation pixel area, annotation connected component area, and annotation occlusion rate of multiple historical annotation regions.
9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.